Honestly it probably just needs a label. It can be debated either way.

The clearest dark mode/light mode switchers have an icon for all options and the current state is highlighted (e.g. https://gwern.net/). Explicitness is simplest.