Interesting, that somewhat sounds like the pain I had with the og google assistant where I asked it to turn something on and it would just proceed to either turn it off or tell me something different entirely. Though that only started happening to me more over the end of last year where I guess google was just trying to kill it.

For Gemini I installed the standalone app that's independent of the Google assistant branded app (Gemini was/is still not available inside the assistant app here).

I have been having more success with it where I can ask it longer commands, such as "turn off kitchen lights with hallway lights, and turn on living room lights with the living room shield".

This kind of command would pretty much end with a "I do not understand" on og google assistant even with a turn of two lights command.

Home control is pretty much the only thing I use Gemini/Google Assistant for. I have been slowly trying to make the switch to a fully local voice with Home Assistant.