I found the rough areas did different by model

e.g. Claude could not correct issues it introduced, it would sort of spiral through half refactors for an hour if unchecked.

Gemini Pro asked for way too much permission. "Should I do this?" Yes, go, do it! "Okay, should I now add my edits" Yes. "Okay I added edits, now we can run make test", okay run make Gemini requests to run make ugh it's worse than a bad intern.

Gemini would also frequently hallucinate APIs because I used a non-standard api for my hash table (allocate/dereference instead of get/set/update, which I find is more natural for managing ownership). And Gemini would rewrite my code style and order of operations for no reason (eg move a counter increment before updating another field).

At no point could I just point the model at a small problem unsupervised. Even "update the test suite for 100% coverage of this module, make sure the tests are as small in scope as possible" had highly mixed results.

And all models I tried would update my cmakelists and break it, and I hate dealing with cmake.

I've been told the new Gemini is good at SQL and programming, but I'm underwhelmed on both. Gemini frequently doesn't even know all the BigQuery functions, which being integrated into BigQuery Studio it should.

They're decent at code review, but a language server is still better at catching bugs.