It's good for me because I'm piss poor at programming. In my defense, I'm not a programmer or even programmer adjacent. I do see how it wouldn't be useful to a pro. It also has occasionally given me garbage advice that an expert would spot right away while I had to figure out in my own that it was 'hallucinating' again. There's nothing better for learning than troubleshooting, though!

There's a project called Tabby that your can host as a server on a machine that has a GPU, and has a VSCode extension that connects to the server.

The default model is called starcoder, and it's the small version, 1B parameters. The downside is that it's not super smart (but still an improvement over built in tools), but since it's such a small model, I'm getting sub-second processing times.

I've found it's pretty good for translating between steps so to speak.

Converted some bash to python relatively quickly by giving it snippets and fixing errors as it made them.

I also had success generating an ansible playbook based on my own previously written install instructions for SillyTavern and llama.cpp.

I could do both of those tasks myself, but thar would be more difficult than having a mostly correct translation and fixing some errors.

You should make sure you are running a model that fits in your vram, for me it runs faster than any online LLM I’ve tried.