There are many excellent options - far too many to list. So I will briefly say - there are some really nice 4B models (like Qwen3-4B HIVEMIND, Nanbeige, IBM Granite 3B) which you should be able to run at higher quants (Q6 and up) quite nicely. Of course, there are always newer models (Gemma, Qwen3.6 - soon 3.7) etc.

Best bet is to poke around hugging face, on TheBloke, Unsloth or DavidAUs archives and see what they have in the 3-7B range that tickles your fancy. Don't immediately jump for the newest releases - the old ones are still good. Qwen3-4B 2507 instruct is still a favourite of mine and more recently Qwen3.5-2B shows promise.

My setup is a laptop with 8 GB vram and 16 gb ram.

I have been using ministral 3b (fast) and 14b (slower but somewhat smarter/capable) via ollama. They work remarkably well considering how small they are.

I have been using it as a text translator, summarizer and assistant for discussing more basic things, including integrating it in pycharm using the ollama assist plugin as a coding assistant.

For autocomplete in pycharm I have to use llama 3.1 8b, since ministral cannot do autocomplete (?).

I can recommend ministral, Mistral are really great at creating small distilled models that have a lot of bang for the parameters they have.

I'm running gemma-4-e4b on my 8GB machine. I'll drop down to e2b on CPU. It's probably the best you'll get. 140 languages, vision, decent at agentic work. Not great at code.

I mean, do you need it to be fast? You could probably run a pretty decent 20b model if you are okay with the speed of offloading.

On my MacBook Air m2, I’m currently using Qwen 3.5 4b with 8 bit quantisation, and even at its maximum context length, multiple web search RAGs, and the model being built for vision and reasoning, it only ever hits 4.3gb of memory tops.

I run it though LM Studio, so paired with the fact it’s a Mac, your mileage may vary in terms of how much memory it uses, but it does have [from my experience] an output quality a bit over ChatGPT 4o, and is actually really solid for research purposes if that’s what you’re looking for.

I’m running Qwen 3.5 4B Q4 on 16GB RAM. Yup, no VRAM haha. 5-6tk/s. Llama.cpp

I have been using qwen-3.5-9b as a general purpose LLM that I can still load up while gaming (on a 16GB card). I never have issues if I'm under 10ish gigs of VRAM usage for the game, so I imagine it should work for your use case.

I've been generally happy with the results on everyday reasoning tasks and programming questions.

Check out unsloth studio. It runs Qwen model local, exposing an endpoint, and I’ve had great success with it.