73
you are viewing a single comment's thread
view the rest of the comments
[-] zongor@hexbear.net 6 points 21 hours ago* (last edited 21 hours ago)

Yes it’s called quantization; it’s like zip file for a LLM model. You can get it small enough to run on a raspberry pi (like 5 amps) and although there is loss in “intelligence” it is still useable for a lot of scenarios. Look up ollama or llama.cpp for details

this post was submitted on 15 Aug 2025
73 points (100.0% liked)

technology

23912 readers
205 users here now

On the road to fully automated luxury gay space communism.

Spreading Linux propaganda since 2020

Rules:

founded 5 years ago
MODERATORS