80
what the fuck is wrong with prices of memory
(hexbear.net)
On the road to fully automated luxury gay space communism.
Spreading Linux propaganda since 2020
Rules:
You need a gpu. Anything nvidia is fine, more ram is better, but you can use system ram to swap out what you’re doing.
If you’re doing it yourself, consider how smaller models built to do one specific thing can do the job. For example: a small 8gb video card can do text inference and its results can be sent to something like kokoro on cpu for tts and you suddenly have a talking llm on an eight year old budget gpu.