11

Do i need industry grade gpu's or can i scrape by getring decent tps with a consumer level gpu.

you are viewing a single comment's thread
view the rest of the comments
[-] red@lemmy.zip 1 points 2 hours ago

this is useless, llama.cpp already does that airllm does (offloading to CPU) but its actually faster. so just use ollama

this post was submitted on 10 Jan 2025
11 points (92.3% liked)

LocalLLaMA

2410 readers
34 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago
MODERATORS