LinkedIn Hidden Code Secretly Searches Your Browser for Installed Extensions by bazinga in c/cybersecurity@sh.itjust.works
[-] bazinga@discuss.tchncs.de 6 points 1 day ago

I am linkedout, I don't care anymore.

JD Vance claims US is not interfering in Hungary election by bazinga in c/news@beehaw.org
[-] bazinga@discuss.tchncs.de 5 points 2 days ago

Hope he looses. Don't give them a platform

Vance arrives in Hungary to back Viktor Orban’s re-election bid by bazinga in c/usa@lemmy.ml
[-] bazinga@discuss.tchncs.de 0 points 3 days ago

Hope he looses

Which Llama Server Hardware do you use? by bazinga in c/selfhosted@lemmy.world
[-] bazinga@discuss.tchncs.de 1 points 1 week ago

Thanks, will also ask in the other group you mentioned. I am still having a gaming rig here with rx6900xt as well but way too big to get it wife approved into the living room and have no man cave to run it 24/7. ;) But maybe good for testing what I actually need in model size, I think it is just 1 generation before all the ai hype took off but going to try now right away.

Which Llama Server Hardware do you use? by bazinga in c/selfhosted@lemmy.world
[-] bazinga@discuss.tchncs.de 2 points 1 week ago

Thank you for the detailed writeup. Are you aware of anything small footprint? I am thinking like dgx spark size maybe a bit bigger?

7
Which Llama Server Hardware do you use? (discuss.tchncs.de)
submitted 1 week ago* (last edited 1 week ago) by bazinga@discuss.tchncs.de to c/selfhosted@lemmy.world
11 comments fedilink

I realize, I need to upgrade my little NUC to something bigger for higher inference of bigger llama models. I want something that you still can have on your living room's tv bench, so no monster rack please, but that has also the necessary muscle when needed for llama. Budget doesn't matter right now, want to understand what's good and what's out there. Thanks

EDIT: Wow, thanks for the inspiration, guess I need to look at bit for "how to stuff a huge graphics card into a mini box". To clarify a bit more what I want with it: I want to build a responsive personal assistant. I am dreaming of models bigger than 8B, good tool calling for things like memory, websearch etc., no coding, no image generation, no video generation required. Image recognition would be good but not a must. Regarding footprint, the no monster ;) Something that you can have in your livingroom, and could be wife approved - so no big gaming rig with exhaust pipes and stuff, needs to be good looking ;)

What device are you guys hosting on? by bazinga in c/homeassistant@lemmy.world
[-] bazinga@discuss.tchncs.de 2 points 1 week ago

NUC i7, 32gb ram. Full Docker stack includes home assistant and all relevant containers for it zigbee2mqtt, esphome, vscodium, sqliteweb, rtl_433, mqtt, kokoro_tts, weaviate, zwavejs, openwebui, ollama, paperless with ai capabilities, a1111, whisper, sync thing, searxng, redis, qdrant, postgres. Runs fine, however ollama starts to max it out as I want to go for bigger models, so looking for something with serious gpu oomphbut still small footprint and low power consumption

Self-Hosted Offline EAS Alerts Over Meshtastic with RTL-SDR by bazinga in c/selfhosted@lemmy.world

bazinga

joined 3 weeks ago