20
1U mini PC for AI? (startrek.website)
submitted 4 weeks ago by nagaram@startrek.website to c/selfhosted@lemmy.world
22 comments fedilink hide all child comments

My rack is finished for now (because I'm out of money).

Last time I posted I had some jank cables going through the rack and now we're using patch panels with color coordinated cables!

But as is tradition, I'm thinking about upgrades and I'm looking at that 1U filler panel. A mini PC with a 5060ti 16gb or maybe a 5070 12gb would be pretty sick to move my AI slop generating into my tiny rack.

I'm also thinking about the PI cluster at the top. Currently that's running a Kubernetes cluster that I'm trying to learn on. They're all PI4 4GB, so I was going to start replacing them with PI5 8/16GB. Would those be better price/performance for mostly coding tasks? Or maybe a discord bot for shitposting.

Thoughts? MiniPC recs? Wanna bully me for using AI? Please do!

top 22 comments
sorted by: hot top controversial new old
[-] GirthBrooks@lemmy.world 5 points 4 weeks ago

Looking good! Funny I happen across this post when I’m working on mine as well. As I type this I’m playing with a little 1.5” transparent OLED that will poke out of the rack beside each pi, scrolling various info (cpu load/temp, IP, LAN traffic, node role, etc)

[-] ripcord@lemmy.world 2 points 4 weeks ago

What OLED specifically and what will you be using to drive it?

[-] TropicalDingdong@lemmy.world 2 points 4 weeks ago

This is so pretty 😍🤩💦!!

I've been considering a micro rack to support the journey, but primarily for house old laptop chassis as I convert them into proxmox resources.

Any thoughts or comments on you choice of this rack?

[-] nagaram@startrek.website 1 points 4 weeks ago

Not really a lot of thought went into rack choice. I wanted something smaller and more powerful than my several optiplexs I had.

I also decided I didn't want storage to happen here anymore because I am stupid and only knew how to pass through disks for Truenas. So I had 4 truenas servers on my network and I hated it.

This was just what I wanted at a price I was good with at Like $120. There's a 3D printable version but I wasn't interested in that. I do want to 3D print racks and I want to make my own custom ones for the Pis to save space.

But this set up is way cheaper if you have a printer and some patience.

[-] InternetCitizen2@lemmy.world 1 points 4 weeks ago

NSFW

[-] ZeDoTelhado@lemmy.world 1 points 4 weeks ago* (last edited 4 weeks ago)

I have a question about ai usage on this: how do you do this? Every time I see ai usage some sort of 4090 or 5090 is mentioned, so I am curious what kind of ai usage you can do here

[-] Diplomjodler3@lemmy.world 1 points 4 weeks ago

I'm afraid I'm going to have to deduct one style point for the misalignment of the labels on the mini PCs.

[-] nagaram@startrek.website 0 points 4 weeks ago

That's fair and justified. I have the label maker right now in my hands. I can fix this at any moment and yet I choose not to.

I'm man feeding orphans to the orphan crushing machine. I can stop this at any moment.

[-] Diplomjodler3@lemmy.world 1 points 4 weeks ago

The machine must keep running!

[-] Colloidal@programming.dev 1 points 4 weeks ago

You could combine both 1U fillers and install a 2U PC, which would be easier to find.

[-] nagaram@startrek.website -1 points 4 weeks ago

I was thinking about that now that I have Mac Minis on the mind. I might even just set a mac mini on top next to the modem.

[-] hendrik@palaver.p3x.de 1 points 4 weeks ago

Well, I always advocate for using the stuff you have. I don't think a Discord bot needs four new RasPi 5. That's likely to run on a single RasPi3. And as long as they're sitting idle, it doesn't really matter which model number they have... So go ahead and put something on your hardware, and buy new one once you've maxed out your current setup.

I'm not educated on Bazzite. Maybe tools like Distrobox or other container solutions can help running AI workloads on the gaming rig. It's likely easier to run a dedicated AI server, but I started learning about quantization, tested some models on my main computer with the help of ollama, KoboldCPP and some random Docker/Podman containers. I'm not saying this is the preferrable solution. But definitely enough to get started with AI. And you can always connect the computers within your local network, write some server applications and have them hook into ollama's API and it doesn't really matter whether that runs on your gaming pc or a server (as long as the computer in question is turned on...)

[-] nagaram@startrek.website 1 points 4 weeks ago

Ollama and all that runs on it its just the firewall rules and opening it up to my network that's the issue.

I cannot get ufw, iptables, or anything like that running on it. So I usually just ssh into the PC and do a CLI only interaction. Which is mostly fine.

I want to use OpenWebUI so I can feed it notes and books as context, but I need the API which isn't open on my network.

[-] Korhaka@sopuli.xyz 1 points 4 weeks ago

Ohh nice, I want it. Don't really know what I would use all of it for, but I want it (but don't want to pay for it).

Currently been thinking of getting an N150 mini PC. Setup proxmox and a few VMs. At the very least pihole, location to dump some backups and also got a web server for a few projects.

[-] Melvin_Ferd@lemmy.world 1 points 4 weeks ago

The AI hate is overwhelming at times. This is great. What kind of things are you doing with it?

[-] umbrella@lemmy.ml 1 points 4 weeks ago* (last edited 4 weeks ago)

for your usecase, i'd get an external gpu to plug in to one of these juicy thinkstations right there. bonus for the modularity of having an actual gpu instead of relying on whatever crappy laptop gpu minipc manufacturers put in there.

you could probably virtualize a sick gaming setup with this rig too. stream it to your phone/laptop.

nice setup btw.

[-] brucethemoose@lemmy.world -1 points 4 weeks ago* (last edited 4 weeks ago)

If you can swing $2K, get one of the new mini PCs with an AMD 395 and 64GB+ RAM (ideally 128GB).

They're tiny, lower power, and the absolute best way to run the new MoEs like Qwen3 or GLM Air for coding. TBH they would blow a 5060 TI out of the water, as having a ~100GB VRAM pool is a total game changer.

I would kill for one on an ITX mobo with an x8 slot.

[-] princessnorah@lemmy.blahaj.zone 1 points 4 weeks ago

I think the mainboard from the Framework Desktop meets your requirements: https://frame.work/au/en/products/framework-desktop-mainboard-amd-ryzen-ai-max-300-series?v=FRAFMK0002

[-] MalReynolds@piefed.social 0 points 4 weeks ago

Pretty sure that's a x4 PCIe slot (admittedly PCIe 5x4, but not many video cards speak PCIe5), would totally trade a usb4 for a x8, but these laptop chips are pretty constrained lanes wise.

[-] brucethemoose@lemmy.world 0 points 4 weeks ago* (last edited 4 weeks ago)

It's PCIe 4.0 :(

but these laptop chips are pretty constrained lanes wise

Indeed. I read Strix Halo only has 16 4.0 PCIe lanes in addition to its USB4, which is resonable given this isn't supposed to be paired with discrete graphics. But I'd happily trade an NVMe slot (still leaving one) for x8.

One of the links to a CCD could theoretically be wired to a GPU, right? Kinda like how EPYC can switch its IO between infinity fabric for 2P servers, and extra PCIe in 1P configurations. But I doubt we'll ever see such a product.

[-] MalReynolds@piefed.social 0 points 4 weeks ago

It's PCIe 4.0 :(

Boo! Silly me thinking DDR5 implied PCIe5, what a shame.

Feels like they're testing the waters with Halo, hopefully a loud 'waters great, dive in' signal gets through and we get something a bit fitter for desktop use, maybe with more memory (and bandwidth) next gen. Still, gotta love the power usage, makes for one hell of a NAS / AI inference server (and inference isn't that fussy about PCIe bandwidth, hell eGPU works fine as long as the model / expert fits in VRAM.

[-] brucethemoose@lemmy.world 1 points 4 weeks ago* (last edited 4 weeks ago)

Rumor is it’s successor is 384 bit, and after that their designs are even more modular:

https://www.techpowerup.com/340372/amds-next-gen-udna-four-die-sizes-one-potential-96-cu-flagship

Hybrid inference prompt processing actually is pretty sensitive to PCIe bandwidth, unfortunately, but again I don’t think many people intend on hanging an AMD GPU off these Strix Halo boards, lol.

this post was submitted on 30 Aug 2025
20 points (88.5% liked)

Selfhosted

51364 readers
8 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz