196
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 14 Dec 2025
196 points (99.0% liked)
Technology
84324 readers
213 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Aside: WTF are they using SSDs for?
LLM inference in the cloud is basically only done in VRAM. Rarely stale K/V cache is cached in RAM, but new attention architectures should minimize that. Large scale training, contrary to popular belief, is a pretty rare event most data centers and businesses are incapable of.
…So what do they do with so much flash storage!? Is it literally just FOMO server buying?
Storage. There aren't enough hard drives, so datacentres are also buying up SSDs, since it's needed to store training data.
Again, I don’t buy this. The training data isn’t actually that big, nor is training done on such a huge scale so frequently.