47
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 20 May 2024
47 points (92.7% liked)
Hardware
5035 readers
1 users here now
This is a community dedicated to the hardware aspect of technology, from PC parts, to gadgets, to servers, to industrial control equipment, to semiconductors.
Rules:
- Posts must be relevant to electronic hardware
- No NSFW content
- No hate speech, bigotry, etc
founded 4 years ago
MODERATORS
What use cases are you planning to use the NPU for?
Similar use cases to what I'm doing right now, running LLMs like Mixtral8x7B (or something better by the time we start seeing these), Whisper (STT), or Stable Diffusion.
I use a fine tuned version of Mixtral (dolphin-Mixtral) for coding purposes.
Transcribing live audio for notes/search, or translating audio from different languages using Whisper (especially useful for verifying claims of translations for Russian/Ukrainian/Hebrew/Arabic especially with all of the fake information being thrown around).
Combine the 2 models above with a text to speech system (TTS), a vision model like LLaVA and some animatronics and then I'll have my own personal GLaDOS: https://github.com/dnhkng/GlaDOS
And then there's Stable Diffusion for generating images for DnD recaps, concept art, or even just avatar images.
Thank you! I currently use my 3080 dGPU for Stable Diffusion. I wonder to what extent NPUs will be usable with Stable Diffusion XL.