19
OpenAI hits back at DeepSeek with o3-mini reasoning model
(arstechnica.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 02 Feb 2025
19 points (78.8% liked)
Technology
63023 readers
759 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Qwen 2.5 is already amazing for a 14B, so I don’t see how deepseek can improve that much with a new base model, even if they continue train it.
Perhaps we need to meet in the middle, and have quad channel APUs like Strix Halo become more common, and maybe release like 40-80GB MoE models. Perhaps bitnet ones?
Or design them for asynchronous inference.
I just don’t see how 20B-ish models can perform like one orders of magnitude bigger without a paradigm shift.
Intriguingly, there's reason to believe the R1 distills are nowhere close to their peak performance. In the R1 paper they say that the models are released as proofs of concept of the power of distillation, and the performance can probably be improved by doing an additional reinforcement learning step (like what was done to turn V3 into R1). But they said they basically couldn't be bothered to do it and are leaving it for the community to try.
2025 is going to be very interesting in this space.
I use 14b and it's certainly great for my modest highschool physics and python (to help the kids) needs, but for party games and such it's a drag its pop culture stops at mid 2023
Thing is, there are a lot of free APIs for 30B-70B class models.
Self hosting is great of course, and if 14B does the job then it does the job.