This sounds like good engineering, but surely there's not a big gap with their competitors. They are spending tens of millions on hardware and energy, and this is something a handful of (very good) programmers should be able to pull off.
Unless I'm missing something, It's the sort of thing that's done all the time on console games.
I think more like was done all the time for console games. These days that doesn't happen as much anymore as far as I know. But I think this shows that CUDA is not a good enough abstraction for modern GPUs or the compilers are not as good as expected. There should be no way they got that much optimization out of hand written/optimized code these days.
What is amazing in this case is that they achieved spending a fraction of the inference cost that OpenAI is paying.
Plus they are a lot cheaper too. But I am pretty sure that the American government will ban them in no time, citing national security concerns, etc.
Nevertheless, I think we need more open source models.
Not to mention that NVIDIA also needs to be brought to earth.
Even if they get banned, any startup could replicate their work if it is truly open source. The best thing about their solution is that it breaks the CUDA monopoly that NVDA has enjoyed. Buy your puts when NVDA bounces because that stock is GOING DOWN. There’s no world where a company that makes GPU’s is worth more than both Apple and Microsoft. It’s inevitable.
Never forget kids the market can stay irrational much longer than you can stay solvent.
True. Thats why I tend to make small plays instead of being an absolute degenerate gambler.
I wish that was true, but this doesn't threaten any monopoly
~~It certainly does.~~
~~Until last week, you absolutely NEEDED an NVidia GPU equipped with CUDA to run all AI models.~~
~~Today, that is simply not true. (watch the video at the end of this comment)~~
~~I watched this video and my initial reaction to this news was validated and then some: this video made me even more bearish on NVDA.~~
Edit: corrected and redacted.
Until last week, you absolutely NEEDED an NVidia GPU equipped with CUDA to run all AI models.
Thanks for the corrections.
mate, that means they are using PTX directly. If anything, they are more dependent to NVIDIA and the CUDA platform than anyone else.
to simplify: they are bypassing the CUDA API, not the NVIDIA instruction set architecture and not CUDA as a platform.
Ahh. Thanks for this insight.
There seems to be some confusion here on what PTX is -- it does not bypass the CUDA platform at all. Nor does this diminish NVIDIA's monopoly here. CUDA is a programming environment for NVIDIA GPUs, but many say CUDA to mean the C/C++ extension in CUDA (CUDA can be thought of as a C/C++ dialect here.) PTX is NVIDIA specific, and sits at a similar level as LLVM's IR. If anything, DeepSeek is more dependent on NVIDIA than everyone else, since PTX is tightly dependent on their specific GPUs. Things like ZLUDA (effort to run CUDA code on AMD GPUs) won't work. This is not a feel good story here.
I don't think anyone is saying CUDA as in the platform, but as in the API for higher level languages like C and C++.
PTX is a close-to-metal ISA that exposes the GPU as a data-parallel computing device and, therefore, allows fine-grained optimizations, such as register allocation and thread/warp-level adjustments, something that CUDA C/C++ and other languages cannot enable.
Some commenters on this post are clearly not aware of PTX being a part of the CUDA environment. If you know this, you aren't who I'm trying to inform.
aah I see them now
This is why Nvidia stock has been hit so hard. CUDA is their moat
Aw, CUDA see this happening...
Reminds me of the Bitcoin mining and how askii miners overtook graphic card mining practically overnight. It would not surprise me if this goes the same way.
They said this is close to metal. Wake me up when they've achieved metal.
This is a most excellent place for technology news and articles.
