69
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 23 Apr 2026
69 points (97.3% liked)
Asklemmy
54075 readers
564 users here now
A loosely moderated place to ask open-ended questions
Search asklemmy ๐
If your post meets the following criteria, it's welcome here!
- Open-ended question
- Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
- Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
- Not ad nauseam inducing: please make sure it is a question that would be new to most members
- An actual topic of discussion
Looking for support?
Looking for a community?
- Lemmyverse: community search
- sub.rehab: maps old subreddits to fediverse options, marks official as such
- !lemmy411@lemmy.ca: a community for finding communities
~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~
founded 7 years ago
MODERATORS
I think by the time AI becomes efficient enough to be profitable, it's going to be efficient enough to run locally and the whole AI as a service business model is going to collapse. We're basically in the mainframe era of AI right now, and we've seen this happen with many technologies before. There's no reason to think this case will be different.
Just to give you an idea of how fast this stuff is moving. Qwen 3.6 was just released and can be run on a high end laptop, it outperforms Qwen 3.5 from February which required a commercial grade server to run. https://qwen.ai/blog?id=qwen3.6-27b
Not even the end of Moore's Law?
I'm not sure if you're aware, but processors aren't really getting much more efficient anymore. They're just getting bigger (more parallel), which is why the price for the newer generations of GPUs has been skyrocketing. A new top-end GPU costs twice as much (or more) as a previous-gen one because it has twice as many (or more) compute units, since they can't make the individual compute units much faster due to fundamental laws of physics.
I expect that software will continue to get optimized, and we'll see new algorithms that are more efficient than what people are doing currently. However, it's possible we'll start seeing hardware specifically built for models as well. For example, there's already a startup that uses ASIC chips to print the model directly to the chip. Since each transistor acts as a state, it doesn't need DRAM and the whole chip requires a small amount of SRAM which isn't in short supply right now https://www.anuragk.com/blog/posts/Taalas.html
The limitation with this approach is that the chip is made for a specific model, but that's not really that different from the way regular chips work either. You buy a chip and if it does what you need, it keeps working. When new models come out, new chips get printed, and if you need the new capabilities then you upgrade.
You can see how absurdly fast their hardware version of llama 3 is here https://chatjimmy.ai/
That is indeed absurdly fast.
There's always two sides to software, one is the power of the hardware, and the other is the efficiency of the software. I think in this case OP means that AI will be optimized so much that it will require tiny fractions of the resources it previously needed, at least for the casual use cases of an average person asking a simple question or performing a small task.
i suspect that we've neared the end of what we can get out of using silicon and the only way forward, at this point, is to switch materials altogether into something like graphene or carbon; but i bet it would take a long time to ever do that because the profit motives that keeps on silicon won't allow for it.
There are a few different tracks here. One is software optimizations where models require less energy to use. That's been moving really fast over the past few years, and there are still a lot of papers that haven't been integrated into production systems that are really promising.
Another track is hardware architecture where the substrate stays the same, but chip design improves. A general example of this is SoC architecture like M series from Apple of Kirin 9000 from Huawei. The architecture eliminates the memory bus which is one of the main bottlenecks, and RISC instruction set facilitates parallelism much better than SISC. A more specific example would be ASIC chips like what Taalas is making which print the model directly on the chip.
And the last track is the one you mention with using a more efficient substrate. Notably this will directly benefit from the other two tracks as well. Whatever software and hardware architecture improvements people come up with, will directly apply to chips made out of graphene or other materials.
Agreed and all of those w tracks to squeeze out as much as we can from silicon.
There's a limit that we haven't yet reached but we will eventually because of those profits.
I bet that China will be the first to reach it since they're willing to spend so much on all infrastructure.
I expect so as well, and China also has a lot of incentive to invest in alternative substrates since they're behind on silicon. If one of these moonshot projects they're pursuing delivers that would make current silicon chips look like vacuum tubes by comparison.
From a basic physics research perspective (as opposed to an engineering process development for production perspective), are we even sure graphene semiconductors have that much potential headroom for improvement beyond the best possible silicon ones? I'm not convinced it buys us more than a couple of process nodes. I mean, we're already making transistors so small you can damn near count the individual atoms in them today. Is making them out of atoms with one less valence level gonna be enough for a 10x, 100x, or 1000x improvement, even in the long run?
The Chinese will likely be the first ones to know for certain considering that they've already demonstrated a willingness to spend a metric fuck ton into public infrastructure like the United States used to do for its military.
Yup. Already working on a suite of local pipeline apps and an orchestration platform for this. Happy to share if interested! Source
That's very cool!
you should patent this!
And thereby lock it away from the underserved communities that need it most? Naw. Open source publishing is the way forward for a truly egalitarian system, which is what I'm aiming for
I was thinking of insulin.
The person who created it likewise refused to protect it from profit motives because he also felt that it belonged to humanity and it became captured to the detriment of humanity as a result.
Ah well the trouble is software patents can cost upwards of 5 figures, so yeah if I start making money I might do that, but it's definitely not within my capacity for now, thus public publishing for copyright establishment
Maybe you could crowdfund it!