repeat after me: LLMs are not AI.
LLMs are one version of AI. It's just one tiny part of AIs that are used every day, from chess bots to voice transcription, but they also are AI.
I would replace the word version with aspect. LLMs are merely one part of the puzzle that would be AI. Essentially what's been constructed is the mouth and the part of the brain that can form words but without any of the reasoning or intelligence behind what the mouth says.
The same goes for the art AIs. They can paint pictures based on input but they can't reason how those pictures should look. Which is why it requires so much tweaking to get them to output something that doesn't look like it came out of a Lovecraft novel.
I think you are confusing AI with AGI.
https://en.m.wikipedia.org/wiki/Artificial_general_intelligence
Not at all. AI is something that uses rules, not statistical guesswork. A simple control loop is alreadu basic AI, but the core mechanism of LLMs is not (the parts before and after token association/prediction are). Don't fall for marketing bullshit of some dumbass silicon valley snake oil vendors.
It's absurd that some of the larger LLMs now use hundreds of billions of parameters (e.g. llama3.1 with 405B).
This doesn't really seem like a smart usage of ressources if you need several of the largest GPUs available to even run one conversation.
I wonder how many GPUs my brain is
It's a lot. Like a lot a lot. GPUs have about 150 billion transistors but those transistors only make 1 connection in what is essentially printed in a 2d space on silicon.
Each neuron makes dozens of connections, and there's on the order of almost 100 billion neurons in a blobby lump of fat and neurons that takes up 3d space. And then combine the fact that multiple neurons in patterns firing is how everything actually functions and you have such absurdly high number of potential for how powerful human brains are.
At this point, I'm not sure there's enough gpus in the world to mimic what a human brain can do.
That's also just the electrical portion of our mind. There are whole levels of chemical, and chemical potentials at work. Neurones will fire differently depending on the chemical soup around them. Most of our moods are chemically based. E.g. adrenaline and testosterone making us more aggressive.
Our mind also extends out of our heads. Organ transplant recipricants have noted personality changes. Food preferences being the most prevailant.
The neurons only deal with 'fast' thinking. 'slow' thinking is far more complex and distributed.
I don't think your brain can be reasonably compared with an LLM, just like it can't be compared with a calculator.
LLMs are based on neural networks which are a massively simplified model of how our brain works. So you kind of can as long as you keep in mind they are orders of magnitude more simple.
At some point it becomes so “simplified” it’s arguably just not the same thing, even conceptually.
Larger models train faster (need less compute), for reasons not fully understood. These large models can then be used as teachers to train smaller models more efficiently. I've used Qwen 14B (14 billion parameters, quantized to 6-bit integers), and it's not too much worse than these very large models.
Lately, I've been thinking of LLMs as lossy text/idea compression with content-addressable memory. And 10.5GB is pretty good compression for all the "knowledge" they seem to retain.
That's capitalism
Seeing as how the full unquantized FP16 for Llama 3.1 405B requires around a terabyte of VRAM (16 bits per parameter + context), I'd say way more than several.
It's pretty obvious that they will hit a ceiling.
Quick buck is over. And now it's time again for base research to create better approach.
I really wish we had a really advanced AI with reasonable resource consumption within my lifetime. I don't think it's unreasonable as we have got really far in the last 30 years of computational technology.
I really wish we had a really advanced AI with reasonable resource consumption within my lifetime.
You only wish that for as long as it doesn't happen. Have you looked at the world we live in? Such tools would be controlled by the same billionaire dipshits for their personal gain as all social media is being used already.
We've come a long way in computing, but the computational power difference between a human brain and a computer is significant. LLMs were just a smart way to have computers learn pattern recognition. While important, it isn't anything close to artificial general intelligence (AGI), which is what the term AI usually means.
Yeah. AI may grind for a while but hardly anyone has put the current stuff to work, yet. We will be feeling the benefits of what is released right now for a decade to come. I am working on a very rudimentary application that will use ML at work and it won't come out for 12 more months, and it hardly does anything but make the most obvious decisions 10m times faster than I can. But it's going to fundamentally change our labor model.
There are regular folks applying amazing technologies that go way beyond content generation.
The tech may grind but the application of that tech is barely getting its feet and should run hard for a decade.
The problem isn't with the AI. It's with how it's being treated. It's currently being sold as if it were general intelligence. Which it's not. It should instead be treated like it's a mindless tool. Something that is inert on its own. Useful for some things but only in a limited sense. Unfortunately the companies, who have spent millions of dollars developing these things, are trying to sell it as the "do-all" artificial intelligence that people have grown up seeing in sci-fi media. Which it 100% is not.
Every company have always oversell their own products. This is not new.
Coca Cola is also just a carbonated sweet drink and it's being sold as happiness, socialization and the meaning of Christmas in a bottle.
Companies oversell, it's called marketing. It's shit practice but it's not nothing new.
That does not make the technology worse (or better). Current AI technology has its uses. With a big problem in how resource hungry it is. But it's fairly useful.
I understand folks don't like AI but this "article" is like a reddit post with lots of links to subjects which are vague and need the link text to tell us what is important, instead of relying on the actual article.
What the fuck you aren't kidding. I have comment replies to trolls that are longer than that article. The over the top citations also makes me think this was entirely written by an actual AI bot that was lrompted to supply x amoint of sources in their article. Lol
So long and thanks for all the fish habitat?
A 4 paragraph "article" lol
Are you suggesting “pivot-to-ai.com” isn’t the pinnacle of journalism?
They might be right but I read some of the linked articles on this blog (?), the authors just come off as not really knowing much about current AI technologies, and at the same time very very arrogant.
It's a known problem - though of course, because these companies are trying to push AI into everything and oversell it to build hype and please investors, they usually try to avoid recognizing its limitations.
Frankly I think that now they should focus on making these models smaller and more efficient instead of just throwing more compute at the wall, and actually train them to completion so they'll generalize properly and be more useful.
Technology
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.