I’m not sure I’d say it’s plateaued today but I definitely think machine learning is going to hit a wall soon. Some tech keeps improving until physical limits stop progress but I see generative AI as being more like self-driving cars where the “easy” parts end up solved but the last 10% is insanely hard.

There’s also the economic reality of scaling. Maybe the “hard” problems could, in theory, be easily solved with enough compute power. We’ll eventually solve those problems but it’s going to be on Nvidia’s timeline, not OpenAI’s.

Let me save you a click: he doesn’t say anything interesting about why he thinks this.

What does Bill Gates know about GenAI? Is he an expert on the subject?

Cool, Bill Gates has opinions. I think he's being hasty and speaking out of turn and only partially correct. From my understanding, the "big innovation" of GPT-4 was adding more parameters and scaling up compute. The core algorithms are generally agreed to be mostly the same from earlier versions (not that we know for sure since OpenAI has only released a technical report). Based on that, the real limit on this technology is compute and number of parameters (as boring as that is), and so he's right that the algorithm design may have plateaued. However, we really don't know what will happen if truly monster rigs with tens-of-trillions of parameters are used when trained on the entirety of human written knowledge (morality of that notwithstanding), and that's where he's wrong.

I hope so. Theyve already got scary implications for creative parts of the economy.

That said, we're in the Cambrian explosion of the tech. As it plateaus, the next step will be enhanced tooling and convenience around it. Better inputs than just text, better, more applications in new spaces, etc.

"GPT-4 should be enough for anyone." -Bill Gates

bill is a wanker. dont be like bill.

Not a single comment yet stating how Gates is a great human being because of his foundation, and how all you haters should fuck the fuck off? sigh, let me the first one.

But we have more areas to apply this to. I still can't ask my PC to do some work, like Unistall OneDrive or change a setting in the OS. Send a message on Teams. Where is Jarvis?

On the one hand, I don't really know enough about AI to comment. What I do remember is that, Bill Gates said the Internet was just a fad in the 90s. This comment caused myself and others problems promoting the Internet in workplaces because those in charge for some reason put some weight to his words. :p

I've been saying this for years!