That's a good text. I've been comparing those "LLM smurt!" crowds with Christian evangelists, due to their common usage of fallacies like inversion of burden of proof, changing goalposts, straw man, etc.

However it seems that people who believe in psychics might be a more accurate comparison.

That said LLMs are great tools to retrieve info when you aren't too concerned about accuracy, or when you can check the accuracy yourself. For example the ChatGPT output of prompts like

• "Give me a few [language] words that can be used to translate the [language] word [word]"
• "[Decline|Conjugate] the [language] word [word]"
• "Spell-proof the following sentence: [sentence]"

is really good. I'm still concerned about the sheer inefficiency of the process though, energy-wise.

This is absolutely in line with who buys into AI hype and why it is infuriating to try to convince them that they are reading way too much into how it seems to know things when all it is doing it returning results are statistically likely to be found as helpful to the audience it is designed for.

I have said that LLMs and other AI are designed to return what people want to see/hear. It doesn't know anything and will never be useful as a knowledge base or an independently functioning diagnostic tool.

It certainly has uses, but it certainly isn't going to solve all the things that are promoted by the AI hype train.

But there isn’t any mechanism inherent in large language models (LLMs) that would seem to enable this and, if real, it would be completely unexplained.

There's no mechanism in LLMs that allow for anything. It's a blackbox. Everything we know about them is empirical.

LLMs are not brains and do not meaningfully share any of the mechanisms that animals or people use to reason or think.

It's a lot like a brain. A small, unidirectional brain, but a brain.

LLMs are a mathematical model of language tokens. You give a LLM text, and it will give you a mathematically plausible response to that text.

I'll bet you a month's salary that this guy couldn't explain said math to me. Somebody just told him this, and he's extrapolated way more than he should from "math".

I could possibly implement one of these things from memory, given the weights. Definitely if I'm allowed a few reference checks.

Okay, this article is pretty long, so I'm not going to read it all, but it's not just in front of naive audiences that LLMs seem capable of complex tasks. Measured scientifically, there's still a lot there. I get the sense the author's conclusion was a motivated one.

There is no reason to believe that it thinks or reasons—indeed, every AI researcher and vendor to date has repeatedly emphasised that these models don’t think.

Geoffrey Hinton, for one

Can someone please paraphrase the following which I didn't understand?

Somebody raised to believe they have high IQ is more likely to fall for this than somebody raised to think less of their own intellectual capabilities. Subjective validation is a quirk of the human mind. We all fall for it.

But if you think you’re unlikely to be fooled, you will be tempted instead to apply your intelligence to “figure out” how it happened. This means you can end up using considerable creativity and intelligence to help the psychic fool you by coming up with rationalisations for their “ability”.

And because you think you can’t be fooled, you also bring your intelligence to bear to defend the psychic’s claim of their powers. Smart people (or, those who think of themselves as smart) can become the biggest, most lucrative marks.