200
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 16 Mar 2024
200 points (90.3% liked)
Ask Lemmy
27062 readers
235 users here now
A Fediverse community for open-ended, thought provoking questions
Please don't post about US Politics. If you need to do this, try !politicaldiscussion@lemmy.world
Rules: (interactive)
1) Be nice and; have fun
Doxxing, trolling, sealioning, racism, and toxicity are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them
2) All posts must end with a '?'
This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?
3) No spam
Please do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.
4) NSFW is okay, within reason
Just remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com.
NSFW comments should be restricted to posts tagged [NSFW].
5) This is not a support community.
It is not a place for 'how do I?', type questions.
If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.
Reminder: The terms of service apply here too.
Partnered Communities:
Logo design credit goes to: tubbadu
founded 1 year ago
MODERATORS
Citation needed that LLMs are passing these tests like they’re nothing.
LLMs don’t have intelligence, they are sentence generators. Sometimes those sentences are correct, sometimes they’re gobbledygook.
For instance, they fabricate real-looking but nevertheless totally fake citations in research papers https://www.nature.com/articles/s41598-023-41032-5
To your point we already know standardized tests are biased and poor tools to measure intelligence. Partly that’s because they don’t actually measure intelligence- they often measure rote knowledge. We don’t need LLMs to make that determination, we already can.
Talked about this a few times over the last few weeks but here we go again...
I am teaching myself to write and had been using chatgpt for super basic grammar assistance. Seemed like an ideal thing, toss a sentence I was iffy about into it and ask it what it thought. After all I wasn't going to be asking it some college level shit. A few days ago I asked it about something I was questionable on. I honestly can't remember the details but it completely ignored the part of the sentence I wasn't sure about and told me something else was wrong. What it said was wrong was just....not wrong. The 'correction' it gave me was some shit a third grader would look at and say 'uhhhhh.....I'm gonna ask someone else now...'
That's because LLMs aren't intelligent. They're just parrots that repeat what they've heard before. This stuff being sold as an "AI" with any "intelligence" is extremely misleading and causing people to think it's going to be able to do things it can't.
Case in point, you were using it and trusting it until it became very obvious it was wrong. How many people never get to that point? How much has it done wrong before then? Etc.
OP picked standardized tests that only require memorization because they have zero idea what a real IQ test like the WAIS is like.
Also how those IQ tests work. You kind of have to go in "blind" to get an accurate result. And LLM can't do anything "blind" because you have to train them.
A chatbots can't even take a real IQ test, if we trained a chatbots to take a real IQ test, it would be a pointless test
Actually, you can give chatbots a real IQ test, and the range of scores fall into roughly the same spread as how they rank on other measures, with the leading model scoring at 100:
https://www.maximumtruth.org/p/ais-ranked-by-iq-ai-passes-100-iq
Nobody is a blank slate. Everyone has knowledge from their past experience, and instincts from their genetics. AIs are the same. They are trained on various things just as humans have experienced various things, but they can be just as blind as each other on the contents of the test.
No, they wouldn't.
Because real IQ tests arent just multiple choice exams
You would have to train it to handle the different tasks, and training it at the tasks would make it better at the tasks, raising their scores.
I don't know if the issue is you don't know about how IQ tests work, or what LLM can do.
But it's probably both instead of one or the other.
You're entirely missing the point.
The requirements and basis of IQ tests are they are problems you haven't seen before. An LLM works by recognizing existing data and returning what came next in the training set.
LLMs work directly in opposition of how an IQ text works.
Things like past experience are all the shit IQ tests need to avoid in order to be accurate. And they're exactly what LLMs work off of.
By definition, LLMs have no IQ.