199
submitted 1 week ago* (last edited 1 week ago) by lgsp@feddit.it to c/technology@lemmy.world

I found the aeticle in a post on the fediverse, and I can't find it anymore.

The reaserchers asked a simple mathematical question to an LLM ( like 7+4) and then could see how internally it worked by finding similar paths, but nothing like performing mathematical reasoning, even if the final answer was correct.

Then they asked the LLM to explain how it found the result, what was it's internal reasoning. The answer was detailed step by step mathematical logic, like a human explaining how to perform an addition.

This showed 2 things:

  • LLM don't "know" how they work

  • the second answer was a rephrasing of original text used for training that explain how math works, so LLM just used that as an explanation

I think it was a very interesting an meaningful analysis

Can anyone help me find this?

EDIT: thanks to @theunknownmuncher @lemmy.world https://www.anthropic.com/research/tracing-thoughts-language-model its this one

EDIT2: I'm aware LLM dont "know" anything and don't reason, and it's exactly why I wanted to find the article. Some more details here: https://feddit.it/post/18191686/13815095

you are viewing a single comment's thread
view the rest of the comments
[-] AnneBonny@lemmy.dbzer0.com 6 points 1 week ago

How would you prove that someone or something is capable of reasoning or thinking?

[-] glizzyguzzler@lemmy.blahaj.zone 6 points 1 week ago

You can prove it’s not by doing some matrix multiplication and seeing its matrix multiplication. Much easier way to go about it

[-] theunknownmuncher@lemmy.world 18 points 1 week ago* (last edited 1 week ago)

Yes, neural networks can be implemented with matrix operations. What does that have to do with proving or disproving the ability to reason? You didn't post a relevant or complete thought

Your comment is like saying an audio file isn't really music because it's just a series of numbers.

[-] glizzyguzzler@lemmy.blahaj.zone 2 points 1 week ago* (last edited 1 week ago)

Improper comparison; an audio file isn’t the basic action on data, it is the data; the audio codec is the basic action on the data

“An LLM model isn’t really an LLM because it’s just a series of numbers”

But the action of turning the series of numbers into something of value (audio codec for an audio file, matrix math for an LLM) are actions that can be analyzed

And clearly matrix multiplication cannot reason any better than an audio codec algorithm. It’s matrix math, it’s cool we love matrix math. Really big matrix math is really cool and makes real sounding stuff. But it’s just matrix math, that’s how we know it can’t think

[-] theunknownmuncher@lemmy.world 5 points 1 week ago* (last edited 1 week ago)

LOL you didn't really make the point you thought you did. It isn't an "improper comparison" (it's called a false equivalency FYI), because there isn't a real distinction between information and this thing you just made up called "basic action on data", but anyway have it your way:

Your comment is still exactly like saying an audio pipeline isn't really playing music because it's actually just doing basic math.

[-] BB84@mander.xyz 3 points 1 week ago

Can humans think?

[-] DarkDarkHouse@lemmy.sdf.org 3 points 1 week ago

Do LLMs not exhibit emergent behaviour? But who am I, a simple skin-bag of chemicals, to really say.

[-] whaleross@lemmy.world 6 points 1 week ago

People that can not do Matrix multiplication do not possess the basic concepts of intelligence now? Or is software that can do matrix multiplication intelligent?

this post was submitted on 01 Jun 2025
199 points (97.2% liked)

Technology

71356 readers
3502 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS