74
Trusting your own judgement on 'AI' is a huge risk
(www.baldurbjarnason.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 10 Jun 2025
74 points (94.0% liked)
Programming
20858 readers
447 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 2 years ago
MODERATORS
My apologies, I was too vague. I'm saying "thinking" by definition is not "statistics". Where Monkeys, birds, and human babies all "think", LLMs use algorithms and "statistics". I also think that "statistics" not meaning the same thing that "thinking" is a valid argument. I would go farther and say it's important that words have meaning. That is what I was attempting to convey. I'm happy to clear up anything I was unclear about.
You are mistaking how LLMs are trained to how they work.
It's not because it's been trained with statistics, that they compute, or think using statistics.
For example, to do additions, internally LLMs do trignonometry: https://arxiv.org/abs/2502.00873
They do probably use statistics for tons of stuff internally, but humans do too: guessing, bias, tendency, preferences.
Anthropics researcher found that their LLMs have "features" for concepts.
I don't think you can disconnect how an LLM was trained from how it operates. If you train an LLM to use trigonometry to solve addition problems, I think you will find the LLM will do trigonometry to solve addition problems. If you train an LLM in only Russian, it will speak Russian. I would suggest that regardless of what you train it on it will choose the statistically most likely next token based on its training data.
I would also suggest we don't know the exact training data being used on most LLMs, so as outsiders we can't say one way or another on how the LLM is being trained to do anything. We can try to extrapolate from posts like the one that you linked to how the LLM was trained though. In general if that is how the LLM is coming to its next token, then the training data must be really heavily weighted in that manner.
You can, heck the example I gave show exaclty this:
It was not trained to do trigonometry to solve addition problem, it was trained to respond to additions, trigonometry is how the statiscal part, the backpropagation, found a way to make the neurons solve additions.
You are mixing up stuff, the way LLM are trained does not impose anything about how the neurons gets organised to get better score at inferrence.
I would point out I think you might be overly confident in the manner in which it was trained addition. I'm open to being wrong here, but when you say "It was not trained to do trigonometry to solve addition problem", that suggests to me either you know how it was trained, or are making assumptions about how it was trained. I would suggest unless you work at one of these companies, you probably are not privy to their training data. This is not an accusation, I think that is probably a trade secret at this point. And if the idea that there would be nobody training an LLM to do addition in this manner, I invite you to glance the Wikipedia article on addition. Really glance at literally any math topic on Wikipedia. I didn't notice any trigonometry in this entry but I did find the discussion around finding the limits of logarithmic equations in the "Related Operations" section: https://en.m.wikipedia.org/wiki/Addition. They also cite convolution as another way to add in which they jump straight to calculus: https://en.m.wikipedia.org/wiki/Convolution.
This is all to say, I would suggest that we don't know how they're training LLMs. We don't know what that training data is or how it is being used exactly. What we do know is that LLMs work on tokens and weights. The weights and statistical relevance to each of the other tokens depends on the training data, which we don't have access to.
I know this is not the point, but up until this point I've been fairly pedantic and tried to use the correct terminology, so I would point out that technically LLMs have "tensors" not "neurons". I get that tensors are designed to behave like neurons, and this is just me being pedantic. I know what you mean when you say neurons, just wanted to clarify and be consistent. No shade intended.
The general way it is trained is known, specifics and technics are not known, but the public do know how one of the flagship model was trained, the training process of deepseek r1 was documented in their research paper: https://arxiv.org/pdf/2501.12948
Which I did read a chunk when it was released.
The LLMs have multiple way to do additions, i'll showcase two as an example, I asked ChatGPT 4.1 to solve a big addition. Here it's output:
You can notice, the whole reasoning is correct, but it wrote the wrong response, I can expand more on this if you want (I do some research on it on my free time)
This reasoning of decomposing the addition was of course learned from training data.
Now, the trigonometry used to calculate additions that i talked earlier, is not for writing a "reasoning" but when it try to write the correct response. It was created by the backpropagation trying to find a local minimum that can solve additions in order to more accuratly predict the next token.
Artificial neurons were made to behave like neurons: https://en.wikipedia.org/wiki/Artificial_neuron
And the terminology used, is neurons, cf the paper i sent earlier about how they do additions: https://arxiv.org/pdf/2502.00873
I don't doubt that it can perform addition in multiple ways. I would go as far as saying it can probably attempt to perform addition in more ways than the average person as it has probably been trained on a bunch of math. Can it perform it correctly? Sometimes. That's ok, people make mistakes all the time too. I don't take away from LLMs just because they make mistakes. The ability to do math in multiple ways is not evidence of thinking though. That is evidence that it's been trained on at least a fair bit of math. I would say if you train it on a lot of math, it will attempt to do a lot of math. That's not thinking, that's just increasing the weighting on tokens related to math. If you were to train an LLM on nothing but math and texts about math, then asked it an art question, it would respond somewhat nonsensically with math. That's not thinking, that's just choosing the statistically most likely next token.
I had no idea about artificial neurons, TIL. I suppose that makes "neural networks" make more sense. In my readings on ML they always seemed to go straight to the tensor and overlook the neuron. They would go over the functions to help populate the weights but never used that term. Now I know.
I've been re reading my response and my bad, I meant "artificial neurons were inspired from neurons", not to behave like, they have little in common.
If you asked an human that speak german and nothing else, a question in english, it would also respond in german (that they cant understand you).
LLMs sometimes (not often enough) do respond they don't know.