This is because LLMs do not inherently understand math. They stick characters together that are likely to go together based on the content they were trained on. They're literally just glorified autocorrect.

If you want a tool that can actually do math from natural language input, try WolframAlpha.

That's why I can't ever trust what it says. No conviction. I'm like stick to your guns chatGPT.

This is what people don't understand about char gpt. It's not a tool for accuracy, even the company who made it says that. Then idiots come in and say "see it does math wrong! And it can't get a fact right! Only a moron would say this is the wave of the future!" And don't get me wrong. Google added it into their search engine because of the hype and low and behold, the LLM was inaccurate.

What is chat gpt food for? Creativity and abstracting. An LLM model is really good when you need a list of 10 terrible names for a dog food company to get your brain thinking. It's good at helping someone outline a story they want to write or make their email sound more professional. It's useful as an aid to help someone plan a schedule. The best use of chatgpt is to work with it, not try to get it to do a task without you. An LLM works best with piecemeal feedback and someone knowledgeable in the subject that can vet the answers it's giving.

Thank you

I'm fairness, a calculator wouldn't explain how to get there. An LLM might not explain it correctly, but it will explain it.

how is a calculator going to explain to you that you forgot to convert units

isn't it funny how humans will end up using a trillion times more energy so that the calculator can be less reliable, but acts like a person