The LLM is unable to determine the truth of its own output. I would argue that is a key to claiming intelligence but determining what intelligence means is itself a philosophical question up for debate.

Yeah exactly and a great way to see this is by asking it to produce two viewpoints about the same subject, a negative and positive review of something you're familiar with is perfect. It produces this hilarious "critic" type jargon but you can tell it doesn't actually understand. Coincidentally, it's drawing from a lot of text where the original human author(s) might not understand either and are merely themselves re-producing a jargon-heavy text for an assignment by their employer or academic institution. If AI can so accurately replicate some academic paper that probably didn't need to be written for anything other than to meet publishing standards for tenured professors, then that's really a reflection on the source material. Since LLM can only create something based on existing input, almost all the criticisms of it, are criticisms that can apply to it's source material.