As humanity has found yet another way to pass the buck, it'll be interesting to see the diminishing returns of LLMs as they begin to feed more and more on derivative content made by LLMs.

While agree with this sentiment, it is important to note that many tasks in school do not enrich the student as a person and their capabilities, but are mundane and/or repetitive failures of a badly designed curriculum. I can absolutely understand why students would want to automate such exercises.

I think of Chat GPT like a sometimes-inaccurate-calculator. There may be some legitimate uses for the technology, but it's still nice to know how to multiply numbers without it.

Chat GPT, and the many other similar systems, are unable to conceive of something new or original, merely imitate what has already come before

This. God do i hate that LLMs are called generative ai

Except they can absolutely come up with new things; their responses aren't just cut and pasted bites of previous text snippets. They are generated based on a neural network's idea of what the most likely next token is, and tokens are often fragments of words. There's a reason you can have it do arbitrary things with text- Because it's doing slightly deeper things than just imitation.

Chat GPT, and the many other similar systems, are unable to conceive of something new or original, merely imitate what has already come before.

So it does what grade school teachers expect of their students?

Most people can't do that and generative AI, no matter how limited is still better than the average shlub.

One main problem is all the wrong answers it generates. Imagine if when we invented the calculator it fucked up the answer 30% of the time.

AI opponents are very butthurt people

I also just feel like I'm not writing words for the fun of it. They're chosen to convey information in a very intentional way to a given target group. Like, just now in that previous sentence, I changed "in a certain way" to "in a very intentional way", because that's more precisely what I wanted to say. I try to convey lots of nuances in relatively few words.

That's my #1 criticism of LLMs, that they just blather on and on. And ultimately, precise nuance requires understanding the topic, the context and the target group, which, if you'd describe it to an LLM, would take longer than to write the actual text itself.

Maybe next time feed it a bunch of your writings and ask it to mimick your writing style.