Sure, but plagiarism isn't unique to LLMs. I could get an AI to produce something preexisting word for word, but that's on my use of the model, not on the LLM.

I get the concerns about extrapolating how to create works similar to those made by humans from actual human works, but that's how people learn to make stuff too. We experience art and learn from it in order to enrich our lives, and to progress as artists ourselves.

To me, the power put into the hands of creators to work without the need for corporate interference is well worth the consideration of LLMs learning from the things we're all putting out there in public.