Is it practically feasible to regulate the training? Is it even necessary? Perhaps it would be better to regulate the output instead.

It will be hard to know that any particular GET request is ultimately used to train an AI or to train a human. It's currently easy to see if a particular output is plagiarized. https://plagiarismdetector.net/ It's also much easier to enforce. We don't need to care if or how any particular model plagiarized work. We can just check if plagiarized work was produced.

That could be implemented directly in the software, so it didn't even output plagiarized material. The legal framework around it is also clear and fairly established. Instead of creating regulations around training we can use the existing regulations around the human who tries to disseminate copyrighted work.

That's also consistent with how we enforce copyright in humans. There's no law against looking at other people's work and memorizing entire sections. It's also generally legal to reproduce other people's work (eg for backups). It only potentially becomes illegal if someone distributes it and it's only plagiarism if they claim it as their own.