I would like to take a crack at this. There is this recent trend going around with ghiblifying one's picture. Its basically converting a picture into ghibli image. If you had trained it on free sources, this is not possible.

Internally an LLM works by having networks which activate based on certain signals. When you ask it a certain question. It creates a network of similar looking words and then gives it back to you. When u convert an image, you are doing something similar. You cannot form these networks and the threshold at which they activate without seeing copyrighted images from studio ghibli. There is no way in hell or heaven for that to happen.

OpenAI trained their models on pirated things just like meta did. So when an AI produces an image in style of something, it should attribute the person from which it actually took it. Thats not whats happening. Instead it just makes more money for the thief.