I thought about the indexing situation in contrast to the user paywall. Without thinking too much about any legal argument, it would seem that NYT having a paywall for visitors is them enforcing their right to the content signaling that it isn't free for all use, while them allowing search indexers access is allowing the content to visible but not free on the market.

It reminds me of the Canadian claim that Google should pay Canadian publishers for the right to index, which I tend to disagree with. I don't think Google or Bing should owe NYT money for indexing, but I don't think allowing indexing confers the right for commercial use beyond indexing. I highly suspect OpenAI spoofed search indexers while crawling content specifically to bypass paywall and the like.

I think part of what the courts will have to weigh for the fair use arguments is the extent to which NYT it's harmed by the use, the extent to which the content is transformed, and the public interest between the two.

I find it interesting that OpenAI or Microsoft already pay AP for use of their content because it is used to ensure accurate answers are given to users. I struggle to see how the situation is different with NYT in OpenAI opinion, other than perhaps on price.

It will be interesting to see what shakes out in the courts. I'm also interested in the proposed EU rules which recognize fair use for research and education, but less so for commercial use.

Thanks for the reply! Have a great day!