Not to mention that a response "containing" plagiarism is a pretty poorly defined criterion. The system being used here is proprietary so we don't even know how it works.

I went and looked at how low theater and such were and it's dramatic:

The lowest similarity scores appeared in theater (0.9%), humanities (2.8%) and English language (5.4%).

So, if the Ai gives you a correct answer to a science question, it’s “infringing copyright” and if it spits out a bullshit answer, it’s giving you wrong, and unsupported claims.

Right? Nod doubt that output can be similar to training data, and I would believe that some of it is plagiarism, but plagiarism detectors are infamous among uni students for being completely unreliable and flagging pronouns, dates and citations. Until someone can go "here's an example of actual plagiarism" (which is obvious when pointed out), these claims make no sense.

Eh, kinda. It’s not like a science paper is just going to be an equation and nothing else. An author’s synthesis of the results is always going to have unique language. And that is even more true for a social science paper.

But also, there is far less training data to mix and match responses from, so naively I would expect a higher plagiarism rate, by its very nature.

Less than 2% of the world's population has a doctorate. According to the US Census Bureau, only 1.2% of the US population has a PhD.

source

I think the issue is more about HOW they wrote it, rather than WHO wrote it.

You can’t write a paper covering scientific topics without plagiarism. A human would be required to. Generative AI should be held to at least as high of a standard.

Yep, Axios straight-up printed an ad as news.

They should show a small, but representative sample of questions they gave it.

Also they should compare the scores to similarity scores for a flesh and blood smart human that answers the questions.

Well, I tried it. So here's an example.

this may soon be a thing of the past as

This fragment was flagged as plagiarism.

Probably very few. The bias for these companies is in false negatives, not false positives, since false positives create controversy when students appeal a ruling.

100% iirc, there are only so many ways to write about how the blue curtains indicate the character is feeling depressed or something.

ChatGPT itself doesn't know where it got the info from, so it makes up links and names - it's a language model, not a search engine.

On the other hand, if you manage to find a reputable source and give it relevant metadata, it can format a nice citation for you, saving you time on that instead.

Badly. This burns my laziest students every semester. Chatgpt just adds nonsense citations.

Microsoft's copilot adds them, it's why I prefer to use it.

There is custom gpts for that. ScholarAI and Consensus are OK.

It depends on how they're using it behind the scenes. Chatbots like ChatGPT can't cite sources, because they are just generating text on the fly. However, some approaches (if links/sources are provided) use an approach called Rag (Retrevial Augmented Generation). This approach uses similarity in search terms to find sources first, then uses the sources to augment/generate its answer.

That being said there are pros and cons to both approaches.

Perplexity AI includes citations every time.

No. It's not really clear what LLMs do, but it certainly depends on context.

What they fundamentally do is continue a text. That's what they were originally trained to do. Then they were fine-tuned to continue a chat log or respond to an instruction. To be able to do that, they have learned a lot. Unfortunately, we do not know what.

If you ask for a summary of some text, it will give you one; regardless of whether the text even exists.

The summary could be one written by a human that it has memorized. Or it could be complete nonsense, that it is making up on the fly. You never know.

59.7%