644

Dreams come true (mander.xyz)

submitted 1 month ago by fossilesque@mander.xyz to c/science_memes@mander.xyz

40 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[-] runner_g@lemmy.blahaj.zone 31 points 1 month ago

Someone's probably already coined the term, but I'm going to call it LLM inbreeding.

[-] Naz@sh.itjust.works 18 points 1 month ago

I suggested this term in academic circles, as a joke.

I also suggested hallucinations ~3-6 years ago only to find out it was ALSO suggested in the 1970s.

Inbreeding, lol

[-] anzo@programming.dev 4 points 1 month ago

In computer science, garbage in, garbage out (GIGO) is the concept that flawed, biased or poor quality ("garbage") information or input produces a result or output of similar ("garbage") quality. The adage points to the need to improve data quality in, for example, programming.

There was some research article applying this 70s computer science concept to LLMs. It was published in Nature and hit major news outlets. Basically they further trained GPT on its output for a couple generations, until the model degraded terribly. Sounded obvious to me, but seeing it happen on the www is painful nonetheless...