663
submitted 1 year ago by L4s@lemmy.world to c/technology@lemmy.world

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi...::ChatGPT went from answering a simple math correctly 98% of the time to just 2%, over the course of a few months.

you are viewing a single comment's thread
view the rest of the comments
[-] spaduf@lemmy.blahaj.zone 13 points 1 year ago

My personal pet theory is that a lot of people were doing work that involved getting multiple LLMs in communication. When those conversations were then used in the RL loop we start seeing degradation similar to what’s been in the news recently with regards to image generation models. I believe this is the paper that got everybody talking about it recently: https://arxiv.org/pdf/2307.01850.pdf

[-] teydam@lemmy.world 4 points 1 year ago

This is peer-reviewed? they use a line in the discussion which seems relatively unprofessional, telling people to join a 12-step program if they like to use artificial training data.

[-] ghostinthessh@lemmy.sdf.org 9 points 1 year ago

I think arvix has no rule requiring a paper be per reviewed before uploading.

[-] spaduf@lemmy.blahaj.zone 2 points 1 year ago

Not affiliated with the paper in any way. Have just been following the news around it.

[-] blue_zephyr@lemmy.world 2 points 1 year ago* (last edited 1 year ago)

ArXiv papers are never peer reviewed.

[-] teydam@lemmy.world 1 points 1 year ago
this post was submitted on 20 Jul 2023
663 points (97.6% liked)

Technology

60101 readers
3271 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 2 years ago
MODERATORS