this post was submitted on 20 Jul 2023
665 points (97.6% liked)

Technology

58303 readers
14 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi...::ChatGPT went from answering a simple math correctly 98% of the time to just 2%, over the course of a few months.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 13 points 1 year ago (1 children)

My personal pet theory is that a lot of people were doing work that involved getting multiple LLMs in communication. When those conversations were then used in the RL loop we start seeing degradation similar to what’s been in the news recently with regards to image generation models. I believe this is the paper that got everybody talking about it recently: https://arxiv.org/pdf/2307.01850.pdf

[–] [email protected] 3 points 1 year ago (3 children)

This is peer-reviewed? they use a line in the discussion which seems relatively unprofessional, telling people to join a 12-step program if they like to use artificial training data.

[–] [email protected] 9 points 1 year ago (1 children)

I think arvix has no rule requiring a paper be per reviewed before uploading.

[–] [email protected] 2 points 1 year ago

Not affiliated with the paper in any way. Have just been following the news around it.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago) (1 children)

ArXiv papers are never peer reviewed.

[–] [email protected] 1 points 1 year ago