156
Stack overflow is almost dead (blog.pragmaticengineer.com)
you are viewing a single comment's thread
view the rest of the comments
[-] TheTechnician27@lemmy.world 16 points 2 days ago* (last edited 2 days ago)

Dude, I'm sorry, I just don't know how else to tell you "you don't know what you're talking about". I'd refer you to Chapter 20 of Goodfellow et al.'s 2016 book on Deep Learning, but 1) it tragically came out a year before transformer models, and 2) most of it will go over your head without a foundation from many previous chapters. What you're describing – generative AI training on generative AI ad infinitum – is a death spiral. Literally the entire premise of adversarial training of generative AI is that for the classifier to get better, you need to keep funneling in real material alongside the fake material.

You keep anthropomorphizing with "AI can already understand X", but that betrays a fundamental misunderstanding of what a deep learning model is: it doesn't "understand" shit about fuck; it's an unfathomably complex nonlinear algebraic function that transforms inputs to outputs. To summarize in a word why you're so wrong: overfitting. This is one of the first things you'll learn about in a ML class, and it's what happens when you let a model train on the same data over and over again forever. It's especially bad for a classifier to be overfitted when it's pitted against a generator, because a sufficiently complex generator will learn how to outsmart the overfitted classifier and it will find a cozy little local minimum that in reality works like dogshit but outsmarts the classifier which is its only job.

You really, really, really just fundamentally do not understand how a machine learning model works, and that's okay – it's a complex tool being presented to people who have no business knowing what a Hessian matrix or a DCT is – but please understand when you're talking about it that these are extremely advanced and complex statistical models that work on mathematics, not vibes.

this post was submitted on 20 May 2025
156 points (99.4% liked)

Programming

20279 readers
568 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 2 years ago
MODERATORS