808
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 06 Jun 2024
808 points (98.8% liked)
Technology
59436 readers
1762 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
Aside from it not really working, though.
Glaze attempts to "poison" AI training by using adversarial noise to trick AIs into perceiving it as something that it's not, so that when a description is generated for the image it'll be incorrect and the AI will be trained wrong. There are a couple of problems with this, though. The adversarial noise is tailored to specific image recognition AIs, so it's not future-proof. It also isn't going to have an impact on the AI unless a large portion of the training images are "poisoned", which isn't the case for typical training runs with billions of images. And it's relatively fragile against post-processing, such as rescaling the image, which is commonly done as an automatic part of preparing data for training. It also adds noticeable artefacts to the image, making it look a bit worse to the human eye as well.
There's a more recent algorithm called Nightshade, but I'm less familiar with its details since it got a lot less attention that Glaze and IIRC the authors tried keeping some of its details secret so that AI trainers couldn't develop countermeasures. There was a lot of debate over whether it even worked in the first place, since it's not easy to test something like this when there's little information about how it functions and training a model just to see if it breaks is expensive. Given that these algorithms have been available for a while now but image AIs keep getting better I think that shows that whatever the details it's not having the desired effect.
Part of the reason why Cara's probably facing such financial hurdles is that it's computationally expensive to apply these things. They were also automatically running "AI detectors" on images, which are expensive and unreliable. It's an inherently expensive site to run even if they were doing it efficiently.
IMO they would have been much better served just adding "No AI-generated images allowed" to their ToS and relying on their users to police themselves and each other. Though given the witch-hunts I've seen and the increasing quality of AI art itself I don't think that would really work for very long either.
That’s really interesting. Will have to watch how this turns out, see if any 2025 image models can imitate Cara artists.