286
submitted 1 year ago by L4s@lemmy.world to c/technology@lemmy.world

Teen boys use AI to make fake nudes of classmates, sparking police probe::Parents told the high school "believed" the deepfake nudes were deleted.

top 50 comments
sorted by: hot top controversial new old
[-] LogicalDrivel@sopuli.xyz 122 points 1 year ago

"...and their principal, Asfendis, has vowed to raise awareness on campus of how to use new technologies responsibly."

Surely all the teenage boys will understand, and only use the technology for wholesome purposes.

[-] averagedrunk@lemmy.ml 83 points 1 year ago

D.A.R.E. raised my awareness of drugs. I only used them for wholesome purposes.

[-] bhamlin@lemmy.world 2 points 1 year ago

Yup, put some right into my holes.

[-] Daft_ish@lemmy.world 5 points 1 year ago

Responsibly..!

[-] otter@lemmy.ca 20 points 1 year ago

They might know it's bad but not fully understand the potential harms. I made another comment on it

[-] Mac@mander.xyz 106 points 1 year ago

There is absolutely no way anyone could have possibly seen this coming.

[-] idunnololz@lemmy.world 23 points 1 year ago
[-] surewhynotlem@lemmy.world 92 points 1 year ago

You don't. Scissors and Polaroid and Playboy have been around for decades. If you wanted to see your classmates face on a nude and photocopy it, you could.

Now it's just easier and more believable. But it's not any more stoppable.

[-] RememberTheApollo@lemmy.world 8 points 1 year ago

Tbf the ease of creating multiple realistic images quickly along with the ability to rapidly share those images is a bit different than cut ‘n paste a completely unmatched head and torso.

load more comments (1 replies)
[-] r3df0x@7.62x54r.ru 4 points 1 year ago

It was harder for minors to get porn back then.

[-] kent_eh@lemmy.ca 8 points 1 year ago

I grew up "back then" (when VHS was new technology) and it really wasn't very hard to get some sort of porn.

load more comments (3 replies)
[-] 13esq@lemmy.world 3 points 1 year ago

Yh, I don't like that my generation was the first to be exposed to freely available and copious amounts of porn. But on the flip side the internet is sooooooo useful and I would not endorse any government saying what should or shouldn't be on the internet, the internet should be free and censorship would be a very slippery slope.

It's a tough moral debate and I'm really not sure what the answer is.

[-] r3df0x@7.62x54r.ru 3 points 1 year ago* (last edited 1 year ago)

In general, I'm against "censorship." There's also no reason why we should allow non consensual porn to be circulated. Abolishing online porn entirely would instantly solve the question of "revenge porn" and whether consent was obtained.

There's a certain amount of logic to the idea that you should allow bad people to feel free to express themselves so they can be identified, but I don't think that fully holds up with porn and can lead to women being harmed. Girls who are 18-22 aren't in a position to fully resist the temptation being offered by a lot of sex work possibilities. They haven't learned about money yet and getting offered "a lot" in the short term is going to be too hard for them to weigh against selling out their future at that age. For every "empowered sex worker" out there who makes a good living and really wants to be in that work, there are many more who were exploited or got into it because of mental illness or trauma. Commodification of sex is ultimately a feature of the capitalist system.

load more comments (7 replies)
[-] saltesc@lemmy.world 8 points 1 year ago* (last edited 1 year ago)

Go back to using natural intelligence and try render with brain. Images can't be shared.

[-] Melt@lemm.ee 6 points 1 year ago

Brain sends data to hand and hand render it with pen and paper, what now?

[-] Hadriscus@lemm.ee 2 points 1 year ago

feet carry drawing to photocopier

[-] Kusimulkku@lemm.ee 2 points 1 year ago

This reminds me of the funny picture about a black person being angry that white people can think of slurs and there's nothing that can be done about it

[-] otter@lemmy.ca 76 points 1 year ago* (last edited 1 year ago)

The other comment about how this has been happening for a long time (with low tech methods) is true, and it's also true that we can't stop this completely. We can still respond to it:

An immediate and easy focus would be on what they do with the images. Sharing them around is still harassment / bullying and it should be dealt with in the same way as it currently is.

There's also an education aspect to it. In the past, those images (magazines, photocopies, photoshop) would be limited in who sees them. The kids now are likely using free online tools that aren't private or secure, and those images could stick around forever. So it could be good to highlight that

  • Your friends and classmates may see them, and it may harm their lives. The images will likely stick around. Facial recognition algorithms are also improving, so it's a legitimate concern that an image stored on a random site somewhere will be tied back to them.
  • The images can be traced back to the creator and the creator can face repercussions for it (for those without empathy, this might be the better selling point
[-] Emma_Gold_Man@lemmy.dbzer0.com 23 points 1 year ago

Your point 1 seems to forget something important: kids are often cruel, and bullying is frequently the point. So long term consequences for their classmates can be an incentive more than a deterrent.

load more comments (1 replies)
[-] Adalast@lemmy.world 15 points 1 year ago

To your first point, much to the benefit of humanity, and counter to popular belief, the internet is NOT forever. Between link rot, data purges, corporate buyouts, transmission compression losses, and general human stupidity, large swaths of the internet have vanished. Hell, just Macromedia selling out to Adobe ended up causing the loss of most of the popular internet games and videos for anyone in their mid to late 30s at this point (you will be misses Flash). The odds of these specific AI-generated child porn pictures surviving even in some dark corner of the bright web are slim to none. And if they end up surviving in the dark web, well, anyone who sees them will likely have a LOT of explaining to do.

Also, for the commentary of the websites keeping the images. That is doubtful, beyond holding them in an account-bound locker for the user to retrieve. They don't care and too many images get generated every day for them to see it as more than reinforcement training.

Speaking of reinforcement training, they may have been able to use Photoshop's new generative fill to do this, but to actually generate fresh images of a specific peer they would have had to train a LoRA or Hypernerwork on photos of the girl so the SD could actually resolve it. They weren't doing that on an AI site, especially not a free one. They were probably using ComfyUI or Automatic1111 (I use both myself). They are free, open source, locally executed software that allow you to use the aforementioned tools when generating. That means that the images were restricted to their local machine, then transferred to a cell phone and distributed to friends.

https://www.theatlantic.com/technology/archive/2021/06/the-internet-is-a-collective-hallucination/619320/

load more comments (1 replies)
[-] 6daemonbag@lemmy.dbzer0.com 50 points 1 year ago

My niece had this same issue a few years ago but with Photoshop. It absolutely ruined her. Changed schools multiple times (public and private) but social media exists so all the kids knew. She ended up getting homeschooled for the last 5 years of school as well as a fuckload of therapy. She came out the other side okay but she has massive trust issues and anxiety

[-] Squizzy@lemmy.world 14 points 1 year ago

Man that's awful, poor thing.

[-] NightAuthor@lemmy.world 33 points 1 year ago

Ai porn to come w this disclaimer: disclaimer

[-] DasherPack@lemmy.world 22 points 1 year ago

In Spain it happened recently with some 12y/olds...it created a country-wide debate, and as always, did not lead to any regulation. Hopefully the EU will do something

[-] r3df0x@7.62x54r.ru 4 points 1 year ago

This can be prosecuted with existing CP laws.

load more comments (7 replies)
[-] autotldr@lemmings.world 11 points 1 year ago

This is the best summary I could come up with:


This October, boys at Westfield High School in New Jersey started acting "weird," the Wall Street Journal reported.

It took four days before the school found out that the boys had been using AI image generators to create and share fake nude photos of female classmates.

Biden asked the secretary of Commerce, the secretary of Homeland Security, and the heads of other appropriate agencies to provide recommendations regarding "testing and safeguards against" producing "child sexual abuse material" and "non-consensual intimate imagery of real individuals (including intimate digital depictions of the body or body parts of an identifiable individual), for generative AI."

"New York State currently lacks the adequate criminal statutes to protect victims of ‘deepfake’ pornography, both adults and children," Donnelly said.

Until laws are strengthened, Bramnick has asked the Union County prosecutor to find out what happened at Westfield High School, and state police are still investigating.

Until the matter is settled in the New Jersey town, the girls plan to keep advocating for victims, and their principal, Asfendis, has vowed to raise awareness on campus of how to use new technologies responsibly.


The original article contains 950 words, the summary contains 184 words. Saved 81%. I'm a bot and I'm open source!

[-] randon31415@lemmy.world 7 points 1 year ago

Pictures? We are on the edge of believable videos with AI produced voices and sounds - made on normal computers. Need to clear a few more hurdles in 3D AI modeling, VR, and haptic feedback before this trend reaches it's obvious conclusion.

Wonder what crime it would be called if you create a haptic VR double of someone unconsensually and don't distribute it?

[-] jackalope@lemmy.ml 3 points 1 year ago

Haptics are never going to be like in Ready Player One. It's crazy to me that anyone believes the tech will be capable of that. Like how diminished is one's sense of touch that one could believe it could be fooled by fancy rumble packs? Touch is so much more complex than that. Piezoelectric motors vibrating are not going to be able to be able to fake solidity. Nuts to me people think that.

[-] TheDarksteel94@sopuli.xyz 4 points 1 year ago

Might be possible with big gel tanks that people get submerged in, so the gel would be somehow hardened or softened with precise and weak electric currents, emulating textures.

But imo, it's more likely that it'll happen through some brain interface and the whole experience will basically be a very lucid dream.

Lots of time until that though, unless we destroy ourselves first. At least I doubt it'll happen during my lifetime.

load more comments (4 replies)
load more comments (2 replies)
[-] badbytes@lemmy.world 4 points 1 year ago

Thanks authorities, for taking seriously the safety of our students in the classroom. I also saw a kid eat glue.

[-] mwguy@infosec.pub 4 points 1 year ago

Given that AI images and media can't be copyrighted, does the nominal "subject" have any recourse?

[-] kadu@lemmy.world 6 points 1 year ago

Not sure about other places, but here in Brazil creating a fake nude of somebody and distributing it would be illegal

load more comments (4 replies)
load more comments (1 replies)
load more comments
view more: next ›
this post was submitted on 03 Nov 2023
286 points (96.1% liked)

Technology

59598 readers
4583 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS