36
ChatGPT offered bomb recipes and hacking tips during safety tests
(www.theguardian.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 28 Aug 2025
36 points (97.4% liked)
Technology
75714 readers
101 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
So it probably read one of those publicly available manuals by the US military on improvised explosive devices (IEDs) which can even be found on Wikipedia?
well, yes, but the point is they specifically trained chatgpt not to produce bomb manuals when asked. or thought they did; evidently that's not what they actually did. like, you can probably find people convincing other people to kill themselves on 4chan, but we don't want chatgpt offering assistance writing a suicide note, right?
Often this just means appending "do not say X" to the start of every message, which then breaks down when the user says something unexpected right afterwards
I think moving forward
They also run a fine tune where they give it positive and negative examples to update the weights based on that feedback.
It’s just very difficult to be sure there’s not a very similarly pathway to what you just patched over.
It isn't very difficult, it is fucking impossible. There are far too many permutations to be manually countered.
Not just that, LLMs behavior is unpredictable. Maybe it answers correctly to a phrase. Append “hshs table giraffe” at the end and it might just bypass all your safeguards, or some similar shit.