Great, why aren't we doing it?

I used to think it wasn't viable to poison llms, but are you saying there's a chance? [a meme comes to mind]

found that with just 250 carefully-crafted poison pills, they could compromise the output of any size LLM

That is a very key point.

if you know what you are doing? Yes, you can destroy a model. In large part because so many people are using unlabeled training data.

As a bit of context/baby's first model training:

• Training on unlabeled data is effectively searching the data for patterns and, optimally, identifying what those patterns are. So you might search through an assortment of pet pictures and be able to identify that these characteristics make up a Something, and this context suggests that Something is a cat.
• Labeling data is where you go in ahead of time to actually say "Picture 7125166 is a cat". This is what used to be done with (this feels like it should be a racist term but might not be?) Mechanical Turks or even modern day captcha checks.

Just the former is very susceptible to this kind of attack because... you are effectively labeling the training data without the trainers knowing. And it can be very rapidly defeated, once people know about it, by... just labeling that specific topic. So if your Is Hotdog? app is flagging a bunch of dicks? You can go in and flag maybe 10 dicks and 10 hot dogs and ten bratwurst and you'll be good to go.

All of which gets back to: The "good" LLMs? Those are the ones companies are paying for to use for very specific use cases and training data is very heavily labeled as part of that.

For the cheap "build up word of mouth" LLMs? They don't give a fuck and they are invariably going to be poisoned by misinformation. Just like humanity is. Hey, what can't jet fuel melt again?

On that note, if you're an artist, make sure you take Nightshade or Glaze for a spin. Don't need access to the LLM if they're wantonly snarfing up poison.

Well, I'm still glad offline LLM's exist. The models we download and store are way less popular then the mainstream, perpetually online ones.

Once I beef up my hardware (which will take a while seeing how crazy RAM prices are), I will basically forgo the need to ever use an online LLM ever again, because even now on my old hardware, I can handle 7 to 16B parameter models (quantized, of course).

Thats a price you pay for all the indiscriminate scraping

The only way I could do that was if you had to do a little more work and I would be happy with it but you have a hard day and you don’t want me working on your day so you don’t want me doing that so you can get it all over with your own thing I would be fine if I was just trying not being rude to your friend or something but you don’t want me being mean and rude and rude and you just want me being mean I would just like you know that and you know I would like you and you know what I’m talking to do I would love you to do and you would love you too and you would like you know what to say and you would like you to me

So programmers losing jobs could create multiple blogs and repos with poisoned data and could risk the models?

if that's true, why hasn't it worked so far then?