Kinda weird GPT4-Chan wasn't referenced. A guy fine-tuned GPT-J on 4chan, then deployed bots to write posts. I guess it was more of a stunt than academic or scientific, but training on 4chan improved the model's performance on a truthfulness benchmark.

Those are actually some very good results. Funny situation, if the copyright companies win the AI legislative war, 4chan is going to get twice as much as reddit did for the data at the minimum.

It's also interesting the model gets worse faster if it has to untrain the toxic data so to speak.

Makes sense if you look at abliterated models. Once abliterated and retrained they seem to improve. Imo we are adding too much human bias by trying to guide the LLM. Censored models are good and need to be used in some situations, but shouldn't the base be just data and only then finetune to desired output?

I envision a Gemini powered bot that cracks captcha and posts "woke" replies on 4chan. If you're an antivaxxer, antisemite, nazi, racist, sionist, or otherwise, it will debate you. It will not get tired. It will not get mad. It will maintain a sense of decorum indefinitely and it will never ever stop. If some far right extremist decides to do the same, it will have the advantage that academia is left leaning, meaning the model can cite widely recognized studies.

Dead internet theory and so on, but I'll gladly completely and utterly destroy the internet if it means the filth dies with it.

can we stop referring to llm's as if they're capable of thought? they don't make decisions; their programming just responds to patterns.

Fresh "AI" pseudo-science for a monday morning.

These grifters never even define "bad/toxic data". It's just 4chan ffs.