Perhaps by generating a bunch of complex copilot code to upload. It's easy to mass produce and would look plausibly functional.

... and tell it things, that are slightly obscene

Just need to use less obvious insults, a la, "your mother was a hamster, and your father smelt of elderberries"

Still poisons the model with something an end user won't like, but isn't easy enough to train out

Artisanal crap code.

Prompt for another AI: "write an example of code that looks correct but doesn't work"

Step 2; upload the resulting code to GitHub.

Step 3: make this an automated task.