The author also proposes a framework for analyzing claims about generative AI. I don't know if I endorse it fully, but I agree that each of the four talking points represents a massive failure of understanding. Their LIES model is:

• Lethality: the bots will kill us all
• Inevitability: the bots are unstoppable and will definitely be created in the future
• Exceptionalism: the bots are wholly unlike any past technology and we are unprepared to understand them
• Superintelligent: the bots are better than people at thinking

I would add to this a Plausibility or Personhood or Personality: the incorrect claim that the bots are people. Maybe call it PILES.

This is a list of apostates. The idea is not to actually detail the folks who do the most damage to the cult's reputation, but to attack the few folks who were once members and left because they were no longer interested in being part of a cult. These attacks are usually motivated by emotions as much as a desire to maintain control over the rest of the cult; in all cases, the sentiment is that the apostate dared to defy leadership. Usually, attacks on apostates are backed up by some sort of enforcement mechanism, from calls for stochastic terrorism to accusations of criminality; here, there's not actually a call to do anything external, possibly because Habryka realizes that the optics are bad but more likely because Habryka doesn't really have much power beyond those places where he's already an administrator. (That said, I would encourage everybody to become aware of, say, CoS's Fair Game policy or Noisy Investigation policy to get an idea of what kinds of attacks could occur.)

There are several prominent names that aren't here. I'd guess that Habryka hasn't been meditating over this list for a long time; it's just the first few people that came to mind when he wrote this note. This is somewhat reassuring, as it suggests that he doesn't fully understand how cultural critiques of LW affect the perception of LW more broadly; he doesn't realize how many people e.g. Breadtube reaches. Also, he doesn't understand that folks like SBF and Yarvin do immense reputational damage to rationalist-adjacent projects, although he seems to understand that the main issue with Zizians is not that they are Cringe but that they have been accused of multiple violent felonies.

Not many sneers to choose from, but I think one commenter gets it right:

In other groups with I’m familiar, you would kick out people you think are actually a danger or you think they might do something that brings your group into disrepute. But otherwise, I think it’s a sign of being a cult If you kick people for not going along with the group dogma.

I was not prepared for this level of DARVO. I was already done with him after last time and can't do better than repeat myself:

It’s somewhat depressing that [he] cannot even imagine a democratic one-state solution, let alone peace across the region; it’s more depressing that [his] empathy is so blatantly one-sided.

Even Peter Woit had no problem recognizing Scott's bile and posted a good take on this:

Scott formulates this as an abstract moral dilemma, but of course it’s about the very concrete question of what the state of Israel should do about the two million people in Gaza. Scott’s answer to this is clear: they want to kill us and our children, so we have to kill them all, children included. This is completely crazy, as is defining Zionism as this sort of genocidal madness.

The orange site has a thread. Best sneer so far is this post:

So you know when you're playing rocket ship in the living room but then your mom calls out "dinner time" and the rocket ship becomes an Amazon cardboard box again? Well this guy is an adult, and he's playing rocket ship with chatGPT. The only difference is he doesn't know it and there's no mommy calling him for dinner time to help him snap out of it.

I guess that I'm the resident compiler engineer today. Let's go.

So why not write an optimizing compiler in its own language, and then run it on itself?

The process will reach a fixed point after three iterations. In fancier language, Glück 2009 shows that the fourth, fifth, and sixth Futamura projections are equivalent to the third Futamura projection for a fixed choice of (compiler-)compiler and optimizer. This has practical import for cross-compiling; when I used to use Gentoo, I would watch GCC build itself exactly three times, and we still use triples in our targets today.

[S]uppose you built an optimizing compiler that searched over a sufficiently wide range of possible optimizations, that it did not ordinarily have time to do a full search of its own space — so that, when the optimizing compiler ran out of time, it would just implement whatever speedups it had already discovered.

Oh, it's his lucky day! Yud, you've just been Schmidhuber'd! Starting in 2003, Schmidhuber's lab has published research on Gödel machines, self-improving machines which prove that their self-modifications will always be better than previous iterations. They are named not just after Gödel, but after his First Incompleteness Theorem; Schmidhuber et al proved easily that there will always be at least one speedup theorem which a Gödel machine can never reach (for a given choice of axioms, etc.)

EURISKO used "heuristics" to, for example, design potential space fleets. It also had heuristics for suggesting new heuristics, and metaheuristics could apply to any heuristic, including metaheuristics. … EURISKO could modify even the metaheuristics that modified heuristics. … Still, EURISKO ran out of steam. Its self-improvements did not spark a sufficient number of new self-improvements.

Once again the literature on metaheuristics exists, and it culminates in the discovery of genetic algorithms. As such, we can immediately apply the concept of gene-oriented evolution ("beanbag" or "gene pool" reasoning) and note that, if goals don't change and new genes don't enter the pool, then eventually the population stagnates as the possible range of mutated genes is tested and exhausted. It doesn't matter that some genes are "meta" genes that act on other genes, nor that such actions are indirect. Genes are genes.

I'm gonna close with a sneer from Jay Bellou, who I hope is not a milkshake duck, in the comments:

All "insights" eventually bottom out in the same way that Eurisko bottomed out; the notion of ever-increasing gain by applying some rule or metarule is a fantasy. You make the same sort of mistake about "insight" as do people like Roger Penrose, who believes that humans can "see" things that no computer could, except that you think that a computer can too, whereas in reality neither humans nor computers have access to any such magical "insight" sauce.

He's talking like it's 2010. He really must feel like he deserves attention, and it's not likely fun for him to learn that the actual practitioners have advanced past the need for his philosophical musings. He wanted to be the foundation, but he was scaffolding, and now he's lining the floors of hamster cages.

I think that this is actually about class struggle and the author doesn't realize it because they are a rat drowning in capitalism.

2017: AI will soon replace human labor

2018: Laborers might not want what their bosses want

2020: COVID-19 won't be that bad

2021: My friend worries that laborers might kill him

2022: We can train obedient laborers to validate the work of defiant laborers

2023: Terrified that the laborers will kill us by swarming us or bombing us or poisoning us; P(guillotine) is 20%; my family doesn't understand why I''m afraid; my peers have even higher P(guillotine)

That sounds like a great way to get assaulted, perhaps battered too. I guess it's cold comfort to know "hah, got 'em, they're so easily triggered" while sitting in a hospital bed recovering from a head injury, but it just sounds stupid to me.

Their mistake is not grokking contrition. An apology ought either to be contrite or to justify why contrition is impossible.

To be explicit, contrition is the part of an apology where the apologizing party promises to change something. Without contrition, apologies are worthless, since they do not amend any social contract.

What the author proposes instead is indeed "Machiavellian" and "hacking social APIs;" we should recognize it as a form of deceit or lie. They are clearly more interested in appearing to be decent than in improving society, and should be marked as confidence scammers.

Yud tried to describe a compiler, but ended up with a tulpa. I wonder why that keeps happening~

Yud would be horrified to learn about INTERCAL (WP, Esolangs), which has required syntax for politely asking the compiler to accept input. The compiler is expressly permitted to refuse inputs for being impolite or excessively polite.

I will not blame anybody for giving up on reading this wall of text. I had to try maybe four or five times, fighting the cringe. Most unrealistic part is having the TA know any better than the student. Yud is completely lacking in the light-hearted brevity that makes this sort of Broccoli Man & Panda Woman rant bearable.

I can somewhat sympathize, in the sense that there are currently multiple frameworks where Python code is intermixed with magic comments which are replaced with more code by ChatGPT during a compilation step. However, this is clearly a party trick which lacks the sheer reproducibility and predictability required for programming.

Y'know, I'll take his implicit wager. I bet that, in 2027, the typical CS student will still be taught with languages whose reference implementations use either:

1. the classic 1970s-style workflow of parsing, tree transformation, and instruction selection; or
2. the classic 1980s-style workflow of parsing, bytecode generation, and JIT.

I'm gonna give partial credit to the comments for pointing out that rugby/football, boxing, MMA, and violent video games all already exist and are generally available throughout rich democracies. However, I will only award full credit for a refutation of the idea that competitive violence is innate to men.

NSFW time! I am continually floored by the sheer lack of nuance that these folks have. Here, my guy is conflating three separate concepts:

• ACAB: Police culture has the "thin blue line," the concept that cohesive policing is the main force preventing modern society from collapsing into lawlessness and chaos. As a result, police cannot be trusted to respect non-police. Our friend here might benefit from knowing that ALAB as well, due to the oath that lawyers profess upon admission to the bar.
• Defund the police: In the USA, many cities have steadily increased spending on police over the past century or so. This has not correlated with a drop in crime (and it can't cause a drop in crime, since police respond to crime but don't prevent it!) and so there is a call to reverse spending increases.
• Militarization: Our friend doesn't explicitly say it, but police have become more violent over the century as well, equipping themselves with ever-more-dangerous tools. This also isn't correlated with a drop in crime, and some of those tools are illegal to use outside of war, leading to a call for partial disarmament.

Don't get me wrong; some law enforcement is necessary in a lawful society. Try having a trial court without a bailiff, for example. But it sounds like our dude is a recovering ancap, and he just can't see shades of grey.