2

So seeing the reaction on lesswrong to Eliezer's book has been interesting. It turns out, even among people that already mostly agree with him, a lot of them were hoping he would make their case better than he has (either because they aren't as convinced as him, or they are, but were hoping for something more palatable to the general public).

This review (lesswrong discussion here), calls out a really obvious issue: Eliezer's AI doom story was formed before Deep Learning took off, and in fact was mostly focusing on more GOFAI than neural networks, yet somehow, the details of the story haven't changed at all. The reviewer is a rationalist that still believes in AI doom, so I wouldn't give her too much credit, but she does note this is a major discrepancy from someone that espouses a philosophy that (nominally) features a lot of updating your beliefs in response to evidence. The reviewer also notes that "it should be illegal to own more than eight of the most powerful GPUs available in 2024 without international monitoring" is kind of unworkable.

This reviewer liked the book more than they expected to, because Eliezer and Nate Soares gets some details of the AI doom lore closer to the reviewer's current favored headcanon. The reviewer does complain that maybe weird and condescending parables aren't the best outreach strategy!

This reviewer has written their own AI doom explainer which they think is better! From their limited description, I kind of agree, because it sounds like the focus on current real world scenarios and harms (and extrapolate them to doom). But again, I wouldn't give them too much credit, it sounds like they don't understand why existential doom is actually promoted (as a distraction and source of crit-hype). They also note the 8 GPUs thing is batshit.

Overall, it sounds like lesswrongers view the book as an improvement to the sprawling mess of arguments in the sequences (and scattered across other places like Arbital), but still not as well structured as they could be or stylistically quite right for a normy audience (i.e. the condescending parables and diversions into unrelated science-y topics). And some are worried that Nate and Eliezer's focus on an unworkable strategy (shut it all down, 8 GPU max!) with no intermediate steps or goals or options might not be the best.

top 21 comments
sorted by: hot top controversial new old
[-] Architeuthis@awful.systems 1 points 1 month ago* (last edited 1 month ago)

I'm still not sure if they actually grasp the totalitarian implications of going ham on tech companies and research this way. He sure doesn't get called out about his 'solutions' that imply that some sort of world government has to happen that will also crown him Grand Central Planner of All Technology.

It's possible they just believe the eight [specific consumer electronic goods] per household is doable, and at worst equally authoritarian with the tenured elites snubbing their noses at HBD research.

[-] istewart@awful.systems 0 points 1 month ago

You thought a right-and-proper Communist Five-Year Plan couldn't also be a self-insert fanfic? Hold Yud's beer

[-] swlabr@awful.systems 0 points 1 month ago

did you hear about Yudoslavia? They have this Eight GPU policy, if a household is about to have another GPU and they find it can’t run Crysis at max settings at 60fps, they leave it outside for the wolves. But this might just be Yudophobic propaganda

[-] scruiser@awful.systems 1 points 1 month ago

In Eliezer's "utopian" worldbuilding fiction concept, dath ilan, they erased their entire history just to cover up the any mention of any concept that might inspire someone to think of "superintelligence" (and as an added bonus purge other wrong-think concepts). The ~~Philosopher Kings~~ Keepers have also discouraged investment and improvement in computers (because somehow, despite now holding any direct power and the massive financial incentives and dath ilan being described as capitalist and libertarian, the Keepers can just sort of say their internal secret prediction market predicts bad vibes from improving computers too much and everyone falls in line). According to several worldbuiding posts, dath ilan has built an entire secret city that gets funded with 2% of the entire world's GDP to solve AI safety in utter secrecy.

[-] dgerard@awful.systems 1 points 1 month ago

tell me you didn't subject yourself to glowfic

[-] swlabr@awful.systems 0 points 1 month ago

Thanks for the lore, and sorry that you had to ingest all that at some point.

Like a lot of sci-fi that I will comment on without having read, that sounds like a Dune ripoff (that I also have not read). Except, rather than "ok we tried the whole AI thing and it turned out bad", Yud is saying "my world is smarter because they predicted AI would be bad, so they just didn't try it, neener neener neener!"

Also, iterating on the name origins of "dath ilan," as I've said before, it's an anagram of Thailand. But I have a new hypothesis. It's formed from "Death Island" minus the letters "ESD". What could this mean?!?!?! Electrostatic discharge? "Eek, Small Dragon?" "Eliezer Sudden Death?" The abyss pondering continues...

[-] scruiser@awful.systems 0 points 1 month ago* (last edited 1 month ago)

Thanks for the lore, and sorry that you had to ingest all that at some point.

Ironically, one of the biggest lore drops about dath ilan happens in a story I initially thought at the time was a parody of rationalists and the concept of dath ilan (Eliezer used a new penname for the story). The main dath ilan character (isekai'd into an Earth mostly similar to our own but with magic and uh.. other worldbuilding conceits I won't get into here) jumps to absurd wild conclusion throughout basically every moment of the story, and unlike HJPEV is actually wrong about basically every conclusion she jumps to. Of course, she's a woman, and it comes up towards the ending that she is below average for dath ilan intelligence (but still above the Earth average, obviously), so don't give Eliezer too much credit for allowing a rationalist character to be mostly wrong for once.

I don't know how he came up with the name... other fanfic writers in rationalist-adjacent space have complained about his amateurish attempts at conlanging, so there probably isn't a sophisticated conlang explanation about phonemes involved. You might be on the right track guessing at weird anagrams?

[-] istewart@awful.systems 1 points 1 month ago

other worldbuilding conceits I won’t get into here

My invite-only internal prediction market is offering good odds that this is referring to either math pets or tentacles, and a solid parlay opportunity on it being both combined

[-] V0ldek@awful.systems 0 points 1 month ago

In Eliezer’s “utopian” worldbuilding fiction concept, dath ilan

In Eliezer's sweet fucking what now?

[-] scruiser@awful.systems 0 points 1 month ago

It would be only light cringey if he kept it as a fiction writing project not strongly linked to his other writings. But as Architeuthis says, he not only links to it frequently, he cites it like it was a philosophy paper or something that obviously everyone should have read. Example from our old reddit discussions: https://www.reddit.com/r/SneerClub/comments/x3pihv/to_be_fair_you_have_to_have_a_very_high_iq_to/

[-] V0ldek@awful.systems 0 points 1 month ago

Is there any place where I could find out wth this is but passed through a sneer filter already?

[-] scruiser@awful.systems 1 points 1 month ago

We've sneered about dath ilan before on the reddit sneerclub, and occasionally I work references to dath ilan's lore into sneers, but other than that no.

[-] swlabr@awful.systems 1 points 1 month ago

My current iteration of the etymology is that "dath ilani" anagrams to "HD Italian". As in, the dath ilani are an idealised version of Italians, making dath ilan a utopian version of Italy.

[-] blakestacey@awful.systems 1 points 1 month ago

Damn, these days you can't even post a recipe without a dath ilani guy getting mad in the comments

[-] prw@mastodon.sdf.org 1 points 1 month ago

@blakestacey I am not familiar with that Sith lord ..

[-] gerikson@awful.systems 0 points 1 month ago

This comment is gold:

I particularly agree with the point about the style being much more science-y than I'd expected, in a way that surely filters out large swathes of people. I'm assuming "people who are completely clueless about science and are unable to follow technical arguments" are just not the target audience. To crudely oversimplify, I think the target audience is 120+ IQ people, not 100 IQ people.

I haven't read the damn book and I never will, but I have a hard time imagining there's any modern science that can't be explained to 100IQ smoothbrains, assuming the author is good enough.

[-] Soyweiser@awful.systems 0 points 1 month ago* (last edited 1 month ago)

To be fair, you have to have a very high IQ to understand ~~Rick and Morty~~ If Anyone Builds It, Everyone Dies. The humor is extremely subtle, and without a solid grasp of theoretical physics most of the jokes will go over a typical viewer's head. (I'm doing a variant of this meme)

[-] scruiser@awful.systems 0 points 1 month ago* (last edited 1 month ago)

There's also Eliezer's nihilistic outlook, which is deftly woven into his parables-- his personal philosophy draws heavily from Godel Escher Bach, for instance. The fans understand this stuff; they have the intellectual capacity to truly appreciate the depths of his parables, to realize that they're not just entertaining- they say something deep about the nature of Intelligence. As a consequence people who dislike IABIED truly ARE idiots- of course they wouldn't appreciate, for instance, the motivation in Eliezier's existencial catchphrase "Tsuyoku Naritai!," which itself is a cryptic reference to Japanese culture. I'm smirking right now just imagining one of those addlepated simpletons scratching their heads in confusion as Nate Soares genius unfolds itself on their copy of IABIED. What fools... how I pity them. 😂 And yes by the way, I DO have a rationalist tattoo. And no, you cannot see it. It's for the math pet's eyes only- And even they have to demonstrate that they're within 5 IQ points of my own (preferably lower) beforehand.

[-] blakestacey@awful.systems 0 points 1 month ago

translator's note: IABIED means "plan"

this post was submitted on 20 Sep 2025
2 points (100.0% liked)

SneerClub

1201 readers
1 users here now

Hurling ordure at the TREACLES, especially those closely related to LessWrong.

AI-Industrial-Complex grift is fine as long as it sufficiently relates to the AI doom from the TREACLES. (Though TechTakes may be more suitable.)

This is sneer club, not debate club. Unless it's amusing debate.

[Especially don't debate the race scientists, if any sneak in - we ban and delete them as unsuitable for the server.]

See our twin at Reddit

founded 2 years ago
MODERATORS