I love this. Especially the ending, talking about the titanic struggle to make AI competent enough to outsmart the people who think it's going to be omniscient. Glad to see I've got another writer to chase down that I had somehow missed previously.
I've avoided reading Greg Egan until like last year because I entirely expected him to be a cold stemlord shithead and people only talk about his earlier books that have more to do with consciousness and identity and stuff, which these days feels very zzzzz, but he is SO COOL and SO FUN!!! He cares in a deep way about people, lived experience, about societies, he loves physics and maths in themselves because they're beautiful and fun and not because they're ways to look smart or reveal the secrets of the universe, his books are very beautiful. Complete opposite of Yud, Scott, nostalgebraist (I have a grudge) et al.'s silly books.
@Amoeba_Girl @Soyweiser I read https://en.wikipedia.org/wiki/Permutation_City and found something about it seemed deeply wrong in ways that I had trouble articulating.
It's like when you see a bogus mathematical proof of a statement that you know to be false, but the mistake is hidden deep and you can't tell where it has gone wrong, you just know it has.
@bencurthoys @Amoeba_Girl @Soyweiser I'm pretty sure that about 10-20 years ago Egan came out with a serious repudiation of his own ideas about achieving AI through iterated simulations of less-intelligent entities: he noted that implementing it was implicitly genocidal (by murdering all entities that didn't *quite* meet some threshold set by the experimenters, you'd inevitably kill huge numbers of sentient beings just for failing an arbitrary test).
@cstross @Amoeba_Girl @Soyweiser My usual handle when playing online games is "Bickel", because I happened to be re-reading "Destination: Void" at the time that I first signed up my World Of Warcraft account, and killing huge numbers of sentient beings in the pursuit of artificial consciousness was definitely not a problem for Frank Herbert =)
Herbert is so obsessed with his particular vision of eugenics it ends up back being endearing. Look at our big boy building his big torture worlds just so they can roundaboutly excrete one superman. Such a specific, endlessly restated fetish.
There's a fun/horrifying scene in Ken McLeod's Stone Canal where the protagonists revive superhuman intelligences from cold storage, get the answers they need from them, then destroy them with nanotech the superhumans have not developed defenses against. As one of them says when confronted: "standard programming practice, keep the source code, blow away the object code".
(It's partially justified that if left alone the superintelligences will just iteratively bootstrap themselves into catatonic insanity anyway)
@Amoeba_Girl @Soyweiser What book(s) of his would you recommend as starters, for those who have not read him?
The Orthogonal trilogy is really really great, very imaginative exploration of a wild concept, and shockingly sharp sexual politics for a male author!
He has a lot of excerpts and some full stories on his website, so one way to start might be browsing there.
I think my favorite novels of his that I've read were Zendegi and Incandescence.
I thought Dichrononauts was batshit-fun world building and I enjoyed The Four Thousand, the Eight Hundred.
@blakestacey @5teverin0
I'll second Zendegi as a great place to start. Also really liked the Orthogonal trilogy.
@5teverin0 @Amoeba_Girl @Soyweiser
First thing I read from him was the short story collection "Luminous" and I still think that's a great entrypoint because it touches on many of his subjects.
I didn't read his more recent works though.
I had the Dark Integers story collection, but I lent it to a colleague and they haven't given it back.
I saw people complain his characters had no debt and character or something because they all were so agreeable, and I was a bit confused. (I have not read any of his earlier work, some of it I wanted to read but never got around to, mentally pushed it further upward now) but it was odd to see that comment after reading this short story. I mean yes they were agreeable (after all they had to work together) but it revealed a lot of character. This bit alone: '“So what do you call mine?” Ken asked bravely. “Peak Conformist,” Helen replied. Ken laughed, unoffended.'
Thanks for posting this, it was entertaining.
This was a year before the stuff with the Zizians happened too.
And the healthcare CEO.
From the comments of the LW article.
"I like and admire both Charles Stross and Greg Egan a lot but I think they both have "singularitarians" or "all of their biggest fans" or something like that in their Jungian Shadow.
I'm pretty sure they like money. Presumably they like that we buy their books? Implicitly you'd think that they like that we admire them. But explicitly they seem to look down on us as cretins as part of them being artists who bestow pearls on us... or something?"
It took me one (1) science-fiction convention to discover that liking the same TV show as somebody does not mean we vibrate on the same soul wavelength. I imagine that professional writers learn rather quickly that just because somebody bought your book doesn't mean that you want to spend time with them.
Yes the rationalists are an incredibly large market and their opinion can make or break an author, sure you betcha
Nit: It's "Death and the Gorgon".
It's linked here, so I'll hazard a guess that the copy is intended to be public.
Thank you, fixed. And thanks for looking it up.
Hurling ordure at the TREACLES, especially those closely related to LessWrong.
AI-Industrial-Complex grift is fine as long as it sufficiently relates to the AI doom from the TREACLES. (Though TechTakes may be more suitable.)
This is sneer club, not debate club. Unless it's amusing debate.
[Especially don't debate the race scientists, if any sneak in - we ban and delete them as unsuitable for the server.]
See our twin at Reddit