151
Maybe the real unaligned super intelligence were the corporations we made along the way ๐ฅบ
(awful.systems)
this post was submitted on 15 May 2024
151 points (99.3% liked)
SneerClub
989 readers
17 users here now
Hurling ordure at the TREACLES, especially those closely related to LessWrong.
AI-Industrial-Complex grift is fine as long as it sufficiently relates to the AI doom from the TREACLES. (Though TechTakes may be more suitable.)
This is sneer club, not debate club. Unless it's amusing debate.
[Especially don't debate the race scientists, if any sneak in - we ban and delete them as unsuitable for the server.]
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It was more Soylent green, but it also is partially based on (friend of the club) the writings of C Stross yes. I think he also has a written lecture on corps being slow paper clipping AGI somewhere.
That is a beautiful comparison. Terrifying, but beautifully fitting.
I read Stross right after Banks. I think if I hadn't, I'd be an AI-hype-bro. Banks it the potential that could be, Stross is what we'll inevitably turn AI into.
stross' artificial intelligences are very unlike corporations though, and different between the books. the eschaton ai in singularity sky is quite benevolent, if a bit harsh; the ai civilization in saturn's children is on the other hand very humanlike (and the primary reason there are no meatsacks in saturn's children et al. is that humans enslaved and abused the intelligences they created).
Banks neatly sidesteps the "AI will inevitably kill us" scenario by making the Minds keep humans around for amusement/freshness. Part of the reasons for the Culture-Idiran war in Consider Phlebas and Look to Windward was that the Idirans did not want Minds in charge of their society.
Noone was trying to force that on them though, the actual reason IIRC correctly that Idirans had a religious imperative for expansion, and the Culture had a moral imperative to prevent other sentients' suffering at the hands of the Idirans.
IMO he mostly sidestepped the issue by clarifying that this is NOT a future version of "us"
OK I misremembered that part. It makes sense that after suffering trillions of losses the Culture would take steps to prevent the Idirans from doing it again.
And by "us" I meant fleshy meatbags, as opposed to Minds. Although in Excession he does raise the issue that there might be "psychotic" Minds. Gray Area's heart(?) is in the right place but it's easy to imagine them becoming a vigilante and pre-emptively nuking an especially annoying civilization.
@smiletolerantly @gerikson AFAIR there was a short story where the Culture takes a look at Earth around the 70ies and decides to leave it alone for now.
Yep. They leave us alone so we'll function as a control group. This way contact can later point at us and go "look! That's what happens if we don't intervene!"