Empire of AI: “Gods and Demons” and “Apex” (Patreon)
Content
so strictly speaking this question isn’t about the reading (though the first chapter’s mention of OpenAI collaborating with Sal Khan of Khan Academy did bring up related thoughts for me), but since we’ve all been reading/thinking about AI, and you’re all ostensibly viewers of science-y youtube, I'm curious to hear if anyone has thoughts on the current Hank Green AI drama. I definitely have opinions seeing it pay out as a creator, but I’m wondering how the whole thing is coming off for viewers. No need to respond to this if you’re not interested, but I welcome your takes~
~~~
On to the actual text: in trying to figure out what chapter i was on i accidentally read the last paragraph of “Gods and Demons”, and damn was it a way to start off this weeks reading:
“Then Sutskever emerged. In the pit, he had placed a wooden effigy that he’d commissioned from a local artist, and began a dramatic performance. This effigy, he explained, represented a good, Aligned AGI that OpenAI had built, only to discover it was actually lying and deceitful. OpenAI’s duty, he said, was to destroy it. Only a few yards away, several redwoods stood like ancient witnesses in the darkness. Sutskever doused the effigy in lighter fluid and lit it on fire.”
As if the chapter’s other comparisons to burning man weren’t on the nose enough…
I found the explicitly religious framing of “Gods and Demons” to be really interesting—I often feel that a certain kind of technocratic belief is just a christian worldview with some one to one swaps of where faith goes and what is the source of evil, but Hao really outlines what that looks like in the current AI landscape really explicitly. The doomer/boomer divide stuff made this feel especially literal:
“...as much as each ideology professed to be the opposite of each other, both were in fact preaching from the same bible. Both discussed AGI as an increasingly forgone conclusion and with a religious ferocity; both fixated on the long term and asserted a moral authority to keep AI development within the control of its adherents. Where one warned of fire and brimstone, the other tantalized with visions of heaven.”
The reason I find this pseudo-religious framing of so interesting but also frustrating is because of how hypocritical it is—the average technocratically minded individual will be very quick to dismiss others’ perspectives that are influenced by “irrational” beliefs around god or people or whatever, but are totally comfortable with blind faith in the goodness/importance of their own work via a blind faith in the goodness/importance of technology. From the circular reasoning of OpenAI leadership in earlier chapters, I don’t think the tech crowd sees this as a matter of faith, but the further we get into this book, the more obvious it seems to me.
I think the story around the Google engineer who believed LaMDA was sentient and the comparison to Koko the gorilla illustrated this especially well. I remember when that story came out in 2022, I kept hearing about it from people who pretty much took it at face value, and I thought, well if this expert believes AI to be sentient there must be something to it. But then I looked at the transcript of his conversations with LaMDA and couldn’t wrap my head around how a person ostensibly intelligent enough to be building this kind of advanced software would also just… believe it? I didn’t have the context of his “mystic Christian priest” thing back then, but I just didn’t get where this faith was coming from. I wonder if it has something to do with being tied up in the creation of the tech—maybe Sutskever and the like have some psychological bias to believing AI is sentient/godlike because of what it would mean for them as its creator?
~~~
As for “Apex”, the only line I highlighted in that chapter was “They knew OpenAI was going downhill once it started hiring people who could look you in the eye.” Is this an autism thing?? I immediately have so many thoughts bc the depoliticization of autistic people in STEM is something I’ve thought about a lottt, especially in the context of people referencing their autism to absolve them of any kind of moral/social responsibility (see, that Google engineer that wrote that sexist memo about women in tech, or basically the entire career of Elon Musk—that sieg heil was just him stimming!). It would be very in line with that whole vibe for the og OpenAI crowd to see the hiring of “people who could look you in the eye” as the introduction of unnecessary and cumbersome social/emotional concerns to their workplace. Idk, I’ve had thoughts to do a video about this for a long while, curious to hear yalls thoughts based on this chapter/in general.