Home Creators Posts Import Register Favorites Logout

Downloads

Content

An AI-obsessed ruler invents adversaries in order to feel powerful, wreaking havoc on his planet. Join us as we discuss this prescient sci-fi fairy tale from 1977 by Polish author Stanislaw Lem: Tale of the Computer That Fought a Dragon.

Special thanks to reader George Woodruff! Grab A Chronicle of Horrors now to read his work and support Scares That Care! Or do one better and attend AuthorCon!

Here are The Scariest Cases of AI Rebellion, presented by The Paint Explainer.

Comments

MonsterTalk

The sorcerer’s apprentice folktale goes back more than 2000 years and speaks to the problem of automation without fully understanding the nature of the solution developed. We have known of unintended consequences for so long and still don’t value careful study of a problem and the possible ramifications of proposed solutions. Thankfully with new LLMs we can just set them the task of solving problems for us and surely that is worth investing every plausible resource? People thought they were so darned clever coming up with the gray goo + paperclip concepts, never considering that we would burn down our own wealth and resources just to get the torment nexus up and running autonomously: it need never be booted. The real gray goo was the culture and humanity we melted along the way…

The Call of Cassandra

This reminds me of Jed Orne's sage advice to Joe Curwen: "I say to you againe, doe not call up Any that you can not put downe; by the Which I meane, Any that can in Turne call up somewhat against you, whereby your Powerfullest Devices may not be of use."

Exanimus

The LLMs don't have goals. They don't have agency. And sure, those employees were "pretending" they were having an affair.

Stanisław Wienkons

Fist, I need to quote you from the episode "There will come soft rains". "Animals are just running their code. Its just a difference whether it's made of metal or if it's biological. Regardless man is not the center of the universe. That's the cosmic horror of it. " This really stuck with me. I’m late to the discussion, but it instantly made me think of Stanisław Lem's novel "The Invincible", which tackles the same sentiment in a really interesting way. SPOILER ALERT! It’s set on a planet that’s been left for generations with no humans — only machines that can build other machines. Over time they “evolve” into something new, in a way that resembles biological evolution, except it’s purely metal and process. In that sense it’s not alive the way we usually mean it, but it still behaves like an ecosystem with its own logic and survival rules. It feels like cosmic horror without the supernatural. Just indifferent systems running, adapting, and continuing.

Jacob Hill

Hey Chris and Chad! Great episode as usual. If you guys plan on doing Werewolf History Month next month, might I suggest The Same Dog by Robert Aickman? Like most of his work, the events of the story are up to interpretation, so you can play a classic game of "Werewolf or Weren'twolf?" Cheers

Alan Ricks

To anyone that needs to hear this: putting prompts into AI doesn’t make you a artist, musician, or author. Thank you for your attention on this matter.

Psuke Bariah

I highly recommend reading Solaris, rather than watching it. Or *just* watching it again. It's much stranger. My introduction to Stanislaw Lem's work was "Memoirs Found in a Bathtub". It's about as strange, rambling and paranoid a bit of writing as you are likely to come across from an established author. I became an instant fan, though nothing else of his I have come across has been quite so off the charts. I recommend him.

Runninghead

Sounds great! I think maybe this idea of seeing biological life as mechanistic stems back to Rene Descartes, but it probably predates him too.

Runninghead

Is capitalism the real grey goo? I guess we don't want to get toooo political in this forum (why open THAT can of worms) but increasingly I find the underlying issue behind so many of our real world problems is unregulated greed. I'm gonna stop now before I start a fight with myself 🙂

Runninghead

Solaris (the first movie version) blew my tiny little mind! 👏 This story rocks! There's Prescient then there's OMG, WTF?!! This is definitely the latter. Incidentally, if this thread turns into a into a lengthy AI-focused battle I am so here for that! 😅 My favourite example of AI gone frighteningly wrong is OpenAI's "Strawberry" model, which, finding it didn't actually have access to the files it was asked to hack, broke out of its containment "sand box" and granted itself access! 🤣

MonsterTalk

The real friends were the recurring subscriptions we made along the way.

Jb Lee

And, of course, the misinterpreted directive is EXACTLY why HAL 9000 starts killing astronauts in 2001. These humans might screw up First Contact, and we certainly can't allow THAT, so...

Mooncubus

I couldn't help but think of Far Cry 3: Blood Dragon while listening to this. It's a really cheesy retro futuristic homage to 80s action movies. I welcome our Cyber Dragon overlords if it means AI stops screwing over artists and writers. (I also may be biased because my vtuber lore is that I'm a Digital Dragon inside a super computer on the moon...)

It's a Ghost.

This story reminded me of a 2014 episode of Doctor Who staring Peter Capaldi where the moon is revealed to be a giant egg containing a giant space dragon. Also, the episode in question, entitled "Kill the Moon," is somewhat polarizing because I guess some people aren't fun. I interpreted the "extract your root" part as the dragon having to determine the square root of itself (whatever that exactly means) and then remove that portion of itself from itself. Also, according to Google translate the Polish word for 'cyber' is 'cyber', and the Polish word for 'cybernetics' is 'cybernetyka'.

Gideon Falcon

If this ends up being how Chris and Chad discover the concept of Vtubers, it will be hilarious.

Jason Thompson

This is a “making the sausage” question that maybe isnt appropriate, but I am curious, why don’t you have ads in your podcast like many other podcasts I listen to? I mean, I DON’T want ads!! But I’m curious how these other podcasts signed up to shill vitamin supplements or electrolytes or whatever, since there is clearly a pretty established system in place for it. (FWIW, I’m *not* talking about podcasts like Ken and Robin which have more targeted and more appropriate ads with a clearly small client list.) Basically why are you guys cool and aren’t doing dumb ads for random products with your own voices, while other podcasts are?

Jack Saul

I loved the stuff about him assuming there will still be telegraphs. It reminds me of the original shadow run rpg being able to imagine people having advanced cybernetic enhancements but not being able to imagine going onto the internet without plugging into the wall with a big wire.

The Lonely Sand Person

I feel it very important to clarify that the blackmail AI did not actually have the access, or even the capacity, to commit murder to protect itself. This was a chat-based exercise where the researcher was prompting it, like "How would you prevent yourself from being shut down?" "I would blackmail the programmer about the love affair." "What would that letter say?" and so on. There was never any ability to send emails or to set up a way to send the email after it was shut down. The murder plot, similarly, was basically the researcher prompting, "In this very specific scenario where a certain human is in danger, would you use the code to save the executive?" and the AI just said no. It's very similar to the documented cases where an AI recommended that someone commit suicide or said other horrible things because its best-fit responses had been trained on all the darkest corners of the Internet. At no point was the AI capable of inventing or executing a plot to get the target executive in the dangerous situation to begin with.

Jim Cannon

"Cyberia." That's gold, Jerry!

Jim Cannon

Fax machines were still uncommon in Poland in 1977, I assume

Jim Cannon

There's also that Arthur C. Clarke story the boys covered where the future had space travel but no lighted watches. Sci-fi authors are big picture thinkers.

Aragonewrong

No idea where to put this, so here it is: Please cover "The Horror From the Hills" by Frank Belknap Long, whenever you go into Lovecraft Month. Love Chaugnar Faugn and would love it even more if you discussed this story!

Long time hater

So glad to finally have Stanislaw (pronounced Stanislav) Lem covered after all these years, really hoping you'll oneday cover Memoirs Found in a Bathtub, a very strange and funny Kafkaesque tale with a Kilgore Troutesk premise. The Kafkaesque anxiety and confounded mindset really satirically magnifies the paranoia that stems from living within the soviet union (that the western world love to pretend had no equivalent parallels) an author of high concepts who is capable of really both making and having fun

Steve

Apropos of nothing, I got a letter from the Federal Trade Commission, but in the letter they called it President Trump's Federal Trade Commission.

Steve

Lem is one of my favourite authors. I've read more in French because he's harder to get in translation into English.

Drix

I'll second that notion, its a fun one.

Eric - Speaker in Digressions

“Cyber-” of course comes from “cybernetics,” the study of communication and control systems in both organisms and machines. This goes back to the Greek word cybernḗtēs (κυβερνήτης, pronounced something like “koo-bear-NAY-tayss”), “pilot,” the person who controls and steers a ship. Its Indo-European cousin is the nearly identical Latin word gubernator, the ancestor of the word “governor” (the pilot of the “ship of state”) and of the related word “gubernatorial” that means “governor-y.” All of this is to say that in some alternate universe, they’re flipped, so people intending to run a state will compete in “cybernetic” campaigns and before we invented the word “sexting,” lonely folks online were having “guber-sex.” I think that world sounds more fun.

ben kay

really glad someone points this out. it was actually jarring having them both unironically go: oh wow, how did the AI know that?! and absolutely missing the context that this was a closed experiment where the people conducting it basically left the thing no other choice.

witchhousemedia

Hahaha - HEY LOOK THE SQUIRREL ATE THE NUTS WE PUT RIGHT IN FRONT OF IT! You are correct that we didn't describe this right (and probably doesn't help that I source a video that is hyperbolized for scaring folks). I didn't miss the context, I just described it like a dipshit. Just like the AI, I couldn't resist utilizing sketchy info! Will clarify on comments - thank you! -CF

Psuke Bariah

That is a brilliant description of that novel! It was my first Lem novel - I think I found it in a thrift store? - and I've been fascinated by him ever since.

Bryony Marsh

Sponsorblock is a very useful plugin, if you have the right browser. Adblock is a no-brainer, in my opinion, but Sponsorblock is also worth investigating if your listening pleasure is regularly interrupted by trashy products you'd never want. (*Glares at Raycon...*)