This might sound very strange, but where

divinity or God fits into the pursuit or

development of superintelligence for

different players in the space, if it

does.

And the reason I bring that up is that

religion does recur in the book both in

the personal story of Demis, but

elsewhere, and it shows up repeatedly in

so much as I'll give you one example,

"The closest to Sabbas had come to

landing a real investor was an eccentric

financier named David Gammon. I want to

hear more about this guy, also.

>> [laughter]

>> Financier seemed open to making this

unusual bet um I'm lighting a few things

because his motives were themselves

unusual. Quote, 'There's a deeply

religious aspect to AGI,' Gammon

explained to me later.

'It's really finding God's algorithm.'

I think

it would seem at least chatting with

people in Silicon Valley that there are

some who take it even further, right?

Maybe this is how we find God. Maybe

this is how we actually elicit the

second keep coming. I mean, there's a

lot there. I'm just wondering to what

extent this has popped up in your

research, whether it's reflected in the

book

or not.

>> I think there's one basic thing going on

here, and I'm going to take a slight

detour, but it answers your question.

>> Sure.

>> What we're dealing with

with AGI, powerful intelligence that

rivals human cognition,

is something that's so powerful that

it's both exciting and scary and just

hard to get your mind around. And so,

if you look, for example, at the 2009

speech that caused the foundation of

DeepMind. This was Shane Legg, Demis's

co-founder, who gave a talk in 2009

about how superintelligence would arrive

in 2030. So, unbelievably spot-on

prediction. And towards the end of that

lecture, which is captured on a grainy

video online, you see him pivot from

explaining how algorithms are getting

stronger, there's more data online,

computers getting more powerful, and so

we're heading towards this intelligence

explosion, and then he says,

"And it's going to be threatening. It's

going to do things we can't control.

It's going to be human level. It might

challenge us." And as he says this, he

has this sort of excited smile on his

face.

And you think, "Well, that's a bit

strange, you know, he's talking about

potential doom, and he's smiling."

And then somebody in the audience says,

"Wait, wait, wait. You've just told us,

Shane, that this could be threatening to

humanity,

and you haven't provided any antidote,

and surely you're going to tell us how

we're going to stop it." At which point

Shane turns around and says, "How do we

stop it?"

And he's kind of giggling.

And you think, "Why are you laughing at

this dangerous thing?"

And you realize that for humans to

contemplate annihilation is absurd.

And the absurd is a close cousin of

humor.

>> Mhm.

>> And the reason I tell this story is that

it's a springboard to the religion

point, which is that

this is such a hard thing to think about

that people reach for religious

terminology

when they're around AI. They just do it

naturally.

So, you know, there's this story about

Ilya Sutskever, the who was the chief

scientist at OpenAI. I talked to him a

lot for this project. And there was a

point when he was at a retreat with his

fellow scientists, and

they were gathered in the evening around

a fire pit.

And he was talking about safety, and he

said, "Okay, I want to explain to you,

we might have an AI that's dangerous. It

wouldn't be aligned with us. So, here's

what we're going to do with it." And he

produced an effigy,

which was supposed to represent a malign

AI.

And he put it into the fire pit, and he

burnt it like a medieval cleric putting

a witch to death.

And so, that's just one example of this

religion. I'll give you another one.

So Demis one day was sitting with me in

a park in North London. We would meet

for 2 hours at a time and we would get

deep into stuff. There was a another

picnic table next to us where two people

having a normal quotidian conversation

about some friend of theirs who'd gone

to hospital

and was she better, was she okay, etc.

etc. I was seated opposite Demis who had

gone into this riff about how he reads

scientific papers from after his kids go

to sleep in the evening from 10:00 p.m.

until 4:00 a.m.

And as he's reading these papers, he

says to me, "Reality is staring at me,

screaming at me,

calling at me to understand it.

And I have to understand it and if I can

understand it,

it's like understanding nature better

and therefore understanding the

intelligence that might have created

nature and I would be closer to what I

would call God."

And so for him it's a kind of quasi

scriptural quest

to build the artificial intelligence.

For Ilya,

it's a way of expressing the power of

the artificial intelligence. There's the

story of Levandowski, I forget his first

name now, but the early early engineer

at what became Waymo later,

started a kind of church

in worship of AI because AI is so

omniscient that it's kind of like a god.

Marc [clears throat] Andreessen

lampoons those who believe in sort of

some

ethereal second coming, a kind of

rapture

where AI will, you know, will have a

singularity, uh the AI will go vertical

in its rate of improvement

and the whole world will change and he

likens that to kind of Christian

kind of messianism.

So yes, all through this topic, there is

this religious expression because

religion is the lexicon

for dealing with something that we find

too mysterious to really understand.