I think the ability to learn is even

more important.

>> Yeah,

>> AI has really changed it. For example,

my startup when we interview a software

engineer honestly how much I personally

feel the degree they have matters less

to us now is more about what have you

learned what tools do you use how

quickly can you superpower yourself in

using these tools and a lot of these are

AI tools what's your mindset towards

using these tools matter more to

Dr. Lee, it is nice to see you. Thanks

for making the time.

>> Hi, Tim. Very nice to be here. Very

excited.

>> And we were chatting a little bit before

we started recording about how

miraculous and I suppose unfortunate it

is. It's somehow we managed to spend

three years on the same campus and

didn't bump into each other.

>> I know. And now I'm wondering which

college you were at and which clubs.

>> Oh, yeah. I was Forbes. I was in Forbes

College. No, I was Forbes, too.

>> Okay, this is for people who don't know

what the hell we're talking about. There

are these residential colleges where

students are split up when they come

into the school. And Forbes was way out

there in the sticks, right next to a

fast food spot like 7-Eleven called Waw

Wa

>> Waw Wa

>> and next to the commuter train. And then

there's something called eating clubs at

Princeton. People can look them up, but

they're effectively co-ed

fraternity/sorities where you also eat

unless you want to make your own meals.

And I was in Terrace.

>> I was not any of that. But for those of

you wondering why we didn't meet, we

should say we were very studious

students who were only in the libraries.

>> Yeah, we were very studious. I actually

made my whatever it was $6 an hour at

guest library working up in the attic.

>> Tim, I work in the same library. I don't

understand why we did not meet.

>> That's really hilarious. Okay. So, well,

now we're meeting.

>> Did you change name or something? Maybe

we did meet.

>> I did didn't change my name, but here we

are.

>> Yes,

>> we've reunited. That's wild that we

didn't bump into each other. I was also

gone for a period of time because I went

to Princeton in Beijing

>> and went to the what was it? Capital

University of Business and Economics

after that. And so I was gone for a good

period of time and then took a year off

before graduating with the class of

2000. So still we had a lot of overlap.

>> Yes.

>> But let's hop into the conversation and

this is a very perhaps typical way to

start but in your case I think it's a

good place to start which is just with

the basics chronologically.

Where did you grow up? And could you

describe your upbringing? Because based

on my reading, your parents were pretty

atypical for Chinese parents in my

experience. Certainly.

>> Yes. You know, a lot.

>> Yeah. Could you speak to that please?

>> I would say my childhood and leading up

to the formative years is a tale of two

cities. I grew up in a town in China

called Chundu. I was born in Beijing but

most of my childhood was spent in Chundu

where it's very famous for panda bears

and at the age of 15 my mom and I joined

my dad in a town called Precipity New

Jersey. So I went from a relatively

typical middleclass Chinese family,

Chinese kid

>> to become a new immigrant in a

completely different world of all

places, New Jersey,

>> to learn a new language, to learn a new

culture, to to embrace a new country.

Mhm.

>> And then from there on I went to

Princeton as a physics major, but I did

take some of the classes you took

and and then went to Caltech as a PhD

student to study AI and and the rest is

history. I want to hear about both your

parents, but I want to hear a little bit

about your dad because he seems like

based on my reading a very whimsical

sort of creative soul which is a sharp

contrast in some ways to for instance I

had Bo Sha on the podcast amazing

entrepreneur and his father was I

suppose what some folks might think of

when they imagine not a tiger mom but

like a tiger ad. So in the case of B's

upbringing, his father was very strict,

but if he if he, meaning Bo, won a math

competition, then he would get extra

love and he would be allowed to have

certain treats and things like that.

>> Could you just describe your parents a

little bit?

>> First of all, clearly you read my book.

Thank you for that. It is true. As a

child, you don't realize as I was just

going through my my own science memo, I

was writing it, the more I wrote about

it, the more I realized, oh my god, I

really did not have a typical dad.

>> My dad loved and still loves nature.

He's just a curious. He finds humor and

fun in unserious things, you know, like

he loves bugs, insects. Mh.

>> Growing up in the 1980s in China, there

isn't much abundance in terms of

material resources.

>> But my city Chundu was expanding. So we

lived in apartment complexes at the edge

of the city. Even though my dad and my

mom worked in the middle of the city. So

on the weekends, my dad and I would just

play in the fields where there's still

rice fields. There's water buffaloos. I

had a puppy. really all my memory is

just like fighting bugs really and then

sometimes my dad and I will follow some

I don't know we took a art class I took

a kids art class and we'll go to the

mountains neighboring mountains to draw

>> but my entire childhood memory of my dad

is is just a very unserious parent who

had no interest in my grades or what I'm

doing in class. Did I achieve anything?

Did I bring back any competition awards?

Nothing to do with that. Even when I

came to New Jersey with my parents, life

became extremely tough, right? It was

immigrant life. We were in a lot of

poverty. And even that my memory is that

he had so much fun in yard sales. Like I

would just go to yard sales and and

those are our every weekend it was just

yay let's go to yard sales and just use

that as a treasure hunt almost. So so

it's just he's a very curious and

childlike mind in that way.

>> I'm asking about your parents in part

because I know you're a parent and

ultimately I'm going to want to ask how

you think about parenting. that will

come up at some point. But since

listeners will certainly be asking

themselves this question and we're not

going to get into any geopolitics

because there are plenty of people who

want to get into that and fight over

that which we're not going to do. But

why did your parents leave China? like

what was the catalyst or what were the

reasons behind leaving what you knew or

leaving what they knew and coming to a

very different foreign country where I

you're going from Chungdu

which is a city to suburban New Jersey

which is as I think you've described it

felt very empty right and then you have

the language barriers and the financial

barriers there's so many things why the

move

>> I'll give you two answers in the early

teenage fay would say I I have no idea

>> because my dad left when I was 12 and my

mom and I joined him when I was 15 and

those years you're you're a teenager,

right? Like there's so many strange

things in your head and all I knew is

that you know they said let's go to

America and I had no idea. I really did

not know what happened. There was this

vague sense of there's opportunities and

freedom. there's this that education is

very different

>> and I had a hunch that I was not a

typical kid

>> in the sense that you know I was a girl

and

>> I loved physics I loved fighter jets of

all things you know I can tell you all

the fighter jets I love from F-17 to F16

to you know to all the different things

that I I loved. So that's all I knew. In

hindsight, as a grownup fa

>> I appreciated my parents. They're very

brave people because I don't know this

age myself would just pick up and and

leave a country I'm familiar with and go

to I don't know a completely different

country that I speak zero language and I

have zero connectivity to

>> and mind you that's pre- internet pre-

AI age so when you are

>> going to a different country you're comp

you're just you might as well go to a

different planet

>> you're cut of

>> Yeah.

>> Yeah.

>> I think they're very brave. The grown-up

Fay realized that they wanted me to have

an opportunity that that they think will

be unprecedented for for my education.

>> Mhm.

>> And it turned out that's that's kind of

true.

>> Yeah.

Well, certainly looking at your bio, I

mean, it's mind-boggling to imagine all

the different sliding door events and

different paths you could have taken.

So, we're going to hop pretty closely

along chronologically, but we're going

to ultimately get to a lot of the meat

and potatoes of the conversation, but I

want to touch on maybe some other

formative figures. And I would like to

hear about your mother as well, because

just with the context of your dad, it's

like, okay, that seems fascinating and

very unusual, particularly if you've

spent any time in China, especially

during that period of time.

>> He is very unusual that way. Yeah, very

unusual. So then people might wonder,

well, where does the drive come from?

Where does the technical focus come

from? And I'd love to hear your answer

to that and also hear you explain who

Bob Sabella was, if I'm pronouncing that

correctly.

>> Yes. There are two questions mostly, you

know, is my mom the one who put in the

drive and the technical passion and and

what role did Bob play in my life? So

first of all, my mom has zero technical

jeans here. I sometimes still laugh at

her. She cannot do math. Let's put it

this way. So I think the the technical

passion is just I was born with it

>> inate.

>> My dad is more technical, but he loves

bugs more than insects, more than

equations for sure. So I think that's

you know as an educator for so many

decades now myself and also as a parent

you have to respect the wonders of

nature there is this inner love and fire

and and passion and curiosity that comes

with the with the package right so

>> but my mom is much more disciplined

person she she's still not a tiger mom

in the sense I don't remember my mom

ever going after me on grades or she

really did not. My both my parents never

ever cared about me bringing any awards

home.

>> Mhm.

>> Maybe I did, maybe I didn't. But I can

tell you in our house there's zero wall

hangings of anything which actually

carried to today. Even for myself, my

own house, my own office have zero of

those decorations of achievements or

awards. It's just uh my mom did not care

about that. But she did care about me

being a focused person. If I want to do

something, she doesn't want me to play

while doing homework. And that kind of

thing would bother her. She would say,

"Just finish your homework. Say by 6:00

p.m. If you don't finish your homework,

you're not allowed to do more homework.

You have to deal with the consequences."

So she she instilled some discipline,

but that's about it. She's tougher than

my dad. She is very rebellious. She had

a unfinished dream herself. She was very

academic when she was a a kid herself

and cultural revolution really crushed

all her dreams.

>> She became a more rebellious person in

that sense that I think I did observe

and and experience as a daughter.

>> Maybe part of immigration is even part

of that. Many years later, she would

say, "I had no plan coming to New

Jersey, but I think I'm going to

survive. I just believe I'm going to

survive and I'm going to make sure Fay

survives."

>> I think that is her strength, her

stubbornness, and her rebelliousness.

>> When does Bob enter the picture and who

is Bob?

>> Bob Sabella was a high school math

teacher in Pipony High School. He he was

my own math teacher as well as many many

students. He entered my life in my

second year in per so it's kind of

bordering sophomore to junior year in

Persip high school when I started taking

AP calculus but he quickly became the

most influential person in my formative

years as a new American kid immigrant as

a teenager because he became my mentor

my friend and eventually his entire

family became my American family

>> and he became my friend when I was a

very lonely ESL English as second

language student.

>> I was excelling in math but I think it's

more because I was lonely and he was

very friendly. He treated me more like a

friend who talks about books we love,

talk about the culture, talks about

science fiction,

>> and also listened to me as a very, you

know, I wouldn't say confused, but

teenager undergoing a lot of life's

turmoil in in my unique circumstance.

And that unconditional support made me

very close to him and his family. And

one thing he did to me that I did not

appreciate till later is that when

Pimity High School couldn't offer a full

calculus BC class because it just didn't

have that, he just sacrificed his lunch

hour, his only lunch hour to teach me

calculus BC. So it was a one-to-one

class. And I'm sure that contributed to

me a immigrant kid getting into

Princeton eventually. But later as I

became teacher myself, it's exhausting

to teach all day long. And the fact that

on top of that, he would use his lunch

hours to do that extra class for me is

just such a gift that I now appreciate

more than I was as a as a teenager.

>> Yeah. Thank God for the teachers who go

the extra mile. It's just incredible,

especially when you

>> get a bit older and you have more

context and you can look back and

realize.

>> I really think these public teachers in

America are the unsung heroes of our

society because they are dealing with

kids of all backgrounds. They're dealing

with the changing times. the kind of

stories Bob would share with me in terms

of how he went extra miles not just with

me but with many students in because

Puberty is is a heavily immigrant town.

>> Mhm.

>> So his students are from all over the

world and how he helped them and their

family. It's just those are the stories

that people don't write about and that's

part of the reason I wrote the book was

to celebrate a teacher like that.

>> Yeah. I have so much I want to cover and

I know we're going to run out of time

before we run out of topics. So, I want

to spend more time on Bob and at the

same time I want to keep the

conversation moving. So, we're we're

going to do that and I'll just perhaps

hit on a few things and then dig into a

number of questions. But certainly at

Princeton you but also your entire

family had to survive. So, you were

involved with operating a dry cleaning

shop in New Jersey as one option, right?

you ran that for 7 years. So through

that, it feels like you've gained

perspective on many different levels

that have then helped inform what you've

done professionally, right? So you you

learn to think about not just people who

are protected in an ivory tower, but

people all the way down in across in

society. So from every swath of society

your mother also although she was not

technical she imbued in you this

discipline and also seems to have had a

very broad

appreciation and knowledge of literature

and international literature. So now you

have this global perspective presumably

at the time in Chinese and

then you end up you end up at Princeton

and I know we're going to be hopping

around quite a bit but I'm curious to

know how Imagenet came about and you can

introduce this any way you like. You can

tell people what it is and what it

became and why it's important and then

talk about how it started or you can

just talk about how it started. But it's

it's such a an important chapter.

>> So let me just explain what ImageNet is.

Imagenet on the surface was built

between 2007 and 2009 when I was an

assistant professor at Princeton and

then I moved to Stanford. So during this

transitional time my student and I built

this at that time the field of AI's

largest training and benchmarking data

set for computer vision or visual

intelligence. The significance today

after almost 20 years of imageet was it

was the inflection point of big data.

Before imageet AI as a field was not

working on big data and because of that

and couple of other reasons which I'll

get into AI was stagnating. The public

thinks that was the AI winter. Even

though as a researcher, young researcher

at that time, it was the most exciting

field for me, but I get it. It wasn't

showing breakthroughs that the public

needs.

>> But imaget together with two other

modern computing ingredients. One is

called neuronet network algorithm. The

other one is modern chips called GPU

graphic processing unit. These three

things converged in a seinal work,

milestone work in 2012 called image net

classification deep convolutional

neuronet network approach. That was a

paper that a group of scientists did to

show that the combination of large data

by imageet,

fast parallel computing by GPUs and a

neuronet network algorithm could achieve

AI performances in the field of image

recognition in a way that's historically

unprecedented.

And that particular milestone is many

people call it the birth of modern AI.

and my work image that was onethird of

that if you count the elements and I

think that was the significance I feel

really very lucky and privileged that my

own work was pivotal in bringing modern

AI to life

>> but the journey to image that was longer

than that the journey to image that

started in Princeton when I was an

undergrad you were in the East Asian

study department I was hiding in Jadwin

Hall which is our physics department.

>> Yeah,

>> I loved physics since I was a young kid.

I I don't know how somehow my dad's love

of bugs and insects and nature

translated in my head into just the

curiosity for for the universe. So, I

loved, you know, looking to the stars. I

loved the speed of fighter jets and the

intricate engineering of that eventually

translated into the love of the

discipline that that asks the most

audacious question of our civilization

such as what is the smallest matter?

What is the definition of spacetime? How

big is the universe? What is the

beginning of the universe? And in that

in that early teenagehood love I loved

Einstein. I love his work.

>> I wanted to go to Princeton for that.

But it turned out what physics taught me

was not just the math and and physics.

It was really this passion to ask

audacious question. So by the end of my

undergrad years, I wanted my own

audacious question. You know, I wasn't

satisfied with just pursuing somebody

else's audacious question. And through

reading books and all that I realized my

passion was not the physical matters. It

was more about intelligence

>> I was really really enamored by the

question is of what is intelligence and

how do we make intelligent machines.

So at that time I swear I did not know

it was called AI. I just knew that I

wanted to pursue the the study of

intelligence and intelligent machines.

And then I applied to grad school and I

went to Caltech. Caltech was my PhD. I

started in the turn of the century 2000.

And I think I consider that moment I

became a budding AI scientist. You know

that was my formal training as a

computer scientist in AI. Then my

physics training continued in a sense

that physics taught me to ask audacious

question and turn them into a northstar.

>> Mhm.

>> And in scientific terms that northstar

became a hypothesis.

>> Mh.

>> And it was very important for me to

define my northstar. And my first

northstar for the following years to

come was solving the problem of visual

intelligence

is how how we can make machines see the

world. And it's not just by seeing the

RGB colors or the shades of light is

about making sense of what's seen which

is you know I'm looking at you Tim. I

see you. I see a beautiful painting

behind you. I don't know it was real. I

see you're sitting on a chair like that

is seeing. Seeing is making sense of

what this world is. So that became my

northstar question. And that hypothesis

that I had is I have to solve object

recognition.

>> And then that was in my entire PhD was

the battle with object recognition.

There were many many mathematical models

we have done and there were many

questions but me and my field was

struggling. We can write papers no

problem but we did not have a

breakthrough and then luckily for me

Princeton called me back as a faculty in

2007. It was one of my happiest moment

of my life. I feel so validated my alma

m would consider giving me a faculty

job. So I happily moved back to

Princeton as a faculty this time and I

continue to be a Forbes member actually.

So at Princeton there was an epiphany is

that I realized there was a hypothesis

that everybody missed and that

hypothesis was big data. Could I pause

you there for a second because this is

the this is the point

>> that I'm so so curious about and I just

want to pause for a second also for

people who are interested in some of the

history of Princeton. It's pretty crazy.

They should look up the history of the

Princeton Institute for Advanced Study

and I remember taking some of those East

Asian Studies classes that you referred

to in classrooms where Einstein taught

and it's just the aura, the veneer. You

want to believe that you can feel it

just permeating the uh the entire campus

and it's fun in that respect. It's very

fun. But I'm going to read something

from a Wired piece that discussed you at

length and as you mentioned big data

before and after in terms of its

integration into the type of research

they were describing as it was written

and please feel free to fact check this

or push back on it but in wire they said

the problem was a researcher might write

one algorithm to identify dogs and

another to identify cats and then you it

says you know Lee began to wonder if the

problem wasn't the model but the data

she thought that if a child learns to

see by experiencing the visual world by

observing countless objects and scenes

in her early years, maybe a computer can

learn in a similar way. I want you to

expand on that for sure. And the

question for me is like why did you see

it, right? Why didn't it happen sooner?

>> We're all students of history. One thing

I actually don't like about the telling

of scientific history is there is too

much focus on single genius.

>> Yes, agreed. We know Newton discovered

the modern laws of physics but yes he is

a genius not to take away any of that

from Newton but but science is a lineage

and science is actually a nonlinear

lineage for example why was I inspired

by this hypothesis of big data because

many other scientists inspire me in my

book I talked about this particular

lineage of work by professor Beerman who

was a psychologist who was he was not

interested AI, but he was interested in

understanding minds. And I was reading

his paper and he particularly was

talking about the massive number of

visual objects that young children was

able to learn in early ages. Right? So

that piece of work itself is not image

that. But without reading that piece of

work, I would not have formulated my

hypothesis. So while I'm proud of what I

have done, my book especially wanted to

tell the history of AI in a way that so

many unsung heroes, so many generations

of scientists, so many crossdisiplinary

ideas

pollinate each other. So I was lucky at

that time as someone who is passionate

about the problem but also someone who

benefited from all these research. So

yes something happened in my brain but I

would really attribute to many things

happened across so many people's work

throughout their lifetime devotion to

science that we got to the point of

imageet. I'm so glad that you're

underscoring this because if you really

dig as a I don't consider myself a

scientist, but I I love reading about

the history of science. There's so many

inputs, so many influences, so many

interdependencies.

>> Yes.

>> And the simplicity of the single hero's

journey is appealing and its simplicity,

but it's almost never true.

>> It probably is never true. Even my

biggest hero, Einstein, right? He

anybody who knows me, anybody who read

my book knows how much I rever him and I

just love everything he's done. The

special relativity equation is a

continuation of Lawrence transform. Even

Einstein, he builds upon so many other

people's work. So I think it's really

important especially I'm sure we'll talk

about it. I'm here calling you in the

middle of Silicon Valley and we're in

the middle of an AI hype and obviously

I'm very proud of my field but I think

that when the media or whatever tells

the story of AI it almost always just

talk about a few geniuses and it's just

not true. It's generations of computer

scientists, cognitive scientists and

engineers who who made this field happen

>> for sure. I mean, everyone knows Watson

and Crick for for instance, but without

Rosalyn Franklin and her X-ray

crystalallography, it doesn't happen.

Doesn't happen. It just doesn't happen.

Point blank. We're going to hop to

modern day in a second, but with

ImageNet, I would love for you to speak

to some of the decisions or let's say

decisions or moments that were just

formative in making that successful,

right? Because for instance, if you're

going to

try to allow a machine to, and I'm using

very simple terms cuz I'm not technical

enough to do otherwise, to learn to

identify objects

closer to the path that a child would

take, you have to label a lot of images,

right? And I was reading about how

Mechanical Turk came into play and then

there's a competitive aspect that seems

to have driven some of the watershed

moments. Could you just speak to some of

the elements or decisions that made it

successful?

>> A lot of people ask me this question

because after image that many many

people have attempted to make data sets

but still only very few are successful.

So what made image less successful? I

think one of the success was timing is

that we truly were the first people who

see the impact of big data. So that very

categorical or qualitative change itself

is part of the success but it's also as

you were asking the hypothesis of big

data is not just size. A lot of people

actually misunderstands image nets

significance as well as other data sets

significance coming with the data set is

a scientific hypothesis of what is the

question to ask. For example, in visual

recognition you could talk about you

could make a data set of discerning RGB

and that would not be as impactful of a

data set that is organized around

objects. Mhm.

>> We can go down a rabbit hole of why not

because RGB is easier per se. It's

because you have to ask the scientific

question in the right way. Another

example is instead of making a data set

of objects, why don't you make a data

set of cities,

>> you know, that's even more complicated

that objects. But then that's dialing

too complicated. So every scientific

quest, you have to have the right

hypothesis and and asking the right

question. So that's one part of the

success is we defined visual object

categorization as the right hypothesis.

>> That was one rightness I guess. Another

rightness is that people just think oh

it's easy you just collect a lot of

data. Well first of all it's laborious.

But even aside from being laborious how

do you define the quality?

>> Mhm. You could say well if quality is

big enough we don't care about quality.

But how do you dial between what is big,

what is good and how do you trade off

that is a deeply scientific question

that we have to do a lot of research on.

And then another decision that is a set

of decision that is really hard is what

defines quality in terms of image. Is it

every image has higher resolution? Is it

it's photorealistic?

Is it because it's everyday image that

look very cluttered? Is it all product

shots that look clean? These are

questions that if you're too far away,

you wouldn't even think about asking.

But as a scientist, as we were

formulating the deep question of object

recognition, we have to ask this in so

many dimensions. And then you mentioned

Amazon Mechanical Turk. That is actually

a consequence of desperation

because when we formulated these this

hypothesis, our conclusion is we need at

least

tens of millions of high quality images

across every possible diverse dimension.

Whether it's user photos or is it

product shots or is it stock photography

like and then we need also high quality

labels. Once we make that decision we

realize this has to be human filtered

from billions of images.

>> So with that we became very desperate.

We're like how are we going to do that?

You know, I did try to hire Princeton

undergrads and as you know, Princeton

undergrads are very smart. But

>> they have very high opinion of the value

of their time.

>> Yes. And they're expensive. But even if

I had all the money in the world, which

we didn't, it would have taken so long.

So, we were very very stuck for very,

very long. We thought we had other

shortcuts, but the truth is human

labeling is a gold standard. M

>> we want to train machines that are

measured against human capability. So we

cannot shortcut that at that time.

>> Right?

>> So we had to go to what we eventually

found out is called crowd engineering

>> crowdsourcing and that was a very new

technology

was barely a year old or so

by Amazon. They they created a lot

online marketplace for people to do

small tasks to earn money. when these

tasks can be uploaded on the internet. I

remembered when I heard about Amazon

Mechanical Turk, I logged into my Amazon

account. I checked the first task I

checked out to do just to try was

labeling wine bottles or transcribing

wine bottle labels. The task will give

you a picture of a wine bottle and you

have to say this is 1999 Berdo and and

all that. Yeah,

>> people upload these kind of micro tasks

and then online workers like someone in

their leisure time like me if I had

leisure time I would just go sign up and

get paid to do that. And we realized

that was again out of desperation that

was a massive parallel processing with

online global population to do this for

us and that's how we labeled billions of

images and distilled it down to 15

million high quality image that images.

>> It's just so wild when you look at these

stories. is I just finished a book on

Janentech and there were all these

little technical inflection points that

also allowed things to happen right so

if it had been 5 years earlier

or maybe 3 years earlier right without

mechanical turk boy like it presents a

challenge

>> y

>> but also as you pointed out in science

it's one thing to get answers but you

need the input on the front end with a

proper hypothesis or a good question and

even with mechanical turk if you're only

focused on the

the mechanics of employing that, you can

get yourself into trouble. Because if

humans are incentivized, right, to let's

just say, I think this was the example I

read about, identify pandas in

photographs and they're paid for

identifying pandas, well, what's to stop

them from identifying a panda in every

photo, whether they exist in the photos

or not? Yes.

>> Right. So, you have to follow the

incentives as well. How did you solve

for that?

>> This is where you know my student and I

had I cannot tell you how many hours and

hours of conversation we have about

controlling the quality. We have to

solve for that in multiple steps. We

need to first filter out online workers

who are serious about doing the work. So

for example, we have to have some

upfront quizzes

>> so that they understand what a panda is.

They read the question and then once

they get into they qualify for that we

ask them to label pandas but there are

some images we know the correct answers

some are true pandas some are some are

not true pandas

>> but the labelers don't know so in a way

we implicitly monitor the quality of the

work by knowing where the gold standard

answers are

>> so these are the kind of computational

tactics we have to use to ensure the

quality of labeling.

>> Amazing. Just incredible. I'll actually

just put a recommendation out there for

a book, Pattern Breakers, by a friend of

mine, Mike Maples Jr. He taught me the

ropes initially of angel investing. But

in terms of identifying inflection

points and in some cases converging

technological trends that for the first

time make something possible which then

opens an opportunity right for something

with the right prepared mind in your

case and those of your collaborators and

the people you built upon for something

like imagageet pattern breakers is a

really good read for folks. So let's

let's hop to modern day then for a

moment and I would love to ask you right

because you've been called the godmother

of AI in our alumni magazine in fact and

elsewhere but you've had such a not just

technical but historical viewpoint

meaning you've over a broad timeline

well broad by AI standards been able to

watch the development and forking and

perils and promise of this technology.

What are people missing? What do you

think is eating up all the oxygen in the

room? What are people missing? Whether

it's things they should know or things

they should be skeptical of or otherwise

>> especially I'm here calling you from the

heart of Silicon Valley and I think

people are missing the importance of

people in AI

>> and there's multiple facads or

dimensions to to this statement is that

AI is absolutely a civilizational

technology. It's I define civilizational

technology in the sense that because of

the power of this technology it'll have

or already having a profound impact in

the economic, social, cultural,

political

downstream effects of our society. So

>> I just heard this is unverified but I

just heard that 50% of the US GDP growth

last year is attributed to AI growth.

>> Apparently this number is 4% for US GDP

have grown 4%. If you take away AI it's

only 2%. That's what means

>> that's civilizational from an economic

point of view. It's obviously redefining

our culture, right? Think about you're

talking about the word sucking oxygen

out of the room everywhere from

Hollywood to Wall Street to Silicon

Valley to political campaign to Tik Tok

to YouTube to Insta.

>> Taxis in Japan. I was just there and the

videos playing on the back of the

headset and the taxi. We're all talking

about AI. It's everywhere.

>> It's culturally impactful. Not only

impactful, it's shifting our culture and

it's going to shift education. Every

parent today is wondering what what

should their kids study to have a better

future. Every grandparent is say, "I'm

so glad I'm born early. I don't have to

deal with AI." but still worry about

their grandchildren's future. So AI is a

civilizational technology, but what I

think it's missing right now is that

Silicon Valley is very eager to talk

about tech and the growth that comes

with the tech. Politicians are just

eager to talk about whatever gets the

vote, I guess. But really, at the end of

the day, people at the heart of

everything. People made AI, people will

be using AI, people will be impacted by

AI, and people should have a say in AI.

And no matter how AI advances,

people's selfdignity as individuals, as

community, as society should not be

taken away. And that's what I worry

about because I think I think there's so

much more anxiety that because the sense

of dignity and sense of agency, sense of

being part of the future is slipping in

some people and I think we need to

change that. I've heard you say that

you're an optimist because you're a

mother. And both optimism and pessimism

to an extreme can bias us in ways that

are unhelpful, right? Or create blind

spots. And I'm curious if you try to put

your most objective hat on, which is

difficult for any human, but if you try

to do that, do you think people are too

worried, not worried enough, or worrying

about the wrong things? for people who

are not CEOs and builders and engineers

behind AI because you're right of course

I mean everybody will agree with this

that a lot of people are very worried

and I'm just wondering if it's if it's

ill-placed because I don't really if you

talk to some of the VCs who are the

biggest investors of course they have

this sort of in my view beyond all

possibilities techno optimist view of

the future where AI solves everything

right and it's hard to believe there's a

free lunch

And then you have the the doomers, the

doom and gloom where suddenly it's

Skynet next year and we're all slaves to

robots or eliminated, turned into paper

clips and reality is probably in between

those two. Do you think people are

worrying about the right things or have

they lost the plot in some way?

>> First of all, I call myself a pragmatic

optimist. I'm not a utopian. So I'm

actually the boring kind. I don't

believe in the extreme on both sides. I

travel around the world. Just last month

I was in Middle East, I was in Europe, I

was in UK and I I was in Canada, I came

back home in America. I think people in

America and people in Western Europe are

more worried about AI than say people in

Middle East, in Asia.

And I think we don't have to litigate

why they're more worried

>> but just to come closer to home just in

talk about us. I wish I have a megaphone

to tell people in the US that you're

known to be one of the most innovative

people our country have innovated so

many great things for humanity for

civilization.

We have a society that is free and

vibrant and we have a political system

that we still have so much say in how we

want to build our country. I do wish

that our country has more optimism and

positivity towards the future of using

AI than what is being heard now. I think

people like me technologists living in

Silicon Valley has a lot of

responsibility

in the right kind of public

communication.

So there's a lot of things that was not

communicated in the effective way. But I

do hope that we can instill more sense

of hope and

self agency into everybody in our

country because I think there's so much

upside of using AI in the right way. And

I want not just people in Silicon Valley

or in Manhattan, but I want people in

rural communities in traditional

industries in everywhere 50 states to be

able to embrace and and benefit from AI.

>> Why are you building what you're

building? What is World Labs? Why decide

to do this?

>> I actually answer this question very

often to every member of my team. Mhm.

>> I built World Labs. There are two levels

of this answer. From a technology point

of view, World Labs is building the next

generation AI focusing on spatial

intelligence

>> because spatial intelligence just like

language intelligence is fundamental in

unlocking incredible capabilities in

machines so that it can help humans to

create better, to manufacture better, to

design better, to build better robots.

So spatial intelligence is a lynchpin

technology. Mhm.

>> But one level up, why am I still a

technologist?

Is because I believe humanity is the

only species that builds civilizations.

Animals builds colonies or herds, but we

build civilizations.

And we build civilizations because we

want to be better and better. We want to

do good. Even though along the way we do

a lot of bad things but there is a

desire of having better lives, having

better community, having better society,

live more healthily,

have more prosperity.

>> That desire is where civilization is

built upon. And because I believe that

humanity can do that, I believe science

and technology is the most powerful

tool, one of the most powerful tools in

building civilizations and I want to

contribute to that. That's why I'm still

a scientist and a technologist and I'm

building world labs for that. Can you

explain to people what spatial

intelligence is and what the product is

so to speak at least as it stands right

now that you're building?

>> Spatial intelligence is a capability

that humans have which goes beyond

language is when you pack a sandwich in

a bag when you take a run or a hike in a

mountain. When you paint your your

bedroom, everything that has to do with

seeing and turning that scene into

understanding of the 3D world,

understanding of the environment and

then in turn you can interact with it,

you can change it, you can enjoy it, you

can make things out of it. That whole

loop between seeing and doing is

supported by the capability of spatial

intelligence. Right? The fact that you

can pack a sandwich means you know what

the bread looks like. You know how to

put the knife in between. You know how

to put the lettuce leaf on the bread.

You know how to like put the bread or

sandwich into a Ziploc bag. Every part

of this is spatial intelligence. M and

does today's AI have that? It's getting

better, but compared to language

intelligence, AI is still very early in

that ability to see, to reason,

>> and also to do in world in both virtual

3D world as well as real 3D world. So,

so that's what world labs is doing. We

are creating a frontier model that can

have intelligent

capability in the model to create world

to reason around the world and to enable

for example creators or designers or or

robots to interact with the world. So

that's spatial intelligence.

>> Could you expand on the you know

designers or creatives or robots

interacting with the world? So does that

mean that you could and my team has been

playing with with some of the tools. So

thank you for that. What does that mean?

If you could paint a picture for let's

say a year from now, two years from now,

how might someone use this or how might

a robot use this?

>> I was just talking to someone a couple

of weeks ago and it was really inspiring

is that high school theaters are very

low budget, right? like, okay, sometimes

I go to San Francisco opera or musicals

and the sets that's built for theater

are just so beautiful.

>> Mhm.

>> But it's very hard for high school or

middle school to have that budget to do

that. Imagine

>> that you can take today's worldlapse

model, we call it marble.

>> Mhm.

>> And then you create a set in medieval

French town.

>> Mhm. And then you put that in the

background and use that digital form to

help transport the actors and action

into that world. And of course,

depending on the auxiliary technology,

whether you're on a computer or

eventually people can use a headset or

whatever, you can have that immersive

feeling of being in a medieval French

town. That would be an amazing creative

tool for a lot of creators.

>> That was the example. Someone and I was

talking about it a couple of weeks ago.

But we already see creators all over the

world. Some of them are VFX creators.

Some of them are interior design

creators. Some of them are gaming

creators. Some of them are educators who

want to build some worlds that transport

their students into different

experiences are already starting to use

our model

>> because they find it very powerful at

their fingertip to be able to create 3D

worlds that they can use to to immerse

either their characters or themselves

into. And just a process-wise, if if

someone's wondering how this works,

let's just say it's a a public school

teacher, let's just say, who's hoping to

inspire and teach their students going

the extra mile. What does it look like

for someone to use this? Are they typing

in text, describing the world they'd

like to create, uploading assets or

photos, almost like an image board? How

does it how does it work? If someone's

nontechnical,

>> they don't need to be technical at all.

They open our page on desktop or in

their phone, but desktop is more fun

because it has more features.

>> And then they can type, you know, a

French medieval town or or they can

actually go to anywhere. They can use

midjourney or nano banana to create a

photo of a French medieval town or they

can get an actual photo about that and

then they upload it. We call it prompt.

And then after a few minutes, our model

gives you a 3D world that is

say a part of the tab. It does have a

limit in its range. And then that 3D

world is generally 3D because you can

just use the mouse to drag and turn

around and walk around and see that

world. And then downstream if you want

to use it, you have many ways to use it.

You can actually create a movie out of

it by using one of our tools on the

website to just put cameras and you can

make a particular movie out of it.

>> You could if you're a game developer.

>> I was just going to say it sounds a lot

like a gaming engine.

>> Yes, you you can put a lot of characters

in it. If you're VFX professional, we

have a lot of VFX professional. they can

actually take this and put it in the

workflow of their movie shooting and

have real actors shooting movies. We've

also have psychology researchers using

that immersive world in particular

psychiatric studies.

>> We could also use that as the simulation

for robotic training

>> because a lot of robotic training needs

a lot of data and then use that for

generating a lot of different data. So

is it almost like a flight simulator for

robots before they go into the real

world?

>> That's part of the goal. We are still

early. So the flight simulator is not

complete yet,

>> right?

>> But that's part of the journey.

>> You mentioned psychiatric studies. I I

think that's what you just mentioned.

Yes. What might that look like? We

actually got this researcher who called

us and they're studying people who have

psychological disorders like

obsessivecompulsive disorder

>> where they're triggered by certain

environments and they want to study the

trigger and also just study how the

treatment but how do you trigger someone

who let's say is particularly have issue

with let's say a strawberry field I'm is

making it up.

>> I mean, you can take them to a

strawberry field, but what about you

want to know if it's strawberry field in

the summer or strawberry field at night

or it's strawberry or it's mating

strawberry like how do you do this?

Suddenly this researcher realized we

give them the cheapest possible way of

varying all kinds of dimensions and they

can test this out and do their studies.

>> That's really interesting. Yeah, I could

see it being applied to it might be

called exposure therapy, but now that

you're describing it, I could see how it

could be

>> added into I mean pretty much

everything, right? I mean, if you think

about how humans operate in the real

world.

>> Yes. And the boundary between real world

and digital world is less and less,

right? Thinner and thinner because we

live in many screens. We live in the

real world. We do things in virtual

world. We do things in real world. will

create machines that can do things in

real world and virtual world.

>> So there's a lot we do in digital and

physical spaces.

>> Who are some scientists or researchers

who you pay attention to who are not

necessarily kind of the big brand names

and marquee lights that are already very

public in the world? Is there anybody

who stands out where you're like, you

know, there's some really tremendous

people doing good work? Well, that's

part of the reason I wrote the book is

especially in the middle chapters where

I wrote about the journey of doing image

that combines cognitive science with

computer science and I actually talk

about psychologists and neuroscientists

and developmental psychologists in you

know some of them are still with us some

of them are not for example the the late

anman

beerman they all passed away in the last

few years But they were giants in

cognitive science whose work has

informed computer science and eventually

AI. You know there are still lots of

scientists around the world. Many of

them are in the US who are thinkers in

developmental psychology in AI. I follow

their work. Mhm.

>> I think the world of science, just to

name some names, right, Liz Beli in

>> in Harvard, Allison Gobnik in Berkeley,

I love Rodney Brookke, who was a former

MIT professor in robotics,

>> and there's just a lot of them. I I

don't mean to just single them out.

>> Sure.

>> But you're asking me for names that are

not in in the news of AI.

>> Yeah, that's perfect. Thank you. I would

also love to get your perspective on

what might be this is a very strong word

but seemingly inevitable in in terms of

developments in the near intermediate

future. And I'll give you an example of

what I mean. In 2009

2008 2009 I became involved with Shopify

the company back when they had like 10

employees. And there were a few things

happening around that time and you could

ask questions, you know, in the next 10

years or 20 years, will there be more

broadband access or less? More. Okay.

Will there be more e-commerce or less?

There'll be more. Okay. And when you

have four or five of those that seem

over a long enough time horizon,

absolutely yeses, it begins to paint a

picture of where things are going. Are

there any things that in the next

handful of years you think are perhaps

underappreciated as near

inevitabilities?

>> You want me to talk about

underappreciated? I mean, I don't know

if they're overappreciated, but

definitely appreciated. The need for for

power is appreciated.

>> Mhm.

>> The trend of more AI, not less AI is

appreciated.

>> The long-term trend of robots coming is

appreciated. So, these are appreciated.

What's underappreciated is spatial

intelligence is underappreciated in the

sense that everybody's still now talking

about language large language models but

really world modeling of pixels of 3D

worlds is underappreciated because like

you were saying it powers so many things

from storytelling to entertainment to to

experiences to robotic simulation. I

think AI and education is

underappreciated

because what we are going to see is that

AI can accelerate the learning for those

who want to learn

>> which will have downstream implication

in our school system

>> as well as in just human capital

landscape like how do we assess

qualified workers?

>> Mhm. used to be which school you

graduate from with with which degree but

that will be changing. Yeah. With AI

being at the fingertip of so many people

that's underappreciated.

I think AI's impact in our economic

structure including labor market is

underappreciated.

The nuance is underappreciated. I think

this whole rhetoric of either total

utopia post scarcity is hyperbolic.

>> Yeah.

>> Or like everybody's job will be gone is

hyperbolic.

>> But the messy middle is how

from knowledge worker to blue collar to

hospitality to all these changes that's

happening. It's underappreciated by our

policy workers, by our scholars, by just

overall society.

>> Well, what are some of the nuances from

the job perspective? Maybe this ties

into what I promised earlier I was going

to ask you, which is what you are

telling or will tell I don't know other

ages your children are recommending.

Let's just say I don't know how old they

are, but if we assume that they just for

the sake of discussion of the age where

they're trying to decide what they

should study, where they should focus,

things of that nature, how how would you

think about answering that even

provisionally?

>> I think the ability to learn is even

more important because

when there was less tools, fewer tools

to learn, it's easier to just follow

tracks. You go through elementary

school, middle school, high school,

college and then get some, you know, get

some training vocationally and that's

kind of a path and with that is a set of

structured

credentials from degrees and all that

but AI has really changed it. For

example, my my startup when we interview

a software engineer honestly how much I

personally feel the degree they have

matters less to us now is more about

what have you learned what tools do you

use how quickly can you superpower

yourself in using these tools and a lot

of these are AI tools what's your

mindset towards using these tools matter

more to Mhm.

>> At this point in 2025, hiring at World

Labs, I would not hire any software

engineer who does not embrace AI

collaborative software tools.

>> Mhm.

>> It's not because I believe AI software

tools are perfect. is because I believe

that shows first of all the ability of

the person to grow with the fast growing

toolkits the open-mindedness and also

the end result is if you're able to use

these tools you're able to learn you can

superpower yourself better

>> so that is definitely shifting so coming

back to your question what do you tell

young people tell children I think the

timeless value of learning to learn, the

ability to learn is even more important

now.

>> Yeah, it it strikes me as we're talking

that it's only going to get increasingly

easier for

the ambitious to act as superpowered

autodidacts, right? We've already seen

this

>> with certainly YouTube has a nice track

record. Now you can either entertain

yourself to death and avoid doing things

that help with self-rowth and

development or you can supercharge it.

And similarly with AI, right, you flash

forward. We don't even need to flash

forward, but it's how does a teacher

audit that their students are doing the

work they're supposed to be doing.

>> Yeah.

>> On so many levels, it's getting to the

point. There are some exceptions, but of

near impossibility.

>> Yeah. and students can either avoid all

work or they can supercharge their own

work but the output might look very

similar at least for a period of time.

So schooling is going to change a lot.

It's very very interesting.

>> I actually think Tim

if the school evaluation is structured

in a way that whatever AI gives and

whatever the student gives is the same

there's something wrong with the

structure of the evaluation.

>> Okay. Can you say more about that?

That's interesting.

>> For example, English essay.

>> This is not me. This is me hearing a

story that I so agree with. I'll retell

the story. Is that as a high school

freshman English class teacher? I heard

that someone told me the story of their

kids school. On the first day of school,

the teacher actually said to the class,

I want to show you how I would score AI.

So the teacher give an essay topic. Show

the students this is what the best AI

gave me and I'm going to show you how I

think this is good, this is bad, how

this is suboptimal and I'll give it a B

minus.

Now I will tell you this is my bar. If

you're so lazy that you ask AI to write

your essay, this is what you're going to

get. But you can use AI, that's totally

fine. But if you can do the work, learn,

think, be the best human creator you can

and work on top of that,

>> you can get to a you can get to A+es.

And that would be in my opinion the

right way to structure the evaluation is

not to pit humans against the AI and

then try to police the use or not use of

AI. is that to show where the tools the

bar of the tools are and where the bar

of the human learner should be.

>> I'm going to sit with that example and

try to think of more examples. It's very

interesting and boy oh boy I've been

shocked by how quickly the models

improve. But yes, that's like as a

thought experiment.

>> Yeah,

>> I'm going to chew on that. I know we

only have a few minutes left. Fifth, I

wanted to ask you a question I ask a

lot, which is if you could put a quote

or a message, something on a billboard,

something to get in front of millions,

billions of people. Just assume they all

understand it. Could be an image, could

be a question, could be a quote,

anything at all. A saying, a mantra,

doesn't matter. Could be almost

anything. What would you or what might

you put on that billboard?

>> What is your northstar?

>> What is your northstar? This is of

course critically important and coming

back to

how you define that or find that for

yourself. I mean you were talking about

audacious questions

and then that leading to a north star or

hypothesis. Is there another way that

you would encourage people on top of

that to think about finding their north

star? I believe that's how that makes us

so human and makes us to be so fully

alive

>> is that we as as a species can live

beyond the chasing of just basic needs

right but dreams and missions and goals

and passion and everybody's northstar is

different

>> and that's fine not everybody has to

have AI as their northstar but finding

That goes to the heart of education

again and I don't mean formal classroom

education. It's just the journey of

education. A lot of that is the ability

to learn who you are and to learn how to

formulate your northstar and how to

chase after that.

>> Last question. Did your parents ever

explain to you why they named you Fay?

>> Yes. is because when my mom was going

through labor, my dad was

characteristically late to the hospital

and along the way he caught a bird. He

let it go, but he did catch a bird. I

don't know, he was just distracted and

it was in Beijing in the city of

Beijing. My dad was bicycling to my

mom's hospital

>> that inspired him to call me Fay.

>> Feet.

>> Oh, wait. Sorry for those who don't

speak Chinese. I forgot you do speak

Chinese, but for those who don't speak

Chinese, Fay means flying.

>> Means flying.

>> Yeah. So, be inspired by a bird.

>> Really quick, I'll just say it's kind of

funny. My first Chinese name that I had

was Fay Ting Chong, which is because I

was very blunt and honest. So, Ting, but

Fing Chong, but when I was first

starting, my tones in China were not

polished and people thought I was saying

that my name was Fiji Chang, which is

airport. So, I petitioned my teachers

and we changed my name to something less

less confusing.

>> What's your new name?

>> Oh, okay.

>> It's like

but it's without the at the bottom.

>> Oh, wow.

>> Fancy name. That's way more

sophisticated than my

>> Well, I get to script it with my Chinese

teachers, so I have an unfair advantage.

Dr. Lee, thank you so much for the time.

We will link to the show notes for

everybody at tim.blog/mpodcast. They'll

be able to find you easily and everybody

should check out worldlabs.ai

and we'll put every other link, your

social and so on in the show links. But

thank you for the time. I really

appreciate it.

>> Thank you, Tim. I enjoyed our

conversation.

>> Yeah, likewise.

>> Okay, bye

>> bye.