foreign

my name is Shai Solomon and I'm honored

to serve as the board member for the

American Friends of the open University

of Israel as well as the global director

of cyber security Workforce Development

a checkpoint software Technologies

joining me today is Dr Ellie Shai

ezatsuo who is not only the principal

investigator of the neuro and biomorphic

Engineering Labs but also hold the

position of assistant professor at the

open University of Israel we are

delighted to have the opportunity to

interview Elia suitskiver a renowned

scientist in the field of machine

learning and co-founder and chief

scientist at openai

as a sponsor of discussion on issues

related to Israel technology and the

world we are proud to support the open

University of Israel a non-partisan

education institution and the largest of

Israeli 10 accredited universities we

believe that foresting open dialogue and

hearing their first perspective from

world leader on issues related to

Israeli and the world is essential and

we are confident that our audience will

greatly benefit from hearing Ilya unique

perspective of the open University and

his professional career

Ilya is an honor and a pleasure to have

you here with us

thank you for joining us given your

expertise we would like to discuss a

wide range of topics related to your

personal Journey machine learning open

Ai and your thoughts on the future of

Education we will be asking a number of

questions over the next 40 minutes or so

so let's jump in hi Elia can you please

share with us your initial academic

Journey at the open University of Israel

and how we became interested in the

field of artificial intelligence

you know I I owe I feel I feel a lot of

gratitude to the open University

what happened was that

I was in school

and I was doing quite well

and together with my parents we were

looking for

some ways in which I could learn more

and it was so it was the case that

the open University

accepts

anyone regardless of whether they have a

high school degree or not

and so for this reason I was able to

start taking classes in the open

University

starting from eighth grade

and

that was that was that was really great

and I really liked those classes it was

you know how it works you get books by

mail and you send the problem sets you

mailed back the problem sets and you go

write the exam and you can study

whatever you want and I I really like

that

and

it was possible only because the open

University

took me even though I was

a young student without the credentials

to study in a regular University

but then

the question of computer science and

math and AI so I would say that

so I think I think in my case it was

pretty clear

that these are

the subjects that I was most drawn on

even as an early child as a young child

and so that's why I studied them at the

open University it was still

a little bit a few years before I really

set my eyes on AI

that's great I mean sounds like great

experience and did you leverage like

remote learning I mean like sending over

your work or did you did you go to a

physical uh classes there were physical

classes but they would be very

infrequent so I would go maybe once a

week or twice a week yeah so the great

majority of the

of the learning was remote and at my at

my at my own schedule

and I found that it happened to be a

good fit for me

I found that I could just and the books

are very well written too so it made it

very

you could you didn't it was

you know if the books were less good

it would have been harder yeah but I

thought the books were very good and but

for that reason it was very possible to

just read it slowly

do the exercises and that's that's all

you needed

yeah

okay so moving from the past to the

present

uh let's talk about open AI so what were

the main reasons for you to establish

open AI

so the time it's the time maybe a year

before

we started openai

I was a researcher at Google

and I was working on deep learning

and I was having a lot of fun I was

really enjoying my time at Google

doing the research there and working

with the people

with my colleagues at Google

but

the thing which I felt

already then in 2014 and 2015.

is that the future of AI

is going to be

much

is going to have that so maybe for a

little bit of context

AI research has strong academic groups

yeah it means that all of the AI was

done in University departments it was

done by professors with their grad

students almost entirely there's also

been some AI being done in companies but

I would say that for the most part the

majority of the most exciting work came

from universities

and then back in the day that was the

the only successful model

and that was also the model that Google

has adopted where you have as an

environment that is similar to the

university environment where you have

small groups of researchers working

together on a project

and already then I felt that that's not

the future I felt that the future would

be much

more

much larger and much more organized

engineering projects

because it was clear that AI was going

larger with larger neural networks and

larger but more gpus which in turn means

more engineer the stack gets very

complex it becomes very difficult for a

small group of people to do to do to do

something like a very small group of

people to

complete a big project like this

teamwork is required

and that was one of the reasons and so I

was kind of sitting at Google and

feeling a little bit Restless

but I didn't know what to do about it

so I was

feeling a bit

like it wasn't quite right and then one

day

basically

like some kind of picture this here I am

Daydream like it was daydreaming that

maybe I could start an AI company but it

really wasn't clear how I would do it

how would you possibly get the money for

such a thing those things would be

expensive

there was there was a daydreaming

element to it but I didn't really think

very seriously about it because it was

obviously impossible

and then one day I received an

invitation to get dinner with some

Altman and Greg Brockman and Elon Musk

and here here I am sitting getting

dinner with these amazing people in mind

you it was a cold email it's reached out

to me say hey let's let's hang out

essentially

how did they reach out to you

email email like uh

just just an email you received the name

and say hey like you know do you want to

join yeah it sounds like in that context

it sounds like a you know uh fishing or

some uh malicious email because it's so

extreme

no I mean you know it looks it looks but

it's it was

that it was definitely not that it was

very clearly authentic but it was a

little bit for me it was a small moment

of wow that is so amazing so of course I

went and here I was at the dinner and

they were discussing how could you start

a new AI lab which would be a competitor

to

Google into deepmind which back then had

absolute dominance

and that was the initial conversation

you know then it was of course for me to

leave Google it was quite uh

difficult decision because Google was

very good to me it was very very a very

good place to be

but eventually I decided to leave Google

and to join and create open Ai and

ultimately the pre the idea of open air

is to take the idea of AGI seriously

it's the idea is to take like you know

because when you are a researcher you

know researchers

are somehow I would say train to think

small

I think researchers

due to the nature of the work small

thinking gets rewarded because you have

these problems and you're trying to

solve them all the time and it's quite

hard to make even small steps so you're

just focused on what's coming at you the

next step and it's harder to see the

bigger pitch

but at open AI we took the liberty to

take to look at the big picture

we ask ourselves okay what's the where

is AI going towards

and the answer is AI is going towards

AGR towards an AI which eventually is as

smart or smarter than a human in every

way

and you think about that and you go wow

this is a really profound

that is a very profound thing

and so with open AI

we thought it

we thought that it made the most sense

to give it the explicit goal

to make AI benefit make AGI benefit of

humanity because this technology is just

going to be so transformative it's going

to turn everything upside down on its

head

Whenever there is such a big change who

knows what's going to happen

so for this reason the goal of open AI

is not only to develop the technology

but also

to find a way to make it

as beneficial as possible to make it

benefit of humanity and so the

combination of those big ideas and those

incredible people that were at that

dinner it just I I just

despite despite all the difficulties

that Google has put in in front of me to

leave I still decided to go for it

and yeah it's been now more than seven

and a half years and

it's been a very

exciting and gratifying Journey

thank you for being so honest and open

with us we really appreciate it so you

know back in the days when people talked

about machine learning it was more about

finding you know small patterns and

maybe find some statistical

and statistical you know

is a statistical pattern within the data

for very specific problems so you had a

model for computer vision you had a

model for language and you had a model

for for this in the middle for that but

here you are talking about general

intelligence

and can you tell can you identify the

moment when you said you know this

technology this this neural networks can

be used for multiple problems for

multimodal sensing they can be something

that can be General enough

because back in the days when we were

limited by you know the hardware

capabilities that we had you know before

the age of the gpus and everything it

was pretty limited to specific domains

but when was the time that you said this

is going to be big this this can get

seriously in the field of general

intelligence to go ahead and

start open AI

it was a bet on deep learning it was a

bet that somehow with deep learning we

will figure out how to make smarter and

smarter realities so in some sense the

creation of open AI was already an

expression of this bet of the idea that

deep learning can do it you just need to

believe and in fact I would argue that a

lot of a lot of you know deep learning

research at least in the past decade

maybe a bit less now has been about

faith about

rather than inventing new things just

believing that the technology that the

Deep learning technology can do it

but now I want to talk about the

question and you said and why I want to

explain just a bit why I think it's not

quite the right question

so

you asked when

do you become clear that you know a

neural network could be General and can

do many tasks which in some sense is

what we are moving towards but I would

argue that this is the less important

dimension

the more important dimension

is that of

capability and act and and competence

rather is the neural network competent

you know you can have a specialized

language neural network where you don't

have a language an image neural network

but is it actually good

if it's not good

then it's not interesting

so the question is not whether

deep learning can be General

but whether it can be competent

and what we are seeing now is the Deep

learning can indeed be competent maybe

you can talk us it take us a little bit

into your journey in the development of

this

large-scale neural network that you

worked in I mean where did you start and

how it was evolved over the years to

become GPT 3 and gpt4

you know it's a it's a long it's a long

story with many

interlocking parts

let's say the evolution has gone

the story of deep learning can be seen

it's quite an old story

maybe a 70 year old story

back in the 40s

researchers have already started to

think about the ideas that were later to

become the Genesis and deep learning

it is the idea of the artificial neural

you see the human brain

is big

in a sense that it has 100 billion

neurons

and the human brain is also

at least until like or arguably steal

the best example of intelligence that

exists in the universe

so then you can start asking yourself

the question of okay so what is it about

the brain that makes it smart

well maybe if you had

a lot of neurons arranged in a certain

correct way

you would get intelligence

and so now you can ask yourself what's a

neuron

so biological neurons have lots of

complicated behaviors but the idea that

the scientists from the 40s have is

maybe you can simplify

those biological neurons down to

something which would be their essential

computation something which is called

the artificially and it is very simple

it's just a simple mathematical formula

and then they started to ask questions

like what can you do with this

artificial neurons how can you arrange

them what kind of little problems they

can run

what kind of functions they can they can

compute

but this was just the first step this

was the first biggest

first big step is to invent the

artificial View

the second big step was

to discover

how these neurons can learn even in

principle

one of the obvious things about human

intelligence and also animal

intelligence

is that we learn

we learn from experience and we learn

and generalize and

this is the basis of us succeeding in

the world

so how does learning work

you know it's not you know right now we

are used to the idea that computers can

learn obviously

but I would say that even in

the year

2003 when I started working on

machine learning in Toronto

it wasn't clear that learning can be

successful they haven't been a really

successful examples

and so

a very big Discovery was an equation of

learning in neural networks a

mathematical equation that tells you how

to change the synapses of the neural

network

so to incorporate the experience

but it was just an idea it wasn't a

proven idea it was an idea that maybe

here is a mathematical mathematical

equation which might have the desirable

properties of learning that was done

that's the back propagation algorithm it

was done in 86.

by my by my PhD advisor Jeff Hinton

but then you so now you have the

artificial neuron and you have the back

propagation algorithm

and it's still an idea it's not proven

so I would argue then the next big step

and that took I would say the two

thousands was to prove that this idea is

actually good

and it is and it culminated this decade

culminated with a few demonstrations of

large neural networks large by the

standards of that decade really really

small by today's standards but a

demonstration that neural networks

trained with the back propagation

algorithm can in fact solve interesting

challenging and meaningful problems much

better than anyone could have imagined

and that was

like one of these demonstrations was the

neural network which beat all other

methods on on imagenet in 2012 which is

a project I was very fortunate to have

contributed to

and

that began

previous decade the 2010s where people

would just say okay well let's just

Tinker with these neural networks and

trying to improve them a little bit more

and progress continuing then continue

then continue

but it was all all of those so now I'm

going to get a little bit technical just

slightly technical for the I apologize

so all the success of deep learning up

until this point was in something which

is called supervised learning

it's a technicality it's very familiar

to those who are

um who have some for experience with

machine learning

or everything was about supervised

learning

in the first half of the 2010s it became

accepted

that if you have a neural network and

you do supervised learning it will

succeed and supervised learning means

that you know exactly what you want the

neural network to do

but then unsupervised learning which is

the much more exciting idea that you can

learn just from General data about the

world and learn everything somehow and

understand how the world Works without

being told without there being like a

like a teacher telling you what you're

supposed to learn that was not done yet

and then at open AI we had a sequence of

projects the first one was

with a sentiment newer and I want to

just explain that because that was an

important project

in our in our thinking

where we've shown that when you train a

neural network to predict the next word

in this case the next character

in Amazon reviews

one of the neurons in the neural network

will will eventually represent whether

this review is positive or negative

represent the sentiment but the

interesting thing here is that the

neural network was not trained to

predict the sentiment it was trained to

predict the next character and so that

project validated the idea that if you

can predict what comes next really well

you actually have to discover everything

there is about

the world or the the data source all the

secrets which are hidden in the data

become exposed to the neural network

as you can guess what comes next better

and better and better

and think about it like there is an

example which I've used a number of

times which I found that people uh like

were like imagine if you're like an

extreme example would be if you were

reading a book and some kind of a

mystery novel and on the last page of

the book The Mystery is revealed and

there is one place where the word or the

name of you know some key person is

revealed

if you can guess that name then wow

you've understood that novel pretty well

and so the neural network is strained to

predict what's going to come next to

guess you can't really you can only

narrow its guesses and have sharper and

sharper predictions

and that led then the scale up of that

led to GPD one and then gpt2 and gpt3

and then

you know with gpd3 in particular

it was a very surprising and a result

because of the really cool emerging

capabilities that showed up and then

further work and improvements and scale

out of led to gbt4

so I would say this is how we got to

where we are right now and obviously the

way everyone thinks about neural

networks is very different from before

if before

it just wasn't clear to

people that this stuff works I think it

is very clear to people now and in fact

right now we are grappling these

questions of well it works too well it's

going to be smarter than other than us

eventually what are we doing about that

right yeah so that's correct yes you

know yeah for sure so thank you for the

historical perspective and and obviously

you've been in you've been in key in

some very interesting key points to the

development of neural networks which was

fascinating to hear from you about it

so maybe you can elaborate a little bit

about how do you think the field of AI

will continue to evolve and many

advances in the in the future

and what do you think should we do in

order to take to ensure it's responsible

development

so my expectation

is that the way the field will evolve

is is as follows

I believe that in the near to medium

term

it will be a little bit like businesses

youth

where I expect

that the various companies that are

working on their AIS will continue to

make their AIS more more competence more

capable smarter more useful

I expect that AI will achieve a greater

a great and greater integration into the

economy more and more

tasks and activities will be assisted by

AI

that's I would say this is the near to

medium term

in the long term eventually we will

start to face the question of AI that is

actually smarter than all of us

with the super intelligence and that

starts to bring you into the domain of

Science Fiction but in reality

rather the idea is that people have

speculated about in the context of

Science Fiction become applicable

so at some point if you imagine a really

really smart AI

that is a scary concept

and as companies that are moving towards

it

will want to

have some kind of rules some kind of

Standards some kind of coordination

around

whatever it is that needs to be done on

the signs

on the

way that we use those AIS and how

they're being deployed on the way that

they are secured

so that we actually get to enjoy this

amazing

future that AI could create for us

if you manage to address all these

challenges so I would maybe phrase it

this way I say I get smarter and smarter

the challenges like the opportunity the

amazing things you could do increases

but the challenges still become

extremely dramatic

the challenges will become very

significant

and I think that everyone who's

developing this will be will somehow

be working together

to Grapple with those challenges to

solve the technical problems and the

human problems

to mitigate and to manage them I expect

that that's

rather I think that's something that

could happen

and I would really like for it to happen

back to education uh

we wanted to ask you how do you see the

future of Education especially higher

education

and uh you know AI tools and education

how it will impact the the the processes

to digest information to make it

accessible for uh students or for you

know the teachers the whole thing is

going through kind of transformation now

and would like to hear your perspective

about you know how it will impact the

curriculum and the whole ecosystem of

Education

yeah

so I mean I can I can you know I can

tell you that my my kids are using the

you know check GTP as an assistant for

their studies but that's you know that's

just a small example if you can take it

for a broader perspective

yeah

so I can talk about the near and medium

term

because I think there you can make

some educated guess is about what will

happen

and I think at this point it's pretty

obvious that we're going to have really

good really excellent AI tutors you may

maybe take

a little bit of time to really iron out

with the various

iron out the I guess issues to make it

really good and really reliable tutor

but it will be possible

so you could just have an amazing

private tutor that could answer detailed

questions about

almost any topic and help

help with any misunderstandings that you

might have and that's going to be that's

going to be pretty dramatic obviously so

like we go from having

being a student

requiring

to interact with one teacher and maybe

wrestle with books on your own to having

a really good teacher that can help you

with the

subject matter

write and answer your questions and

that's very interesting but um

and so I would say that this

all the students obviously going to use

that they'll want to use them

now I think a related question for

higher education or education in general

is what to study

because the nature of

the jobs that we would be having do

change

and I think that

probably being a really good generalist

who can

study new things quickly and be

versatile

and it can it and be very comfortable

with these AI tools I think that will be

very important for the near and medium

term

long term I don't know

but for the near and medium term I can

make that same

so I think now we will switch to Hebrew

if it's okay with you yeah certainly

so divisions what

um

foreign

you tell me that I

Gua

the amateur

s it is

a

pashup

like

Angeles

foreign

Ty

is

um

[Music]

foreign

itomer

shitzu

is statistics

foreign

foreign

because I share

images

they

Allah

statistics

open source foreign

[Music]

foreign

foreign

foreign

[Music]

foreign

[Music]

foreign

not

[Music]

coming

now holistic

it's a it's fine

[Music]

foreign

foreign

foreign

foreign

[Music]

machine learning

foreign

[Music]

foreign

[Music]

[Music]

very often clearly

foreign

upside down

almost

foreign

[Music]

in Cola

um

GPT

statistics

foreign

foreign

foreign

foreign

[Music]

AI

is

my own

um

atsuma

the law

living

beneficial

commercial today

line of shoes

English effect

volume

foreign

foreign

[Music]

as

an important

videos

[Music]

foreign

for taking time out of your very busy

schedule

to be with us today and speak about your

journey and the involvement of openai

let me add a word

to your comments although Israel now has

some natural gas

its key resource remain its human

capital and it must continue to invest

in it in order for Israeli to remain a

global Innovation leader

higher education in particular is the

critical investment needed to enhance

Israeli skill set and its ability to

innovate

in that regard we see the open

University with 53 000 students by far

the largest of its 10 accredited

University in Israel with nearly 40

percent of students studying stem

the open University is by far the

largest educator of Highly skilled

Talent into the Israeli Innovation

economy

educating press one quarter of all stem

students studying

across all Israeli University and with

80 percent of students at its open

University being first generation in

their family to attend University

including many who came from geographal

and a social periphery of Israeli

Society

it is also broadening the pie of who can

access higher education and thereby in

parallel addressing some of the Israeli

demographic and Social Challenges

among Israeli most vital institutions

that when that tremendous positive

impact of the open University on Israeli

Society is invaluable

I want to thank you our listener for

showing your commitment to Israeli and

the topics discussed here today thank

you all

thank you