How's it going, everyone?

My name is Logan Kilpatrick.

I'm part of the Google DeepMind team.

Welcome back to Release Notes.

Today we're here

in the Gemini Robotics lab,

with Carolina, Stuart, Kanishka, and Jie.

We're talking about the new embodied reasoning model and actually

the whole suite of Gemini Robotics launches that are coming out.

So I'm super excited. I have a million questions.

We were talking off camera.

But Carolina, maybe you can kick us off

with sort of the headline for this moment.

And then we can actually take a step back after that and

talk about the arc, maybe, across all the work that it's taken to get here.

And we'll get to see cool robots, which I'm excited about.

- Yeah, definitely.

What we're building here is the intelligence layer to power

any robot to do a broad range of useful tasks.

And we've been building towards this moment for a very long time, actually.

Our team has been working

towards general-purpose robotics

since the inception of the team.

And we had a long history of always thinking,

how do we solve the problem

from first principles in a way

that doesn’t just take shortcuts

and tries to solve the entire problem?

And so the entire problem

really means infusing

a robot with human-level intelligence, right?

So that it can understand the environment as you and I can,

so that it can reason about what it means to complete a task.

And then it can actually take action with all

the dexterity that us humans have and take for granted.

-Yeah. -So that's the general goal.

- I love that.

And the actual release that we're talking about today

is the new package of Gemini Robotics 2 models,

starting with the ER model, but a bunch of other stuff as well.

Do you want to talk through the different models,

the suite of models that are becoming available?

- Yeah, so Gemini Robotics 2 essentially

is bringing whole-body intelligence to robots.

So what that means is that we are enabling

a model that can understand where the entire, every part of the robot

is in space, and it can reason about doing more complicated tasks.

So, imagine that you get in your closet and you're

trying to clean it up and put away all your clothes and your shoes.

That requires you to move your body in all kinds of ways, avoid obstacles,

reach things high, pick things from the floor,

this is something that we could not really do before

in a way that understands what's going on.

We literally put the robot in a garage and ask it,

"Can you please clean up this garage?"

Right? And then—

- Is that the real prompt in those examples?

It's literally "Please clean my garage?"

- Yeah, you'll get to see it. Inspired by our own needs, I think.

- I love it.

- So, then the robot needs to reason about what

does it mean to clean up a garage?

It needs to think about,

"Oh, I'm going to put all the cleaning supplies in the same place."

It needs to be able to

put things high, up high.

If something falls on the ground, it has to understand that,

and then go pick it up, and then put it away.

So that's one of the big things that we're bringing in this release.

The second thing that we worked significantly on,

and we still have a lot more work to do, is dexterity.

So, again, us humans take for granted

this really dexterous hands that we have,

but pretty much everything you do every day requires dexterity.

And that means folding things, opening doors,

just picking anything that will fall out of your hands.

And so, dexterity is an area that we worked on a lot.

And if you just, again, if you just get yourself, in the morning, coffee,

that is actually a pretty dexterous task.

- Yeah. - And one thing to know is that

because we're building this intelligence layer across robots,

is we definitely work a lot on making this work really well

on humanoids, but we also bring other robots that we're controlling.

So, the one that is right behind you is actually

our Franka Duo and it has two Franka arms.

And we also control with the exact same model,

this robot in order to do pretty dexterous things with those grippers,

like pack things neatly.

So, that's another form of dexterity.

And then the third thing that we bring in is what we call

multi-robot collaboration,

which is different robots have different capabilities, right?

And what we're doing here is that you have, you're bringing the robot

the intelligence to know what it needs to do to complete a task,

but also the understanding that it can call

other robots to accelerate the task,

or to do things in parallel, and do things faster.

So, those are roughly the three things that we're doing in this release.

- I'm excited for this. I have a home robot vacuum,

and then I have another robot that I have ordered.

Hopefully it'll be powered by Gemini Robotics 2,

that doesn’t vacuum but does a bunch of other stuff.

And so I feel like this is actually going to be—

I did not think about this today, but I was thinking to myself,

I was like, can I get the one to sort of carry the other one around,

and go deploy the robot to go do certain tasks?

But ideally they could just communicate as robots together

and get the work done, which is super interesting.

Maybe we can also talk about the arc to get here, and obviously,

this is the second Gemini Robotics model,

but is there other things that are worth--

Obviously, Google's been doing a lot of robotic stuff.

DeepMind, maybe, I actually don't know.

Maybe also had a bunch of robotic stuff, but you can talk about any of

the research and the arc to get to this launch moment?

- Yeah, I mean, it's been a long path.

I think many of us have been here

also for a while and have seen all of these steps through.

But yeah, I mean, we've always, from the beginning, like I said,

we were really thinking about general-purpose robotics

before, I think, the field really realized that was feasible.

And so we've done a few iterations where we've brought

techniques that now are table stakes for the community.

So for example, we first introduced reinforcement learning,

to learning simulation, how to control whole robots, right?

So you see a lot of robots today, dancing and doing acrobatics.

Those robots are actually using those techniques in order to move the robots

in a way that feels stable and that can mimic a particular sequence.

We've also shown what's possible when it comes to bringing

LLMs and VLMs, when it comes to planning for robots.

Before that effort, basically, robots did not understand semantics.

They did not understand our world.

They did not know what you meant when you say,

"Bring me a cup." - Yeah.

- Right? You actually had to say, "No,

bring me that object at position X Y Z, in space."

So, that was one clear breakthrough.

We also introduced transformers to robotics, and that shifted the field

into this era of data-driven robotics, where you now had to just

collect a lot of data, and teach the robot how to do many different tasks.

And then introduce even the concept of a VLA,

which is a new type of foundation model called

Vision Language Action (VLA) model,

that essentially enables robots to understand natural language

and visual input and then directly control them in a way that is general.

So, this VLA type of foundation model

is again, adopted in the community.

And then we've also shown what's possible when it comes to dexterity.

I think many of us did not really think that it

was possible to tie shoelaces, for example, in our careers.

And that's something that, today, people tie shoelaces,

they fold their laundry.

These are things that we showed that was possible.

And I think all of that has come together

towards this Gemini Robotics models that we introduced last year.

And so last year, what we did was we brought all of the power of Gemini's

multimodal world understanding, combined with those techniques

in order to bring essentially Gemini's intelligence to robots.

And all it is, is that we

enable Gemini to also think about moving robots.

So, we add actions as a modality in Gemini.

And so that essentially enables Gemini to understand when you ask it to

turn around this, it understands what it means to move around the bottle.

Or if you ask it to do something more complex,

pick up all the things that are pink,

then it understands what that means.

So, that's essentially what Gemini Robotics is.

And then we continuously improve it.

And Gemini Robotics 2 is a pretty big

step function with respect to our previous one.

- I love that.

- I think, yeah, our team

aggressively thinks about this problem

from a frontier model, lab perspective.

So, really leveraging,

you don't have to

solve robotics from scratch.

There's all this world understanding

in these big models.

So, the idea is how do you

latch on to that understanding

and then use the robot for useful things?

So, you've had this history of using data as a scaling paradigm, but now,

it's more like these frontier intelligent things,

how do you hook up this

physical thing to those models and then

kind of bootstrap robotics from those?

- And what ends up being the limitation in practice?

Because I'm thinking, for example, I've seen the demos of the robots

dancing and all this crazy stuff, and doing backflips, which I cannot do.

But then you also see

for AI agents, the meme canonical demo is like booking travel.

And I feel like for robots, it's folding laundry.

But yet, it feels like also maybe the robots cannot actually fold laundry yet.

Maybe that's not true. Maybe Gemini Robotics 2 closes that gap.

But I'm curious, there's so much understanding

and intelligence baked into the model.

What's the place to actually hill climb

to get to the place where you start to see

robots folding laundry successfully in most cases?

Or actually are we already there?

And maybe I'm not fully calibrated on model progress in this regard.

- So, there is a lot of

progress in the last ten years.

So, one thing that you mentioned

is locomotion or whole body control

has reached a new height

where you see the humanoid robots

that can backflip and

doing very agile motions.

Also, our team pioneered a research called reinforced learning

and sim-to-real transfer that makes all those things happen.

So, it seems that the locomotion is nearly a solved problem.

What's remaining is actually a very hard problem, is dexterous manipulation,

which is that how can you use a hand or grippers to interact

with all these objects in order to accomplish the task in your daily life?

- Yeah.

- The reason that's very complicated is because it's very contact-rich.

So you need to think about a lot of contact points

on the object in order to move them in a desirable way.

And to control your hand, it has over twenty degrees of freedom.

So, you need to coordinate all these joints and muscles

in order to do those tasks.

And all these things are way, way harder than locomotion problems.

You only need to control yourself, on usually a flat ground,

or slightly perturbed ground.

So, dexterous manipulation with all these different objects

in real world is really an unsolved problem for robotics for now.

- Interesting.

Where is the quality gain come from these days?

Is it we just get more data or it's

new techniques or just scaling up general purpose models?

How do we actually,

it's an unsolved problem, but where do we make

progress on actually solving it?

- I think data is a big part of it.

As Jie mentioned, we're missing this internet of physical interaction data.

As I open this cup, there's a sequence of,

I make a move and the environment moves in response to it.

So, this kind of sequence of interaction with the physical world,

there's no internet of this.

So, I think data is a key part to this physical AGI component

that we have not unlocked yet.

And that is an open question, how do you collect it?

Data quality really matters here.

So, I think, yeah, just how do we collect the scaled

digital version of physical interactions is an open thing.

And we're basically hill climbing that

as one of the big levers on unlocking physical AGI.

- Do we not have the ability to, and I'm guessing, actually,

one of the threads that I'm always talking to folks about is

back to this building on the world model, this interop between main Gemini

and some of these domain-specific cases.

Can you not take a video of someone unscrewing the water bottle and intuit

some of this data and get some of it out of those types of,

and it feels like there's a richness in that type of data.

Maybe there's not richness in the labeled format

that we actually need to make progress,

but have we gotten closer to being able to actually leverage

some of the existing non-robotics data to do these tasks?

- I think if you look at the data progression

in the last couple of years,

so, people usually talk about this data pyramid.

On the top of that is teleoperation data.

These are the data where you move some controllers,

and the robot is going to move accordingly,

so, this is called teleoperation.

These data are very useful because you get all the signals,

basically how you should move the robot in different scenarios

to train the robot.

But, these data are not very scalable because teleoperation is very costly.

You need a human there, you need a robot in the loop, and so on.

And people say, "Maybe we need something more scalable."

And people think about wearable device.

There is something called the UMI, which is in the academic world,

where people build these wearable grippers so that human

can collect the data without the robot in the loop.

- Pretend you're a robot in your house,

and do stuff all day.

- I've seen some of these videos, it's interesting.

- Then, those data becomes really scalable.

But the problem is human and robots are different.

There is this embodiment gap you need to cross, right?

And down beneath it,

maybe the widest base is really egocentric human data.

Basically, human— you take a video of human doing things,

then hopefully we can learn from those.

But again, you don't know how much actuation

or how much muscle force you do with each movement of humans.

So you are missing a very important label of the actions.

At the same time, as I mentioned, that human robots are different,

so it's the hardest data to leverage.

Of course, we are making progress on leveraging

all the entire pyramid of data, but we are not quite there yet.

- Yeah, I'm thinking about in the context of mainline Gemini,

for some of these use cases where like the models don't really work,

you start to see like 1,000 high-quality trajectories,

it makes a massive difference in overall quality of the model.

Do you see that type of, again, without getting into the specifics,

is it low-hanging fruit, free hill climbing everywhere?

Or is it really you actually need the reason tele--

because I'm thinking, it's Google.

We could we could go get 1,000 tele-operated,

examples of people going in, collecting that data.

But you're saying we need it would be

the scales of millions of millions in order to get any...

Interesting.

- This was Gemini,

which is pre-trained on the internet, right?

So there's a lot of nice biases there for language and vision stuff.

- Yeah.

- But whenever we add this physical thing into the model,

it doesn’t play well with the pre-trained stuff.

So in fact, that's one of the reasons why we have

our Gemini Robotics model is because it is hard to upstream something

without killing all the other generational properties of the model.

So,

this physical thing, our datasets are so tiny compared to the other

digital token sets, that they don't play well yet.

So, either we scale these up and they start playing well,

or there's some other way we can connect them.

But it's still an unsolved problem.

That's why you don't see these frontier models

directly controlling robots.

- Yeah. You inherit things like natural language understanding,

visual understanding.

I don't have to pick up a bottle that is black and white and different shapes,

like all of that generalization, which we now take for granted.

- That's true. I did not even think about that.

- But before you had to collect every single object.

So, we do get some generalization,

but I think the big part that is missing

is really understanding motion.

And motion is not something that you inherit from Gemini today, right?

Motion is something that we have to teach Gemini.

- Or understand force.

- Or force. - Interactions.

These are the things that I think Gemini is not trained on.

- Yeah. I'm also super curious, there's obviously such a distribution

of the actual robotic use cases.

And I'm curious, for us, and for Gemini Robotics,

has there been a focus?

Is it like we want to enable home robotic use cases and we see traction?

Obviously, there's a huge amount of industrial automation stuff happening.

Is there-- I know, Carolina, you said we want general purpose robots.

And so theoretically you could do all of those things,

but I'm curious, actually,

if there's been-- is it jagged as far as progress or capability,

or things that are actually working more today versus not?

- Yeah, there is, I think, two answers to that question.

I would say in the capability front, we certainly believe that you want to

be able to solve a broad range of tasks, and actually by going narrow,

you're going to build a policy that is going to be a lot more brittle.

It might work better in that environment, but the minute you change

anything about the environment, things are going to start to go wrong.

So, our approach is certainly don't compromise on,

you're trying to solve general-purpose tasks,

and understand general-purpose motion,

and handle a broad range of different objects.

I would say in terms of deployment, I think of it a bit differently, right?

I think it's much more likely that these robots are going to be useful

in environments like industrial environments that are semi-structured,

that have safety pretty much under control, and that you can get

a lot of real-world experience

of what it means to launch these models.

And then from there, I imagine that we would go to things

like retail and other environments that are also

starting to get into human-centric spaces, but they're less vulnerable

than in the middle of your house, with your kids and your pets.

- You're raining on my Q4 2026 home robot orders right now.

I'm waiting.

I've written off all of my chores for 2026,

as soon as I start getting some of these robots delivered.

So it's not—

- Yeah, I mean, there's plenty of people out there

that think that they're going to go home first.

And I think there's, the appealing thing there

is that home first requires that diversity, that generalization.

So it forces that problem, it makes it very front and center.

- And you have to check, I think we always found

it's the opposite where, if you do train on this one narrow thing,

as yeah, it becomes worse.

So having it collect on many different diverse cases,

just helps the general intelligence of it.

So yeah, from a strategy perspective and a learning perspective,

it makes sense to go super broad first.

- Yeah.

- I mean, then there could be the super fans, like yourself,

that decide to have the robot at home, even if it's a little early.

- I hope it doesn't break my stuff.

Yeah, I think it is interesting to see like what people's— because I assume

these robots will be delivered to people and actually,

I think it'll be super interesting for all of us to

just see what's people's reaction.

If it works 90% of the time and 10% of the time it's,

you know, cracking a wine glass or something like that.

Are you happy with that?

If it's a $3 wine glass, maybe, I have no idea.

Part of this challenge of scaling up data, I'm curious why,

actually, back to this home example, and maybe there's a bunch of industrial examples,

the tension and you see those-- opposite extreme of this

in the context of mainline Gemini where,

you know, there's billions of people using, et cetera, et cetera.

We sort of have a flywheel of getting signal from the real world.

Why don't we see more real live deployments of robots today in,

actually, to help us scale getting a lot of this data

and some of these ways, is just like it's not scalable?

And I'm thinking back to, and maybe this is not right,

But you see, obviously, Waymo was doing this for a long time,

and had cars driving around, not being used.

There's actually a ton of other self-driving car startups still doing

similar things today and in operation, theoretically collecting data.

I don't know what they're actually doing.

And maybe I'm wrong about this, but it feels like that isn’t the case

of what is happening in robotics today,

at least visibly to an external observer.

And I'm curious why.

- Do you want to answer that?

- We are going to see over the next two years if that changes a lot.

But I think right now, when we talk to partners

about their own experiences,

we hear two things consistently.

One is, while the policies are showing

incredibly cool generalizing capabilities,

it's actually difficult to get

them to be narrowly successful,

but broad enough

that they can handle

all the little things that vary and break

over the course of an entire day. And I think you look back to autonomy.

This is a really serious challenge there as well,

where you can give a really impressive demo,

but you're not ready to remove the safety driver for a long time.

- Yeah.

- And one of the distinctions between autonomy and robotics

is this teleoperation option.

So, you can just like, "I'll just put a safety driver in."

And if the car gets stuck, they'll just take over and drive.

For a lot of the robots, you know, we saw this with Aloha.

We could do that.

We could actually get somebody right there.

And if the robot got stuck, they could take over and fix it.

But as the robots get bigger, get more capable,

gain more degrees of freedom, which we should actually want in a deployment,

it's harder and harder for somebody to jump in and help.

And so, I think we're seeing this challenge right now,

where we're struggling to get that bootstrap system

where it's good enough that you're ready to deploy it.

And then also we're trying to figure out what is this analogy where

if it does get stuck,

how do I both fix it quickly and learn from that moment?

And so there's little patterns forming, but right now I think everyone

is really focused on how do I get this kind of baseline performance?

And so I think there's a big rush in the industry

and I get that baseline capability.

- Yeah.

- And this is part of what we're doing with our partners actually,

so, as you notice, there's many different robot types here,

and none of them were built by us actually.

- Yeah.

- So the way we work is that we have these deep partners that we work with

in order to accelerate both the AI capabilities and the hardware.

And so we're deeply connecting with each other

to see what's missing to get to that deployment.

And our goals together is to accelerate that deployment

as fast as possible and to bring it to real-world applications.

So that we can learn whether what we're learning is actually

useful and valuable and where to spend more time next.

But that's absolutely the goal that we have with our partners

is how to bring this to useful applications as soon as possible

and how to learn from it, how to deploy them in a safe manner

so that we can get as much information as possible early on.

- Yeah. Back to this two year time horizon,

potentially of where we see things changing.

I always make this comment to people that if you, ten years ago,

were put in a very short-term time machine and landed in 2026.

If you looked outside, minus anywhere where there's Waymos deployed,

and you looked into a city,

essentially, physically, everything looks the same.

like you wouldn't be able to--

maybe you'd spot a new phone that somebody had or something like that,

but more or less,

the physical world around us looks the same.

And they'd miss the fact that we actually have these

extremely intelligent, we've almost--not actually solved intelligence

but made a huge amount of progress on solving intelligence.

And so it's going to be very interesting to see

the physical world around us start to change as like you get these

new autonomous systems doing things in the real world.

I'm actually curious, assuming that we get some breakthroughs

the sort of dexterous manipulation problem is solved.

Is it basically a manufacturing problem then,

at that point, to actually just scale--

or is there still fifty other things that need to land if we were to somehow,

we could make dexterous manipulation work really, really well.

Everything out-- navigation is solved,

all the other bits of the story have been—

- Yeah, I mean, I think— first of all,

dexterous manipulation is the hardest task problem.

- So, we would all be very happy when that is solved.

But my guess is that if you want to

get to the point that I think we all dream up,

which is like, you walk around, and there's robots doing different things,

regardless of where you are, they're helping society in a useful way.

And not just in industrial settings,

but I think the second problem we're going to hit

is the human-centric aspect of it.

It's understanding humans, being able to be useful to humans,

and safe to humans in all that context.

And that is something that we're also making progress towards.

I think for them, for robots to be in everyday spaces,

there's all kinds of safety aspects that need to also be solved,

all kinds of security, privacy aspects that need to be solved

that are completely parallel.

And very different to dexterity.

But, I certainly think if we crack the dexterity in a general way,

it would definitely just blow up the opportunity

of what's possible with robotics.

- I think you will start seeing robots around once we have that problem.

We go to robotics conferences and we get a little peek in the future.

And it is crazy.

You see these robots walking around giving demos.

So, it feels like Star Wars, the future.

So, I feel like we're a few years away from that.

And this dexterous manipulation is one big chunk of that.

So, maybe there's one or two of these, and then, yeah,

then the tail of the things that needs to be solved, we'll get to, yeah,

we'll have robots around us, and maybe the cities will start looking different.

- Yeah - I think robotics is incredibly hard.

- Yeah. Make a t-shirt, robotics is incredibly hard.

- Yes. So, actually, I always get asked this question,

when do you feel robots is going to enter our daily life?

So, if you ask me like three years ago, I would say,

probably beyond my lifetime.

- Interesting.

- If you ask me two years ago, I said maybe ten years.

- Is Waymo considered a robot or no?

- No. General-purpose robot.

Enter our daily lives.

So, if you ask me now,

I think that it's between five to ten years.

So, you can see the speed of evolution of this technology is amazingly fast.

But there are still a lot of things that we need to solve.

- I'm curious actually, though, like five to ten years away framing.

Also, if you stack that up on framing,

are we going to claim

general purpose intelligence if you don't have this embodied characteristic?

Or are these two things completely separate?

- You have a biased group here.

You cannot reach AGI until you solve physical AGI.

So if I walked up to a robot and say, do anything that I could do,

I would expect it to be able to do it.

So I think that definition maybe sometimes

gets lost but we all just live it every day.

So I think robotics, we call this the Moravec's paradox,

where things that are really easy for humans are very difficult for robots.

Like, the AI is-- it's passed the bar exam,

and code up all these operating systems,

but they cannot cook you eggs, or flip a burger.

So, there's some paradox there where physical AGI,

I think is a part of AGI, at least for me,

but it is in some ways more fundamentally different

than the digital agents.

So, it'll take, I think, a bit more work to get that.

I think that will land after the digital AGI thing has happened.

- Yeah. - So, I think there's,

that's why you add the two plus three to get to five.

- Yeah, and I think it is very possible that getting to digital AGI,

and I'm sure it will, actually, dramatically accelerate

the speed to our physical AGI, right?

- Yeah.

- Not only on the intelligent aspect, but you could also use this

in order to build better robots, on the hardware side.

Because we haven't talked much about sensors,

but everything that we're using today

is primarily ignoring all of the sensors that you have in your hand.

So today, we're just using vision, and that's basically it,

and the position of the hand in order to determine

whether you have picked up this glass.

But when I pick it up, I can feel it all over my hand.

And that's something that we are not even scratching the surface on today.

And we think that if you want to be able to do everything a human can,

you definitely are going to need to have more sensing capabilities

than what we have today on robots.

So, there's an aspect of also the hardware

catching up to getting to the level

that is capable of achieving human-level behaviors and manipulation.

- Yeah, skin is an unsolved hardware problem.

- Yeah, I can imagine that being true.

- You talked about these recursive loops.

I think, that even in the past release,

we're starting to see the very first times where

the embodied reasoning model can actually watch the robot do something,

and have opinions about it.

And over time,

that actually starts to form a real loop.

You're like, I think you should go collect a little bit of different data.

Or I think you should actually, and increasingly the researchers are asking,

"Can you please provide an interface by which the higher level model

can actually give guiding instructions to lower level models?"

It's actually painful to watch sometimes because

the high level model is like, "No, no, just grab it a little higher."

And so we're starting to see these more and more.

And so I do think, as more and more of the core capabilities

of Gemini spatial reasoning, and things we do with embodied reasoning

start to really get closer to AGI,

you will get some of those feedback loops to start to form.

- Yeah, that's super interesting.

Well, let's look at a demo maybe, of the dexterous hands,

because I want to see, I want to see it come to life.

- Sure. Yeah.

So, here we're looking at GR2, and it's controlling

these very high degree of freedom hands.

I think there are twenty different joints that it can control per hand.

And the cool thing is that we trained GR to control

the whole body and the hands with the same recipe.

So, there's nothing special about the hand.

It's just like we collected much more diverse,

rich dexterous data, and the model is able to perform the tasks.

So let's take a look at some of these tasks.

- In order to be useful,

a robot needs the dexterity that we take for granted.

- You probably don't think about how to drive twenty-two separate joints

when you operate your hand, but that's what we're asking these AI models to do.

In the Gemini Robotics 2,

we came up with a set of tasks in order to test and develop dexterity.

So we're asking the robot to pack lunch

by putting the grapes into the Ziploc bag.

So this requires a lot of precision, but also a lot of coordination.

Now, the really hard part is getting the Ziploc closed.

Very nice job, Apollo.

Hey Apollo, can you unscrew the bulb?

The bulb is actually a sphere, right?

So those contacts need to be very precise.

For it to be engaged with the fingertips.

There's actually a lot of motions

and a lot of dexterity that's involved when you do that.

You have to... - That was the easy Ziploc bag, too.

I'm like, I can't even do the regular ones.

- that's actually very complicated.

Thank you, Apollo.

We're advancing what we can do even with parallel grippers.

Need to have dexterity, precision, and 3D space understanding.

- This is the robot that you have behind you.

- ...that we are trying to solve.

We can move the kit around, we can move the tools around.

The robot is going to be able to understand how to reorient

the objects in space and then precisely put them in.

The robot's task is to tie a knot, tie off the trash bag.

Multifingered hands are a key ingredient

for this kind of intricate knot-tying dexterity.

Come on, robot, you got this.

- I still have not figured out how to do it.

- Yeah, what? I've never seen a tie like that before.

- We are pushing our understanding of how robots may interact

with complex objects in the real world, trash bags, or hazardous waste.

It would be great if we could send a robot to do that,

rather than have humans put themselves at risk.

- Very cool.

And so in this example, the— this is like a partner hardware

that we've generalized the ER model to be able to

work on that specific set of hands in that context?

- So, this is the Gemini, the VLA, the action model.

- The action model. Okay. - Yeah.

So, that one is trained to then use these high dexterity hands.

So, we did collect teleoperation data to see how the task can be done.

And then that data helps the model understand how to control these robots.

- And does the dexterous hand use case generalize? Or is that also something

that as you see across all the different hardware robotic partners,

the hands are all different or the degrees of freedom

are different and that's why it makes it complicated?

- So, I think that hands are a good

place to just push the limits of dexterity,

but we are seeing some really cool signs of cross-embodiment transfer.

In our GR 1.5 release, we talked about this more explicitly where, yeah,

we are seeing transfer between the gripper task and the hands task.

So, these models, when we train it with all the data,

we don't train per-embodiment models.

GR 2 is trained on many robots.

And we do see these signs of life, like it understands basic concepts,

and it can transfer that action from one robot to the other.

- And this is actually really important because I think

robots will continue to evolve all the time.

And we see it even in all the robots that we have every year,

like they evolve in some interesting way, right?

And the hands is one of those areas that is

very ripe for a lot of acceleration over the next year.

So, we fully expect the hands to be changing constantly.

So, I think it is really important to enable models

that can work across all these different embodiments.

And fundamentally, if you're actually doing a task

where you're organizing things,

I mean, 90% of that task is not about exactly how you move your hands.

It's about understanding where you're putting things.

And then the last 10% is about exactly how you move your hand to achieve it.

And so a lot of that transfers between robots.

Of course, there is some limitations,

like a gripper can only grasp things this way,

and a hand could actually do something more complex.

But there's a lot of semantics that are shared between them.

- I'm curious actually if models having code quality

is at all correlated with some of these use cases.

And maybe my mental model is off on this, but you imagine you can

deterministically program robots in certain cases.

And so could you, I'm curious if we do anything around that,

or if that's actually a use case that's helped, assuming we get

super intelligence at code, whatever, in the next couple of years

because we're hill climbing it, does that somehow help?

You could almost deterministically program

sequences of things that the robots are doing, or is that not?

- I can give you one example.

- So, I think one place that can really come into play is in simulation.

So on the real robot, it's very difficult to write deterministic code

that can actually take in just a raw set of pixels

from a bunch of different cameras,

and actually give you thoughtful, correct joint angles.

So, you can play some games with inverse kinematics,

but it's really hard.

In the simulator,

you often have access to privileged information.

So, you actually secretly know exactly how far away

this lid is from my fingers.

And so if you can get to a point where you're really starting to

build confidence, where the simulator is actually either a source of data,

or a place you want to evaluate a policy,

now, you can basically use that code to guide

yourself in much more precisely, because you actually have access to

really correct resolution, really precise information, rather than forcing

that code to kind of interpret sort of the messiness of the real world.

- Yeah, yeah, yeah. No, that makes sense.

- Interesting.

- Even for accelerating the research loop, right?

We're using agents today, right?

To be able to run experiments, see what works, see what did not work,

plot the differences,

detect that something is going the wrong direction early,

and then change parameters. All of that is already happening.

Right. So, in that sense, yeah.

- It's exciting. - So those will not be simple code.

So I think that we have tried to write code to control robots for decades.

- Right.

- So, because those code, if it's rule-based,

it's very hard to generalize to all kinds of environments.

This is why we are switching to this very data-driven paradigm.

But in theory, all the neural networks, training,

data-driven, optimization, they're still within the code space.

We still write code to generate all these things.

So, I think eventually it's possible,

but it requires a lot of auto research to make that happen.

- Yeah.

- I was thinking about routines almost very explicit,

I guess, maybe the action space is too unconstrained to make that happen,

but could I, I don't know.

I was thinking of examples where like it might, you could do something useful,

if you could write code to do some of these use cases deterministically.

But no, I hear what you're saying that it doesn't generalize well.

Well, so how can people actually start getting access to the model?

I feel like it's, we've made a bunch of progress.

What's the availability story?

Where can people actually start using it?

- Yeah, we're really excited.

So, Embodied Reasoning 2.0 or 2 is going to come out,

that'll be directly available via AI Studio.

And I think I can say this but soon it'll be available

via the Gemini Enterprise

- Agents Platform. - Agents Platform.

- But Gemini Robotics 2, we'll have the embodied reasoning model

available directly via an API, and that will be generally accessible,

and we really encourage people to use that.

And then the action models themselves are also going to be available.

We work directly with our deep partners,

so they'll be the ones who use the biggest strongest versions.

We also have an on-device version of that, that our trusted testers can use.

And so people are welcome to join.

We currently have a waitlist-- but we're trying to do more about it--

our trusted tester program.

And then, they can actually get access to an on-device deployable version

of the action model, where they can

actually fine-tune that model directly on either

their tasks or their robots and actually try it out in practice.

- Yeah, that's awesome. I'm excited.

What--

any advice? And maybe maybe I'm misremembering this, but I feel like we

do have a bunch of customers who use the ER models who actually aren’t

robotics companies, and they just

happen to be in one of these domains for video,

audio, spatial understanding, or something like that.

I don't know if that's a suggested path for folks,

but I assume it's those domains of video, spatial understanding,

where the ER model is better on a bunch of these core benchmarks.

- Yeah, I mean, these models are better in a few ways.

The ER model in particular.

This release is a lot better at video understanding.

So before, it's always been very good,

and state-of-the-art at spatial understanding.

So 2D and 3D bounding box,

understanding where objects are in 3D space.

Now, it can understand videos.

It understands also the semantics of a task.

So if you ask it,

at what point should I stop pouring my coffee?

Or am I done closing this Ziploc bag?

It actually understands how far along you are in that progress.

And so that's extremely useful whether you're doing

any kind of like video understanding capability, or for robotics, exactly.

If you use it as your agent, then now, it can be the agent

that understands how far along you are, and decides to

switch to a different task or decide that you're done.

And then these models are also significantly safer.

This is our safest model yet, and it's safer not only in the regular way

in which all of our Gemini models are safe, in terms of content safety,

but it's also safer because it understands

the likelihood that a model is going to be completing a task.

So, it also helps you understand the— if you give

an instruction that is very ambiguous, for example, then,

it will ask proactively the human, "Oh, that instruction is very ambiguous.

What do you mean?"

And so it helps with proactive clarification.

And then the last one is that we're also

making it really strong at detecting humans,

and humans' proximity to robots, which is very important when

you're talking about collaborative robots that are in human-centric spaces.

So those are just a few areas.

We're also introducing a new safety benchmark that we have open source.

It's called Asimov Agentic.

Asimov is our benchmark and it essentially has a large set of examples,

real-world examples, where you have to make a decision about what the robot

would do next or what you should do next based on this situation.

So it's a lot about semantic physical understanding.

It's a lot about common sense that robots would need to have,

if they're going to be operating and doing lots of tasks around us.

- Very cool. This was an awesome conversation.

It was super interesting to hear about the launches.

I'm very excited for folks to get their hands on the models.

It's cool to also come to y'all's space.

I feel like there's, it's very-- much more interesting

than the normal Google offices.

So, I'm glad to be a guest and see

all the cool hard work that you all are doing.

So congrats on the launch.

Very excited. Thanks everyone for watching this episode of Release Notes.

We'll see you in the next one.