Get started with the Interactions API
149 segments
The interactions API is now generally available.
Now, this is exciting because it's
a unified interface for all the Gemini models and agents,
allowing you easy access through a single API.
And this is really, really powerful
because you can see here you can create
an interaction with a model.
So this will be synchronous.
You make this call and you'll get back the response.
Or you can do this where you can have
an anti-gravity agent perform something really,
really difficult. For example, in a remote environment,
multi-step installation, setup all that single API call
on the interactions API.
So there's tons of things that you can do now.
Managed agents is the most recent release
that's available on the interactions API.
You have background execution tool improvements.
You can mix in build tools with your own tools as well.
You have Deep Research built into this
as well because you can run it in the background media
generation.
You can generate images with Nano banana.
You can generate music with lyric 3,
you can generate speech with TTS.
So tons of good things that are in here,
and it really makes the development so much easier
across all the Gemini models.
And also it is an agent first ecosystem.
So you can simply grab the Gemini interactions API skill
and get running with your favorite coding agent.
So let's do exactly that.
So here in my anti-gravity Ide, I can simply go ahead
and I can install the Gemini interactions API skills.
I will install it here in the project so that you can see it.
And so now we can also install it globally of course.
So here we have our reference.
We have our skill.
And so our coding agent will know all the good things
to do when building with the interactions API.
And what's also really exciting is that it is a stateful API.
So if you're building something that
builds upon the previous interaction.
You can very easily just pass in the previous interaction ID.
So let's do exactly that.
Please build a simple application
that allows me to edit my photos using the Gemini interactions
API.
I want to be able to put in a photo of mine,
as well as a text or a speech prompt,
and then I want to iteratively get image generations
and just iterate on them with the previous interactions ID.
All right, there we go.
And now, anti-gravity obviously also
has audio understanding built in, which is really cool.
And so you can now see that we're looking at our skills
here.
Our Gemini interactions API skills and anti-gravity
is going off and building our application here.
So yes let's allow this.
And so first of all we get our integration plan here.
So we can see yeah we'll need an API key env file.
We want Gemini 3.1 Flash image.
Yep That's good 3.5 Flash as well.
Yeah we'll just use simple NPM.
Yep That's good.
Server JS fantastic.
Yep that looks good.
So let's proceed with that.
OK go.
So we got a server JS file here.
And we can see we have an API backend here
that's being built for us.
And that has our previous interaction ID.
We'll see here our model that we'll use.
And yeah.
And then we can see our interactions create here
with our payload.
Fantastic our model.
Yeah that looks good.
OK fantastic.
So let's accept all these changes.
We will need to set up our env variable here.
So what we can do is we just can simply go to ai.dev.
So in ai.dev we can go to our API keys
and we can grab maybe one of our API keys here.
Great So I put in the API key.
Let's do a quick NPM install.
And then NPM run dev.
OK cool.
So here we go.
Anti-gravity Canvas.
We can drag or drop our photo in.
We can select the aspect ratio the resolution and then
our prompt OK sweet.
So let's drop my photo in here.
And then we're just going to say change the color of the cap
to be green.
And so now we are going to generate our hour edit.
So in the first step, we're transcribing the voice note
and then we are doing our image edit.
And so if we look into the logs here,
we can see OK we've got the input.
So that's our image input.
And then we have our transcribed text.
So in the first step, you can see here
where having an A transcribe API endpoint.
So that is also using the interactions API.
Just Gemini 3.5 Flash here to transcribe what we said.
And then we're passing that into our image editing transaction.
So Nano banana does currently not support audio as an input.
So we'll first need to transcribe it.
And then we can edit our image using the text prompt.
So you can see here change the color of the cap to be green.
And so now this is nice.
It gave us this little slider to see the outcome.
And so now here the real powerful thing
of the interactions API.
Now change the shirt to be pink.
And so now this time we actually don't need to pass in our image
anymore.
But we can simply just pass in the previous interaction ID
here.
And we will see, in combination with the prompt we have access
to the cached tokens.
And now you can see the cap is still green.
We only wanted to change the shirt to be pink.
And this is really cool Yeah Really awesome.
So you can see here and then obviously you
could go back to this, actually didn't like this.
And you can branch from here.
So you could say.
Actually change the color of the shirt to be green as well.
And so now I'd hope because it's the cash tokens,
it would use the same green as the cap.
Let's see if that is the case.
There we go.
No, it is using a different green.
OK, maybe that's a bit too much green.
But yeah, this is really, really cool.
Here the interactions API.
Try it out now.
Really recommend the coding agent skill.
It's a really, really good way to get started
with the interactions API.
Awesome let me know if this was helpful
and I can't wait to see what you will build with the interactions
API.
Cheers
Ask follow-up questions or revisit key timestamps.
The Interactions API is now generally available, offering a unified interface for all Gemini models and agents for easy access. It supports both synchronous calls and complex multi-step tasks through a single API. The API is agent-first, integrates tools, offers Deep Research, and enables media generation like images, music, and speech. A key feature is its stateful nature, allowing developers to build applications that iteratively refine previous interactions by simply passing a prior interaction ID. A demonstration showcased building a photo editing application where edits could be continuously refined without re-uploading the original image, leveraging the API's state management.
Videos recently processed by our community