Everybody, welcome!

Great to see all of you.

I have a special treat for you.

We have two sessions.

We have so many great speakers for you.

We broke it up into two sessions.

Let's get right to it.

I think

we love a world where there's proprietary products,

but we also we also need a world

where, a whole bunch of companies and different industries

in different domains need models as a technology,

that we could then transform into products.

So today, really what I wanted to celebrate is not the

proprietary versus open,

because I don't think that that's a thing.

Proprietary versus open is not a thing.

It's proprietary and open.

And we we love the fact that there are

these terrific models that are really products.

Folks think that there are only two

types of companies up at the software level of AI.

I think that they think that there are foundation model companies

that build very large general models, sell access

to those models through APIs,

do lots of products in different verticals

and then there are application companies

that don't really do much AI work themselves,

but they build great products on top of the models.

And I think that we are really seeing the birth and flourishing

of a third type of company that sits in between the two.

A company that uses the best the market has to offer at an API level,

but then also does great work on the modeling front too,

and takes both the best the market has to offer

on an API level,

their own models, and wraps them all into one,

into the best product for a certain vertical.

And in particular, I think we're seeing the rise

of a new type of agent happening over the course of the next year or two,

where the way you used AI models,

it started as you're just calling a model.

And then it got a little bit more complicated.

It became calling a model,

and that model has a bunch of tools that it can use.

And I think we're soon going to see agents really be coworkers

that can take on tasks that take many hours or many days

and do incredibly complex workloads.

We've experimented with things in this domain, in software,

where we've experimented with building, for instance, prototype browsers

from scratch over the course of many weeks, entirely intent with agents.

And when you start to get to these much more complicated workloads, under the hood,

you want to be farming out that workload to different models,

because different models have different strengths.

And they're going to be times when you want to use the computer,

use abilities of a foundation model from one of the APIs.

And there's going to be other times where you want to use

the industry-specific intelligence you have as a company

that's focusing on one domain to format your own models.

And so I think we're going to see the rise of these compound agents

that can be smarter than any one model on their own and mix them all together.

As Jensen said, AI is not the model—it's the system.

It's the computer.

Perplexity computer is the idea that

you should build the orchestration system of everything AI can do.

Every single capability—coding, writing,

generating multimodal content.

So what you want is a multimodal, multimodel

and obviously multi-cloud orchestra of every single

two-model file system connectors put together.

So that all you got to do is delegate your task.

You don't have to worry about which model is good at what.

It's for the orchestration system to figure it out.

These sub-agents are like musicians,

and the models are just instruments, and the work that AI gets done

for you is the symphony or the music that they play.

And that makes it all pretty simple.

And, you know, you get to basically

think of any task

that AI can do today, any different model.

You don't have to feel any vendor lock-ins.

And as Jensen was saying at the beginning—

It doesn't have to be a dichotomy between open models and closed.

We have open models in computer.

We have closed models in computer too.

And there are different needs for each of them.

Open models tend to excel

at being very token efficient, cost efficient.

Closed models are very good at orchestration,

reasoning, two calls, and basically what's happening is

models are essentially becoming just tools like file systems and connectors.

And we're able to operate at an abstraction about models.

Finally.

The first one is that

model companies are not actually model companies.

Like, they don't just build a model. They build a whole stack, end-to-end.

So when you're buying a model from a proprietary or otherwise,

you're actually buying the chips, this orchestration software, the inference

and product—and all of that has been optimized end-to-end.

Now that yields great products.

And what openness allows is for other people

to basically optimize the whole thing end-to-end.

So you're not just buying a model— you're buying a whole system.

I think the other big misconception is on open models,

that somehow open models are fundamentally

going to be behind the frontier, that, you know,

the hit that you take when you adopt an open model is you get control,

but you get something that’s a few months behind.

I think that's just an artifact of the time where we are today.

There's nothing fundamentally different between an open and a closed model.

And, you know,

models in general in this system, these are

it's fundamental knowledge infrastructure.

And fundamental knowledge Infrastructure yearns for openness.

You know, like a like an animal, you know, it yearns for the hills

or for the forest, like it wants to be open.

Like books used to be closed, and the printing press made them open.

Science used to be you know, the real alchemists,

and then the scientific journal made it open.

And strong encryption,

there was a debate between closed and openness.

And actually, it ended up being both that started out closed,

and then there's a whole flourishing ecosystem of strong encryption

that became open. And I think the same thing is about to happen in AI,

where there's a flourishing ecosystem of powerful, closed models

but equally capable open models

that are going to be coming over the next couple of years.

I think that's really exciting.

One last thing I did is that, progress is extremely fast.

We are on an exponential and

everything is very compressed, and there is a lot to learn.

There's a lot of study to be done, and it cannot be done completely

in the large labs.

Because there are tons of smart people out there.

But they're lacking access to knowledge, access to tools.

And this is where openness can be very helpful.

And it doesn't have to be just models—

also infrastructure data, general research insights.

And this can enable a ton of people out there

that can study various aspects of research.

And it advances the science of AI, science of intelligence.

I see this as a very positive sum.

Pretraining is memorization and generalization and some basic knowledge.

That basic knowledge gives you the foundation to go learn skills.

If you didn't have that basic knowledge, you wouldn't even be able to teach.

I mean, you know, you can't teach

someone how to be an engineer if they don't have any basic skills

in math and science and, you know, some basic understanding of technology.

And so the pretraining part was just the beginning.

Most people misunderstand that,

in fact, all of your labs do enormous amounts of model development,

particularly in post-training and the post-training area.

If you were to think about the amount of computing scope in the future,

the amount of computing use in pretraining

was like 90% of training two, three, or five years ago.

But in the future,

the amount of training percentage in pretraining is probably going to be tiny.

It's going to be mostly post-training.

It is also probably the case that

these proprietary models are going to be the best generalists.

But it's very unlikely they're the best specialists.

And most value is derived from specialists.

We need generalist capability all the time.

And they're going to get better and better.

And as you say, when we integrate them into a system, you get the benefit

of an insane generalist as well as an incredible specialist.

The first inflection point that I believe

at least I saw that basically made me switch from theoretical physics to

AI was a system that my co-founder Ioannis has helped build called AlphaGo,

which was the first super- intelligent agent at scale,

and it was a 60 million parameter network.

It was tiny, relative to what it is now. And it beat

the best kind of player in the world at Go.

And the thing is, that system never stop learning.

It was just an economics problem.

How much compute are you willing to put in to get it,

you know, to be 10 times better?

And at some point, they cut it off because, you know, would you put

10 billion more dollars to make AlphaGo beat Lee Sedol even harder?

Probably not.

But you know,

now that RL has started working on language models,

those are the kinds of questions we'll be asking now, and a year from now, of

am I willing to put in $10 billion,

$100 billion to solve, to cure a particular disease, right?

When you have RL working at scale,

the things that you can solve, because these are mechanical brains

with endless capacity to learn, become just a matter of economics.

And we're seeing the first generation of that

in coding, in agentic kind of enterprise applications.

But we will be coming to a point where there's

kind of fundamental scientific problems, and we're just making economic decisions

of whether we want to allocate those resources to have a breakthrough.

And now we're at a point where models are extremely capable.

And in order to unlock usefulness, you need to work on this orthogonal

capabilities—connecting the context, being able to operate

within your data, within, you know, what you're trying to do in your domain.

And this is an example of that, having an agent that sort of

can operate in your domain within your data,

connecting everything together, building this system

that's more of an operator.

And we have a long way to go to actually make

this very reliable and take actions

that you want to do with intent that you have.

But it sort of like shows that capability

and where we need to go, in terms of unlocking more usefulness.

The models and the systems orchestrating

the models are going to get much more capable.

And so you'll be able to have personal productivity agents

that can take on more complex tasks that run for longer.

And I think that also these compound agents

they will get much better at using tools.

Open models is how we got known, with Mistral 7B in 2023.

And effectively, we started Mistral to create

an enterprise business on top of an open model foundation.

And we've been lucky to work with you

on Mistral Nemo, which was a 2024 release.

Which is quite good.

And the reason why we believe that open wide models

should actually be the basis for building all the AI software in the world

is really two things.

The first is control, because the AI agent sits at the execution layer.

You want to have control over where this gets deployed.

You want to make sure that you have the turn-on button

to all agents that you're deploying in your company.

And so if you actually own the systems entirely,

so from the models to the orchestration layer,

and you know what code is actually running and you can actually modify it, suddenly,

you're much more confident than if you're depending on only APIs

that can be turned off or it can fall into certain problems.

So that resilience is actually very important.

The second reason is really customization.

Agents are great when they're operating in the virtual world,

where they're operating on knowledge.

They are great on health care because they see a lot of documents.

But whenever you have something where you have a physical footprint,

you have a machine, you want to model a physical system,

you have an engineering team that is actually to build things,

in the physical world.

Well, suddenly you have a lot of IP

that you want to put into the models themselves.

And it's never going to go back to the general purpose models

that are trained fully on the virtual world.

And so you suddenly you have access to models that you can modify

to whom you can plug,

time series, stream, etc.

You can actually build agents that understand the physical world

and actually make them useful for teams like

engineering teams that currently do not benefit a lot from generative AI.

So I think that customization aspect, we've released the product called Forged

that is precisely meant to

connect models to various sources of data that may come from the physical world

is, the second reason why open models are very important.

And I would say, like, finally, open models are great because it allows us

to make cheaper versions of everything, by working together, by sharing R&D cost.

That's the new neutron and coalition that we're happy to be part of,

by having an open ecosystem of people that have aligned incentives

to create assets that are going to be great for humanity.

We can actually accelerate progress and make sure that everybody

gets access in a fair way, across the world, to artificial intelligence.

And so that's why I think it matters a lot.

Open models are strictly better than closed models.

Because I actually think in some contexts, closed models are great.

But there is one context in which open models are extraordinary.

If you think about all the applications that you guys just mentioned,

they're increasingly mission critical.

You know, 3 or 4 years ago, I think the questions we often used

to get is, will AI be useful for anything more than chat bots?

And now because of RL and agentic infrastructure,

these are getting useful at mission-critical applications.

And the thing about mission critical applications is you’ve got to trust

that they will perform the way you do in a high-stakes situation,

where the margin of error is very low.

And the reality is, if it's not an open model,

and you don't control where,

you don't know where,

you can't introspect it,

you can't host it, and you're dependent on third parties for that.

Hey, they're good at some things,

but at the end of the day, you're delegating trust.

And I think that will become more and more a topic of conversation.

As those of you who are deploying agents in critical industries,

I think we will come back to trust.

And it's much easier to trust an open system

that you can introspect and you can say, look,

this parts of this system are closed and they're good. It's good at that.

I'm going to deploy it. and I'm gonna have some guardrails.

I'm going to manage the risk.

But at the end of the day, especially in healthcare, defense, you know,

places that if we want to welcome these agents into the most

mission-critical parts of our lives, I think we're going to have to find a way

to trust them.

And as far as I know,

open models are one of the fastest ways to trust a system.

Now, the flip side is, I think the infrastructure also has to be open.

And that's not happening as quickly as it should.

We are heading into a phase of the industry

where infrastructure is consolidating very fast.

If you study the history of the industrial revolution—you asked us

to consider the industrial applications— and I think we are in a new era.

But there's some clues that the the 1800s gave us, where

if you looked at factories that had a steam engine

and knew that you could make interesting products,

they were starting to realize that, hey, the inputs to production

are pretty critical. And they started hoarding them.

And if you just do a flyby of 1800 Victorian England,

you'll see factories with generators running at half capacity,

because everyone's hoarding generators and trying to run their own generator,

and you have piles of coal just stockpiling, not being used.

And I think what we need to go to is open models and open infrastructure.

We need to develop a mechanism that allows us to share,

not to overprovision per peak, but have enough secure infrastructure

that we get our baseload, but then they can spike up and down.

And that's called a grid.

And that's what we're working at AMP. As you know, we're building an AI grid.

But I really do think for open models to be developed at the frontier,

we're going to need open infrastructure too.

When we have a language model

or an AI system these days, we just look at the final snapshot

of this whole very long, curated process, of how these models are developed.

However, we argue that or I argue that there is a long process

and releasing different parts of it, enabling

researchers and developers to use different parts of it,

it enables infinite customization, not just the final way,

but also on top of different checkpoints of the model.

So my team built Olmo, and through that, we developed model flow,

which we call it the full development cycle of the whole process,

including the data, all the model weights, all the infrastructure, and everything,

so that researchers on hard-core developers could actually play with it,

integrate their finding, and even build on top of that.

So why is this important?

So this is particularly really important to make advances

in the next generation of AI.

Because these days, you might argue that the progress in AI

is getting limited into a few close labs, but it's actually very important

to the vast majority of academia and researchers or kind of nonprofit

and other places who want to also be part of this progress.

And we've seen that these

all this progress already has happened by everything being open.

So for example, you've probably all seen that

hybrid models are being used in many of these open models,

and even closed model like Nemotron, a very good example of that.

So most recently, we—actually our team— has started releasing and studying

why hybrid models are probably only better than only transformers.

And based on analysis and more theoretical findings, we actually found that

okay, there are theoretical reasons why hybrid models are better.

And then empirically we show that yes— it actually is much more efficient,

much more token efficient in training.

So having access to this infrastructure, training, and being able to kind of

publicly talk about it, and also, kind of study these models,

enable the next generation of even model architecture, better AI systems, and so on.

So enabling research is also a very important topic.

Open models are strictly better than closed models.

I think they enable diffusion of innovation, research,

and competition. I guess like competition in the end is good for like for you.

Right?

And, in addition to that, it's really also,

I don't know, l’m here in Silicon Valley, every two months,

then I'm based in Germany for my usual day-to-day work.

And I come here now and just like this weird AI psychosis going on, where people

even like AI researchers, think that they cannot really contribute anymore

because of the advances in coding agents, stuff like that.

But I think

even if you're, like, really convinced of that, then like

the way that you can have impact should be thinking about

how can I make an open model that actually can replicate these capabilities.

So I think, it's actually

one of the most exciting times to work on, you know,

like the frontier models, the big models, or more specialized open models

that then get deployed, like on device and all that.

And there's like so many different frontiers,

and all of them should have some open component.

Even in a company that is a closed model company,

I really believe that open models will be used

as part of the agentic system, where the closed model is your crown jewels.

And I'm fairly certain that even in

visual intelligence that used to be text-to-image in the future.

well, you're already there now— increasingly, it's in robotic

visual intelligence system.

and so it's reasoning, it’s trying, it’s taking inputs,

and the prompts could be text and many other conditionings.

It's solving problems and ultimately generating

an image or a stream of images

that you can even interact with in a very robotic way.

And so I think the future of visual intelligence

that you're working on is just so exciting.

And even though it's a proprietary crown jewels today,

you're going to surround it with a whole bunch of open models.

It's my prediction.

And so all of these different industries

are starting to get solutions that are very impactful and very capable.

And so I think that this year we're going to see, really instead of

the question, “What is the ROI of, you know, the last three years?”

All of the questions are, “What are the ROIs’ of AI?”

I think now, this year, starting with with coding, of course.

And coding is not just software engineering.

Coding is the description, the codifying of business processes,

codifying basically the rules

of all businesses and all engineering and almost all work.

And so coding is a very big deal. But nonetheless, we're going to see an

inflection this year of

real business economics taking off.

And you guys are all experiencing that.

Ladies and gentlemen, hey, let's get the last crew back up here.

That’s it.

The future!

Thank you, guys.

Good job!