Cool. Right. So I want to talk about agency in

language, and so I want to start

everyone off by think about a conversation that you've had that you remember

from time to time, maybe with someone you care about,

someone who cares about

you, maybe one with a stranger. Do you have one in mind? Okay.

So what made that conversation meaningful to you?

What are the words that sort of replay in your head

when you think about it?

What are the concepts that stick with you?

So the language that we use and create shape us and the world around

us. When we encounter a concept that we think was placed there by someone

else

in their use of language, we feel compelled to do certain things.

I think of the things we're compelled to do by a concept as that concept's

meaning, and there are several different kinds of meaning.

So,

intentional meaning

is the meaning of a concept with respect to what

criteria sort of comprise it. So if I have the concept of a

lecture,

we might have a couple of criteria for what counts as a lecture and what

doesn't. So a lecture is a kind of language use.

It's a monologue from a speaker to an audience,

and it's typically about transferring some true knowledge

to the audience, and this contrasts it with storytelling.

And of course, you can see that the meaning of a lecture,

its intention is

built on other concepts, right?

It's built on concepts like language or monologue.

But for the sake of understanding lecture, we can just take these other

concepts' meanings for granted.

This is often contrasted with something called extensional meaning,

which is whatever this refers to in context.

So in this context, the lecture is this time that we're sharing together,

the

activity of me saying things and you listening.

And this kind of meaning, when it's invoked,

asks us to shift our attention to

a particular kind of thing.

There's a third kind of meaning that is somewhat harder to pin down, and that

is connotation. And

that's sort of everything that sort of sits outside of this concept.

This is what this concept does to the context in which we place it.

So, for example, we wouldn't put it in the intentional

meaning of a lecture

that the lecturer is someone who has authority.

But if I say Elaine is giving a lecture at a conference,

we might infer from it

that Elaine is the kind of person who would have authority to give a

lecture. So when we encounter language

that other people use, we're compelled

to sort of use these different kind of meanings to interpret that word.

And when we decide to say something, we have to search through all the concepts

we have available to us to try to get as close as we can to what exactly we

want to say. This is a hard problem.

We

can never fully say exactly what we mean, even though we try.

And because it's so hard, there's a lot of things we can do to make

it easier.

We can create new concepts. We can also have concepts that

are definitionally

vague. So vague concepts allow us to sort of

broadly gesture

at what we mean, but they don't require us to really fill

in

the meaning perfectly. So one example of this would be the

word thing,

which definitionally has no intentional meaning because we can

sort of use it to refer to anything, right?

I could talk about that thing or that thing or whatever.

If it has a general connotation when I use it,

what it sort of evokes might be

that me as the speaker has not chosen some more specific word to use.

So I'm either intentionally being vague or I can't be any more precise.

So that's thing, but thing is not the only thing like it.

Lots of other words have this as their sort of intentional meaning where it's

undefined. For example, we like to debate the intention of

different concepts,

like we might ask, is a hot dog a sandwich?

We take the hot dog for granted, and in this question, we're sort of asking

ourselves what goes in here for a hot dog.

Another example, these concepts end up somewhat

self-referential.

So if we talk about nobility, to be noble is to be recognized as noble

or to have some particular relationship with someone who is.

And it doesn't really matter why, like what is the constituent stuff inside

of being noble. What really matters about that concept

is everything it

implies, which is maybe a certain kind of respect.

Okay, so I think that a lot of concepts

that we like to evoke today in

these spaces like these are vague, and vague concepts are very useful.

We say things like learning or consciousness or thinking or

reasoning, and

I've never seen any sort of satisfying intentional definition of any of these.

But we use these terms, and it is useful to have terms like this.

So what I want to focus on is the term AI, artificial

intelligence. Okay. There's a lot of things

that the invocation of this concept

asks us to do. So when I repeat things

that I've heard other people say, like,

"AI consciousness is inevitable," or, "We have not yet absorbed what AI is

about to do to us," what comes to your mind or to your heart, right?

That feeling is part of the connotation, what that concept

evokes to us. So one of the really special things

that this concept in

particular does

when it's invoked is it asks us to believe that there is

some criteria that we don't know about, but there is a possible intentional

meaning that if we were all presented with it,

we would agree on it, right?

And then we could say, what is AI and what is not AI.

I don't believe we actually can have this criteria, and I'm not the only one

who believes this.

And when we have a concept that promises something to us

that it actually

fundamentally cannot deliver, and we're not aware of that as a fact,

we're

going to be led down paths

that this concept asks us to be led down,

which in this case might be something like relinquishing our agency to

something we can only believe in and don't allow ourselves to fully

understand.

Okay, so this is where it might be useful to go

to the

second type of meaning, which is reference.

And we can talk about the reference of

AI today, which is

LLMs.

Right?

This is not the only reference that AI has had.

We've had other kinds of AI in the past, obviously.

So what is a large language model?

I'm actually really happy that several of the talks already have

almost said

the same thing I'm about to now, so hopefully it'll be more reinforcing,

which

is

all LLMs are trained on text data, which was written by people on the

internet.

And when someone chooses to write something, they construct what they say,

given the concepts and other sort of structured processes

available to them.

And there's a lot of different structured processes like this.

So you might be familiar with something like syntax.

So here, different concepts and different words fall into different

categories.

So we might have nouns, we might have verbs, and we also

have some structure in what kind of combinations of these categories are more

likely to appear in the data than others.

Analogy is another example. So because of how we structure information

in our

world,

we will, even with the most rudimentary language

models that have the same

number of parameters as there are words, we can encode relationships

like between a country and its capital

in consistent ways.

So,

right, US and DC, and France and Paris.

Even with the most rudimentary models, we are encoding this.

And even something like common sense, is something that we can lean on when we

decide what to say. So even if I don't say it's explicitly due to

gravity, I'm

more likely to say something will fall than that it will fly off into the air.

So every piece of text on the internet was generated by exploiting and building

on these structured processes.

And those concepts and processes give us, that is the language model that we're

operating on. To repeat what we heard earlier,

today's neural models are compressed representations of the data

that

we train them on. Any particular neural network has a

capacity, which is the

maximum amount of information it can store, and this is defined by the number

of parameters in the model. And we can also count the amount of

information

we're trying to stuff into it through training.

And it turns out there's way more information on the internet than there is

parameters in the model. So we have to stuff a bunch of stuff into

a small

space. What would be the most economical way to

stuff that in?

It would be to represent the structures that generated that data.

So it's not surprising to me that we learn things like analogy

or syntax or

even common sense when the models that we're training are just compressed

versions of this. Those kinds of models, the space language models, are

apparently useless for most of the things that we do want to do with LLMs.

We want something that does what we want.

We do this through instruction tuning, where we're going to adjust the model

ever so slightly so that when Mia sends in something to say,

it appears that

something's coming back to us that appears like a response,

that we interpret

as a response. And we can do this very easily because

another structure that

underlies the language use on the internet is that people talk to each other.

We have people asking a question and another person responding.

So this kind of structure, which is also called an adjacency pair in the

study

of conversation, appears in the training data already.

And then the last step of this recipe is, of course, reinforcement learning

from human feedback, where we're sort of shifting the

distribution of the model

a little bit. We're having different paths in the

activation space more likely

than others. Maybe we want to put everything through

one region, which

represents the region of the helpful assistant.

And when we talk about something like agents, these artifacts appear to be

using external feedback to iteratively adapt and manipulate the systems in

which they're placed. But really crucially

is we're choosing to place those

artifacts in those systems that we have designed,

in the environments we have

designed. And we're sort of institutionalizing the

interpretation of whatever

comes out of our model in those systems. Okay, so what's an LLM?

It's a computational artifact into which we've dissolved the structure

underlying language use. And that is what a language model is intentionally.

That is sort of what makes it up, right?

I think it's uncontroversial to say that that's what a language model is.

But what I'm sweeping under the rug here is that there actually is a lot of

other connotation for AI and LLMs.

Because these models are compressed versions of text,

then it really is hard to

distinguish the things that come out of the model from things

that a person

would say. It appears to exploit our drive to read

intentionality into language

use, especially when it becomes personal.

And if I had slides, I would pull up the Wikipedia article for

deaths linked to

chatbots. People do take these things more seriously

beyond their intentional

meaning. So I say that modern generative AI technologies

are the reference of

AI, but I think I'm being imprecise, because if they were, then why are we

continuing to put so much time and energy and emotion into aspiring

towards

something that we apparently still don't have?

I'll go back to this sort of connotational meaning of AI, which is that it sort

of asks us to continually seek this intentional meaning,

whatever the criteria is, but so that we can agree on when we've sort of

achieved AI.

And I think another concept that presents itself this way

is intelligence.

And I think to those who let themselves be driven by this concept of

intelligence, this project of reaching AI

is both to discover this

unquestionable criteria and also manifest it in some artifact.

And so if you're empirically driven, and you believe in this concept, you might

develop criteria like benchmarks that allow you to say, "I don't know yet

whether we have AI, but here's something that if it

was passed, I will give it

the check."

We also might be a denialist where we're like, there's some fundamental

property of AI that can't be true.

There's this piece, I think from Gary Marcus, which is like,

probabilistic

models can never be AI. That's another example.

But the pattern that both this empirical person and the

denialist face is that they both still believe in this

concept and let

themselves be driven by it. This concept asks more of us than its

connotations.

It asks us what we ought to do or even tells us what will happen to us

once we've agreed on this criteria and once we've had people in positions of

authority agree with having reached it.

So, think back to how you feel when you hear someone say AI is imminent,

right?

So we've imagined for a while what life might be like if we

were forced to live

with intelligent machines. And so the kind of fictional characters

like the

Terminator or HAL 9000, which are really brought to life by this

concept, and

what it connotes, are really just the products

and property of the stories

about how we might lose our agency to systems that we don't allow ourselves to

fully understand. So AI is a concept that justifies itself.

But to be clear, concepts don't have agency. We do.

We justify the concept and allow it to justify itself through

stories,

prophecies, benchmarks, predictions, et cetera.

So AI is the kind of concept that really does exist

if we believe it does, and

we allow ourselves to be governed by it.

But like many other concepts like this, we have the agency to determine its

fate. So what shall we do if we wake up one day and it seems

that everyone's

decided we have AI? What is there to do next?

If we've allowed ourselves to respond to this inevitability by saying, okay,

computing is finished, humanity is finished, what comes afterwards?

I think we have the agency to say, so what?

We can choose how to respond to such a claim.

We have agency to make decisions about technology.

I think we can demand more from the technology we build, and I think we have

more agency than others. We are in the rooms where these decisions

are being

made, and we can design our technologies around

human values, for example, like

empowering other people.

And then the last thing is, remember, we have agency over language.

We have the agency to reject framing of inevitability, and powerlessness over

technology, rather than depending on proselytizing

people into believing

something that we are actually making real through prophecy.

And so remember, we have agency over language,

and with that, we have agency

over meaning, and with that, we have agency over ourselves

and the world around

us. Thank you.