[Music]

[Applause]

Thank you. Thank you. Thank you, Nicole.

And thank you for Greg for organizing

this and Matt and thank you all for

being here. It's kind of crazy to see

how much excitement there is for this

ambiguous and and broad topics we call

memory. Um, so I'm Will, uh, engineer at

Langchain. One of the honors that I get,

uh, at Langchain is to work with a lot

of our customers and users in building

successful memory and context systems.

Um, and so this set of slides is just a

few takeaways, a few lessons that I've

learned, um, or a few lessons that I've

had reinforced over the past year about

sort of what it takes in order to build

a successful memory system. Um, and if

you take nothing else from this, it's

that it's really hard to make a general

purpose one. They're best if they're

focused on your specific application and

to always look at your data and have an

experimental approach.

So, first we're going to roll it back. A

little over a year ago, we launched our

first memory server. Um, it was a server

designed to take in interactions between

a user and an agent or a chatbot, and it

would extract derived uh insights about

users that you could then later use. Um,

we launched Lang Friend, which is this

journaling app, a demo in order to show

how to use it. And we had a bunch of

enterprise design partners.

Over the subsequent months, we worked

really closely with these teams and

learned that, you know, first of all,

memory is very hard. Um, and and had a

bunch of lessons that I'm going to try

to share to you today. Um, that caused

us to sort of rethink how we wanted to

share a lot of these tooling and a lot

of these approaches for memory. The

first being that there's no really real

one-sizefits-all memory solution. Um,

and if you think about it for more than

a few seconds, this this kind of makes

sense. Um, you know, memory that we call

touches on a lot of downstream jobs to

be done or skills. Um, we need to be

remembering facts, information

relationships, this deeply

interconnected web of information that

we need um to know. And an agent also

expects to know all this information in

order to perform well. Um, but that's

not all. It needs to remember all of

that situated in a temporal context. It

needs to have more like episodic memory

um in order to remember these

experiences. And it also needs to be

understanding and able to learn, you

know, procedures and new information in

order to execute on particular tasks.

Not all of these memory types are

necessary for every application. Um this

is also non-exhaustive. Um but these are

some of the different types of ways um

that we see people wanting to be

learning new types of things in their

application. Um and another

reinforcement of this is you can see the

different types of things that people

want to be learning in real

applications. Uh so Nicole mentioned SHA

GBT u launched their updated memory

implementation recently. Um and they

really catered this around their

particular UX and use case. They have

assistant response preferences so it can

try to learn sort of your preferred way

of it learning uh of responding to you.

Uh in general it has some topics that

it's trying to model out. It's got

useful insights about the user um that

may or may not be useful depending on

the context. It has summaries of recent

conversations to sort of extend that

conversational buffer into a sort of

compressed representation. So it has

more temporal context there. And then

has a bunch of random interaction

metadata that it considers to be

relevant depending on the use case. Um

contrast that with something like an

email assistant where you really want to

be focused on a particular task. You

want to be focusing on the style and

content that you're going to be putting

into it. Whenever it's scheduling, you

really need to know what you prefer,

what you don't where you're available,

all that kind of stuff. Um, you need to

be having a deep understanding of

different sort of procedures. If someone

is emailing you and reporting a security

event in event, um, and you need to

follow all of that versus um, something

where it's just trying to meet up for

coffee.

All these things you can try to program

ahead of time, but if you want your

agent to be able to learn this, you

might not get too far if you're just

reaching right off the shelf with them.

Uh the second lesson is that you know

updates in in validation are especially

errorprone. You know it's one thing to

extract what you think is particular

interesting about a uh bit of a

conversation or a document. You know LMS

are good as summarizers. It's even

harder to then synthesize that and

connect that to all the existing

knowledge and domain that you have in

order to create a consistent world model

that improves your prediction

downstream. Um and so a lot of this ends

up being application specific. This

reinforces our lesson too. Uh we've

released a couple of tools um for this.

There's things like trust call to help

with like updates so your LMS aren't

willy-nilly deleting things. Um we've

released some like other tooling and

example around it. But at the end of the

day, a lot of this is about how you're

presenting and mapping your memory

context to the information architecture

or the needs of your application.

Um lesson three is that it's part of a

broader context system. Memory isn't

just this one isolated thing where

you're learning about the user

preferences. this is a part of you know

the situation in which the agent finds

itself be that your codebase be that the

um you know recent events the user's

gone through and other sorts of things.

Um, and when we had originally built our

memory server, we had really focused and

leaned in on to what we could

differentiate. But a lot of the people

we worked with wanted to then integrate

this with all the other existing

predictions, preferences, and other data

that they have for their model. And

while you can synthesize this all at

prediction time or retrieval time, um,

it's hard to create a sort of coherent

worldview over everything user engages

with or everything the a engages with if

you're not treating this more

holistically.

Uh lesson four and this is one thing

that we already sort of believe but we

especially believe now is that memory is

really software not necessarily

hardware. Um we often have people come

to us and talk about oh I need a graph

DB or I need a very particular

instantiation of memory for this when

really it's backwards. You need to start

from what you're trying to solve. And we

sort of made this mistake in that we had

a very particular memory server with a

particular instance. We got great

feedback on that. We found that wasn't

sufficiently flexible for people um

depending on the deployment context.

So all of these insights drove our

creation of lang SDK

and we organized that around three main

principles. One is we wanted to support

easy customization and experimentation.

All the actual extraction runs in the

code that you define. Um we have a

number of primitives to try to address

some of these memory types that we've

talked about before. Um but we really

want you to be taking ownership over the

type of information how it's updated. We

want to support flexible storage and

organization of memories. Um, so you can

sort of define what makes sense for you

and we aren't going to lock you into a

particular like vector database or

anything like that. You can swap it out.

Um, and we wanted flexible processing as

well. A lot of people talk about

different sort of, you know, batch

versus online, all these types of

different ingestion methods. Wanted to

make sure that we weren't really only

speaking to one particular thing in this

SDK.

Um so to go a little bit deeper on the

first topic of customization

experimentation we have some primitives

that we included in the library um or

the memory types that we found that

people often lean towards. So one is

this learning of knowledge and facts. We

have this background memory manager

where you can really customize in terms

of instructions the steps they're taken

how to reconcile um information and all

this is sort of orchestrated in your own

code. Um, another way that we have it

that's not shown here is we just have

some simple tools and you can be

defining it with an agent. And so as

models get better, if you want to treat

this as purely reasoning over things,

you can be putting that.

Um, we have learning instructions or

code books or however you'd like to call

that complex workflows. This is very

similar to prompt optimization. It's

typically not the whole prompt. Um, but

you have different sections of the

prompt where you're going to have

instructions that are custom to the

user. Uh and this is especially like

data dri driven where you want to be

going over batches of of conversations

uh extracting insights from them looking

at explicit user feedback and then

incorporating all that all into updates

that you can then measure.

I think if you put all of this into a

very generic graph and everything kind

of lose out on a lot of the ability to

elicit the proper responses that you

want. if you want to look at things like

tone, if you want to look at learning

new capabilities or looking at more

complex multi-step interactions that you

actually um need your agent to be

learning.

Uh and finally, there's also plenty of

episodes. This is probably the least

supported in the me in in the the

library. Um but we let you define

synthetic fshots to be extracted and

then you can incorporate it down.

The second principle we wanted to

support was about flexible storage and

organization. Um so we allow you to

store it in pretty much any back end

where you have a get input method um and

a search method. And so we have all

these integrations as well. One thing as

a tangent that we're kind of excited to

test out a little bit more is this file

system as a back end or at least

exposing a file system like API. Um and

the reason for that is a lot of these

coding or a lot of these LLMs and these

foundation model companies are really

optimizing for software engineering. And

so one hypothesis that we're testing out

um we'll get back on more results in a

bit is uh that perhaps they'll be more

lending them to um managing memories as

if it were a file system. Uh and so you

know that that's the flexibility of this

uh library allows you to be

experimenting with a lot of these things

as LLM tend to lean into different

directions over time. Um you can

organize the memories by user agent role

um organization and all those types of

things. So you're not locked into a

particular like only organizing things

around a user. You can have agents learn

things just for themselves and share it

across u and this is all sort of

orthogonal to the actual process of

information processing. Um and you can

process how and when you like. We have

some instruction around being able to

execute this on a deferred basis. So you

can be batching all of this later on if

you have a really rapidly occurring

interaction. You can have it actually

process in real time or you can delay it

so that there's dduplication of

information. Um and all this can be

managed either through a local executor

or um online with line graph platform.

So it can be scaled in a very

horizontally scalable way. Um and that

concludes my talk. I guess here's a

summary slide of all the things that I

wanted to share. Again, if you take

nothing else, it's that really no one

sizefits-all and you really want to

start with what you hope to accomplish,

what you hope your agents learn and then

work your way backwards to pick the

right solution for you. You know, test

things out, but don't just be willing to

offload all of this to one particular

solution that claims to be a holy grail.

Um, we've put Langmama SDK out there.

Encourage you to experiment on it,

encourage you to give us feedback. Um,

we're always looking to improve it. Um,

but I think, yeah, would encourage you

to take ownership of that. So, thank

you.