HomeVideos

Daloopa CEO Thomas Li: The Magic Isn’t in the Model

Now Playing

Daloopa CEO Thomas Li: The Magic Isn’t in the Model

Transcript

1775 segments

0:00

And human labor comes with a couple of

0:01

main challenges, right? The first

0:03

challenge is accuracy. You know, humans

0:04

will always make mistakes.

0:06

>> How have you shifted the way you've

0:08

deployed LLMs both in building Dupa, but

0:11

how has it changed the market

0:12

environment for for your business?

0:14

>> Yeah. So, when we think about the

0:16

problems that we can solve for our

0:18

customers, right? Because at the end of

0:19

the day, Dupa has two sides to the

0:21

business. There is the data side, the

0:23

asset side, what we call content side.

0:26

Um there is also the workflow side, what

0:28

we call product side. A product at the

0:30

end of the day means you're solving

0:32

someone's problem for them. The job of

0:34

content is to acquire data sets that can

0:37

get us deeper and deeper into our

0:40

coverage.

0:46

All right, welcome to the next episode

0:48

of Invest with AI where we explore the

0:51

intersection of investing in artificial

0:53

intelligence. I'm really excited to have

0:56

Thomas Lee from Dalupa here with us

0:58

today. Um I I know about three people

1:02

who both sat in a hedge fund investing

1:05

seat and we're talking about AI before

1:08

November 2022 and Thomas is one of them.

1:11

So, uh, really excited to have Thomas

1:14

here to talk about, uh, what he's

1:16

building at DUPA and answer our

1:19

questions about the sort of current

1:22

state of AI and investing and the future

1:24

state of AI and investing. So, thanks

1:26

for being with us, Thomas.

1:27

>> Thank you for having me.

1:29

>> Maybe um, maybe just tell us a short

1:31

origin story of of Dupa. you worked at

1:34

some really well- reggarded hedge funds

1:36

and then in 2019 made the decision to go

1:40

after the the data oligopoly of

1:42

Bloomberg Faxet and Cap IQ which sort of

1:44

seemed like a crazy uh a bit of a crazy

1:48

uh decision at the time. So just walk me

1:50

through sort of your your headsp space

1:52

back then and in founding Dalupa.

1:55

>> Yeah, so in 2019 I think [clears throat]

1:58

this is obviously pre you know CHBT cla

2:01

and so on. uh but it's not pre- AI. So

2:04

we um and in the course of my job I

2:07

started to understand a little bit more

2:09

about you know the AI space and what AI

2:11

was capable of. And I think a big part

2:13

of that was I was fortunate to cover

2:15

semiconductors and a lot of hardware

2:17

equipment companies back in the day. So

2:19

I just got closer to like what these

2:21

things were being used for. Um

2:24

[clears throat] and the hypothesis was

2:26

you know there are two basically missing

2:29

pieces uh for data when it comes to

2:32

investing right the first piece is uh

2:35

can you do it at scale and at quality so

2:38

it's really really hard to collect high

2:40

quality data because uh the only

2:42

resource you have available to do so for

2:44

the most part were just human labor um

2:47

and human labor comes with a couple of

2:49

main challenges right the first

2:50

challenge is accuracy um you know humans

2:53

will always make mistakes. Um the second

2:55

problem was latency. So if you wanted to

2:57

collect a lot of data um you you would

3:00

just be slow right you can't have you

3:03

know a 100 people collect the same

3:04

income statement and make it 100 times

3:06

faster that unfortunately just wasn't

3:08

possible. Um and then the last problem

3:10

was uh trust right how do you collect

3:13

data in a way that you can essentially

3:15

not only guarantee that your existing

3:17

data is correct but also predict that

3:19

your next outcome is going to be at the

3:21

same accuracy and for humans that's

3:23

that's near impossible to do right

3:25

asking a human to like predict their

3:27

next set of accuracies for the next task

3:29

is near impossible um but those were the

3:33

early days of like AI as we know it

3:36

today but certainly not early for

3:37

machine learning some of these new

3:39

algorithms coming out. Google just

3:40

dropped this paper, attention is all you

3:42

seek. Um, so all of that started coming

3:45

together where we basically had the

3:48

hypothesis that you could leverage AI to

3:51

change how you think about data

3:52

collection, right? Maybe not completely

3:54

removing the human in the loop process,

3:56

but rethinking that architecture. And I

3:59

would equate it to thinking about like

4:01

how you build cars, right? So today the

4:04

way we build cars as you know as is

4:07

accepted as the norm is you have

4:09

essentially a production line right 100

4:11

years ago with the model T we created

4:13

this production line there are a

4:15

combination of machines and humans

4:17

operating under machines there are

4:19

process controls operational controls

4:21

six sigma and whatnot that sit on top of

4:23

this production line and therefore we

4:26

can get high quality you know bulk um

4:30

output of cars and we take that almost

4:32

for granted said today it's like so well

4:34

understood we take that for granted but

4:36

the way data is being collected and most

4:38

investment institutions um is actually

4:41

like you know the coach building

4:43

business it's all done by hand there's a

4:45

taste element to it um a good analyst

4:48

will be able to collect better and more

4:49

data than a bad one which we looked at

4:53

and we said this requires a production

4:55

line you can't build a production line

4:57

that's fully autonomous but a production

5:00

line is better than handcuffed coach

5:03

And that was the foundation of what

5:05

created Dupa, right? Using AI to build a

5:08

business at its very core and rethinking

5:11

like how you extract data, how you

5:13

quality control data. And obviously with

5:16

the birth of AI as we know it from the

5:19

consumer side, um it was very natural

5:22

for us to say we are an AI first

5:23

company. We understand how everything

5:25

works. Let's work with the labs. let's

5:28

put out data into an MCP and let's

5:30

distribute the data because not only

5:32

should data be collected using um

5:35

operational controls and AI and and

5:37

algorithms, you should also deliver it

5:40

in the same mechanism to maximize the

5:43

output for your customers.

5:45

>> That's great. Thank you. Thank you for

5:46

that. And and and what what have been

5:48

the more recent impacts on your

5:52

business? My sense is that sort of the

5:54

rise of the agentic workspace has been a

5:56

nice tailwind for for dupa. But how have

5:58

you shifted the way you've deployed LLMs

6:02

both in building DUPA but how has it

6:04

changed the market environment for for

6:06

your business?

6:07

>> Yeah. So when we think about the

6:09

problems that we can solve for our

6:11

customers, right? Because at the end of

6:12

the day, Dupa has two sides to the

6:14

business. There is the data side, the

6:16

asset side, what we call content side.

6:19

Um there is also the workflow side what

6:21

we call product side and if you think

6:23

about what product means right product

6:25

at the end of the day means you're

6:27

solving someone's problem for them right

6:29

if you don't have a problem then you

6:31

cannot have a product by almost

6:33

definition right you just have a trinket

6:35

or a bell whistle so when we used to

6:38

think about how to solve a problem for

6:40

our customers we had to go build a whole

6:42

you know we had to build our Excel addin

6:44

we had to build an environment we had to

6:46

do all of these things with a very clear

6:48

problem in mind. And if you look at our

6:51

content, right, we've collected all of

6:52

these data over time. On average, only

6:55

about 5% of all of our data was actually

6:57

being used by by all of our customers

6:59

every quarter. So 95% of the data that

7:02

we're collecting um basically never sees

7:04

the light of day. But because we collect

7:06

everything companies um disclose, we we

7:09

don't know which 5% is going to get

7:11

used. We just know 5% gets used um any

7:13

given quarter. What AI flips is because

7:16

an agent is able to take in so much more

7:19

information than humans can. Um it

7:22

changes how much data is being consumed

7:25

by our database and therefore it changes

7:28

the problem sets that we can go after.

7:31

So in you know early days of dupa the

7:34

problems that we went after was earnings

7:35

updates right we help people update

7:37

their models in their format. You click

7:39

a button you update your model. Your

7:41

model might have a lot of data, but it

7:43

doesn't have every single data point the

7:45

company has disclosed. So after you've

7:48

updated your model with us, a lot of the

7:50

data actually just doesn't get touched

7:52

by you. But when you use our data in an

7:55

MCP and you're running a research query

7:57

with the MCP. So now you're out of the

7:59

modeling um workflow. You're in the

8:02

research workflow. Now research requires

8:04

a lot more data because your model might

8:07

not have let's say inventory breakdowns

8:09

but in your research process that might

8:11

be something you're doing right you're

8:12

tracking gross margins visav you know

8:14

work in process inventory. Um so that

8:17

becomes something that gets used. Uh so

8:19

what we saw is essentially now almost

8:22

our entire database is getting used like

8:24

every single quarter because there's so

8:26

many agents from our customers that

8:27

deployed on top of it. Um and we saw a

8:30

remarkable shift in the volume of the

8:33

documents we provide, the data we

8:35

provide. Um I think year on year we've

8:37

seen data consumption grow over a 100

8:40

times.

8:40

>> So that 5% has that changed?

8:44

>> Yeah, it's almost 100% now.

8:46

>> Wow.

8:47

>> Yeah.

8:49

Whether or not so an agent would consume

8:51

all the information, right? But after

8:53

the agent consumes the information,

8:54

whether or not they they surface all the

8:57

information to their end user, um I

9:00

don't know, right? Because that's their

9:01

internal system. Um chances are they're

9:04

not, right? But the agent is processing

9:06

all of that information. It's almost

9:08

like if your analyst went to read the

9:10

10Q cover to cover, they might not give

9:13

give you everything in the 10 Q because

9:15

a lot of it is irrelevant, but they

9:17

would have read it cover to cover and

9:18

they'll give you the top three most

9:20

important things.

9:22

And naively is that was that growth from

9:25

kind of the easier agentic services like

9:27

co-work cloud codeex

9:30

or did something else happen?

9:33

>> I think the most the the biggest change

9:36

and unlock for firms is going from using

9:40

chat as just a chatbot to connecting

9:42

MCPS with it. Um so that's like the

9:45

first lift. Um it's a it's a simple lift

9:48

which is which is sort of why the

9:49

adoption rates have been so high and we

9:52

just saw explosions there. Uh when we

9:54

dig into how users use us um you also

9:58

have these like very spiky users right

10:00

we have users using you know thousands

10:03

of times more data than our average user

10:05

is and that almost always is because

10:08

they're in a code engagement. Can you

10:09

can you frame Thomas a little bit of

10:12

just the path we've been on of

10:14

abstraction into the MCP?

10:17

Um and even 18 months ago, you know that

10:20

people were sort of building data lakes

10:22

and just like a much more complic O

10:24

talking about OCR and PDFs. It was much

10:27

more complicated

10:28

situation to get fundamental data into

10:31

into a chatbot. Can you just give us a

10:34

little bit of the highlights of what's

10:35

happened and and your sense of where

10:37

we're we're at right now because even

10:40

six months ago people were telling me

10:41

like MCP is still too brittle for some

10:44

of these use cases which seems to be

10:46

have become a solved problem. So just

10:48

give us that context if you would.

10:50

>> Yeah. So not all MCPS are created equal

10:53

because if you if you really think about

10:55

what is an MCP, right? An MCP is a set

10:58

of prompts like English prompts that sit

11:01

in front of your API. So really it's no

11:05

different like if you wanted to create

11:06

an MCP it's no different than writing a

11:09

very long prompt into claude or chatpt

11:12

and then ending it with your API

11:15

endpoint your URL right so now what is

11:18

this prompt the prompt in short is as a

11:20

set of instructions on how to use your

11:22

API so in human world we actually have

11:25

that we've had that for a long time it's

11:27

called documentation and API so when you

11:29

have a software engineer use the API

11:31

they always ask what where's the

11:32

document doumentation because just

11:34

giving me a URL for the API like is

11:37

meaningless if I don't know how to use

11:39

it right I don't know what's behind it

11:40

what are best practices what are limits

11:43

and all of that so the MCP prompt is

11:46

basically the documentations docu um you

11:50

know portal summarized into a text file

11:53

but ultimately what makes an MCP good or

11:56

bad or brittle comes down to what is in

11:58

the API right MCP is just an API that's

12:01

being used by a large language model

12:03

versus a human. And how you set up the

12:06

API still is the same as what it is

12:10

before. First of all, you need to have

12:11

the right data in it, right? I anyone

12:13

can create an API endpoint with nothing

12:15

behind it. It's a functionally useless

12:17

API, but you could create one. Um, two

12:20

is once you put the data in it, how do

12:22

you structure it, right? All the best

12:24

practices that exist for humans to use

12:26

an API is identical here, right? Are you

12:30

creating a pathway and an index to help

12:33

search through the API? Right? If an

12:35

engineer looks at API and they say,

12:36

"Hey, there's 15 different API like

12:38

endpoints within this first endpoint,

12:40

like how do I search through them? How

12:42

do I know what the syntax is? What a

12:45

best practice is like all of that still

12:47

needs to exist, right? Um, do does your

12:50

MCP have or the API in your MCP have

12:53

access to different tables? How do the

12:55

tables work together?" So, I give you an

12:57

example. In our MCP there are multiple

12:59

different tables right one table is just

13:01

numbers one table are documents so when

13:05

you hit our MCP and you ask a question

13:07

like summarize you know Amazon's

13:09

earnings right a few things needs to

13:11

happen the first thing is it needs to go

13:13

through all of the data and essentially

13:16

say okay what has happened to earnings

13:17

what are the historical numbers like and

13:20

so on and so forth but it also needs to

13:22

corroborate the numbers with the actual

13:24

transcript with the Q with the AK K with

13:27

the investors act right pull all that

13:29

information out to create a holistic

13:31

view. It also needs to gets access to

13:33

stock pricing right and look at how

13:35

Amazon stock price has change visav's

13:38

benchmark or other peer. So it needs to

13:40

pull in a bunch of stock pricing. So to

13:43

make your MCP really good, you need to

13:45

not only have multiple data sources in

13:47

there, um, but also the data sources

13:49

needs to be connected internally within

13:52

your API in a way where if the if the

13:55

large language model is pulling Amazon's

13:57

gross margins, it knows where it's

13:59

coming from. It knows the notes that

14:01

relevant to gross margin and it knows

14:02

based on these notes, what are the

14:04

historical notes that I should look at

14:06

um because that might be relevant to

14:09

helping me understand gross margins

14:11

today. So connecting all of those

14:13

tables, setting up an index, making sure

14:16

that you know the latencies are low,

14:18

that's what makes an API good. And if

14:21

you have a good API for humans, you

14:22

almost always have a good API for MCPS.

14:25

>> And where do you think is sort of

14:27

understanding that there it's a probably

14:29

heterogeneous answer? Where do you think

14:32

the MCP structure is today in terms of

14:35

reli reliability? Has that improved as

14:37

the the foundation models have improved?

14:40

agentic search etc. And sort of where

14:43

where where are we at in terms of you

14:46

know trusting that data to not

14:47

hallucinate or you know sort of break

14:49

break retrieval and MCP structure.

14:52

>> So I I think it has definitely improved

14:54

on the model side. Um but I think the

14:57

bulk of the improvement that we are

14:58

seeing with MCPS today is actually not

15:01

because of the models or the labs but

15:04

actually because of how data providers

15:06

are creating their MCP. I think people

15:08

are starting to get comfortable around

15:11

you know an MCP is not like a business

15:13

catchall right like someone senior says

15:15

we got to build an MCP we build an MCP

15:18

at the at the end of the day is still a

15:20

product it's a product that needs to be

15:21

maintained you need to talk to your

15:22

customers you need to understand what

15:24

problems you're actually solving for

15:26

what are the most likely used prompts

15:28

why they're using something and how do

15:30

you build your MCP to help that um what

15:33

other people have figured out is you

15:35

know one of the fastest ways you can

15:37

help people adopt your MCP is to teach

15:40

them what are the best use cases. So can

15:42

you create skills such that when your

15:45

MCP is invoked your customers can also

15:47

pull those skills to to invoke the MCP

15:50

right and your skills will have a clear

15:52

guide to how to use the MCP. So a lot of

15:55

it is just like how you know when you

15:57

deploy an API you would sit down with

15:59

your customer and say hey let me teach

16:01

you how to use the API here are some

16:03

case studies of how people have been

16:05

successful using that API. It's the same

16:07

thing here just instead of case studies

16:09

you're using skills or agents right so I

16:13

think there there is a lot of groaned

16:16

understanding amongst the vendor

16:18

community um about that where we see

16:20

roadblocks happening comes primarily

16:23

down to um this cannibalization debate

16:27

right because you could argue that if

16:29

all of my data is in the MCP so first of

16:32

all the question is does all my data go

16:33

into the MCP right but hypothetically

16:36

let's say you're you're a content

16:38

company or data company, you say all of

16:39

my data goes to EMCP and it goes to my

16:41

customers. Then if you have any other

16:45

products that are reliant on your data,

16:48

then the question is, is it still

16:49

relevant? Right? Like a a simple example

16:52

is let's say on the DUP website, I have

16:55

a summary table for a summary of

16:57

financials for every company and

16:59

customers would go to that page and just

17:00

look at summary financials. um if I put

17:03

financials in my MCP then theoretically

17:06

you don't have to go to my website to

17:07

look at summary financials because if

17:09

you prompted claude you will get the

17:10

same thing moreover if I gave you that

17:13

skill I'm making it even easier for you

17:16

to go and view my data in claude and not

17:18

on my website so I think for

17:20

organizations where they look at that

17:23

setup as a loss of adoption then it's

17:27

rightly so you get concerned right I

17:29

have a different view my view is so long

17:31

as I am the reason I can solve a

17:33

customer's problem gets solved. I don't

17:35

really care if you're solving it in my

17:37

system or, you know, in someone else's

17:40

system so long as I I am solving your

17:43

problem. Um, so I'm willing to put every

17:46

single asset I have into the MCP. But

17:49

you can see how at different

17:50

organizations you end up at different at

17:52

different conclusions.

17:54

I've I've noticed that I'm a Faxet

17:56

subscriber and I've found that my the

17:58

time I spent in the faxet terminal is

18:00

down materially which is sort of nice

18:02

because the UI is like not great. U but

18:05

I'm accessing a lot of that data via an

18:07

agentic workspace. Um, how do you think

18:11

about like one of the things I'm

18:12

struggling with is and I get hear you

18:15

sort of get this question from my

18:16

students and clients is like what's the

18:18

right MCP stack

18:20

within an agentic workspace because dupa

18:23

was early as a as a connector into many

18:26

of these workspaces but now I could get

18:28

fundamental data from six or seven

18:30

different players in the same agentic

18:33

workspace. How do you think about that

18:35

competitive environment where more

18:37

fundamental data pipes, KPI pipes are

18:40

coming in in into the agent? How you

18:43

would counsel an investor to decide on

18:45

that stack and how you think about DUPA

18:48

as being differentiated within that

18:49

stack?

18:50

>> Yeah. So, I I love the way you frame it,

18:53

which is like pipelines, right? So, yes,

18:55

we were the first to build this

18:56

pipeline. Um but you know anyone can

18:59

build a pipeline and ultimately if

19:00

you're an investor you should have you

19:02

know hund hundreds of these pipelines

19:03

coming into your firm. Um but just like

19:05

a pipeline if let's say every pipeline

19:07

is carrying water right um what you

19:10

really want is to get the pipeline with

19:12

the highest quality water. Usually the

19:14

highest quality water tends to also come

19:17

at a premium. Um and you might say hey I

19:20

am a science lab and this water needs to

19:23

be so pure it doesn't even conduct

19:24

electricity right pure water. Or you

19:27

could be a household where you're like

19:28

that doesn't really matter. It just

19:30

needs to be portable. I just need to be

19:31

able to drink it. Or you could be a

19:33

factory where it's like it doesn't even

19:34

need to be like portable. I just needed

19:36

to cool a machinery. Who cares? So

19:38

depending on your needs, you probably

19:41

have different types. You could connect

19:43

to all of those pipes, but you should

19:44

only turn on the pipe that solves your

19:47

problem. So I don't pretend that the

19:49

Lupus data and content can solve

19:51

everybody's problem at at their

19:53

perceived price to value, right? What we

19:55

do is we go for the extreme quality, the

19:59

low latency, the high coverage, right?

20:01

No fundamental data set out there has

20:03

deeper data than we do, has more

20:05

accurate. So, we are the most accurate.

20:07

We have the lowest latency and we are

20:10

probably by far the most uh the most

20:13

comprehensive, right? We cover data

20:15

points that other firms just simply

20:17

don't cover. We have, you know, guidance

20:18

and adjustments and KPIs and guarantees

20:21

and enterprise SLAs's. Um, so we are

20:24

sort of like the pure water that goes

20:25

into a lab, right? It's ent truly

20:28

enterprisegrade targeting some of the

20:31

largest institutional investors and

20:33

today serving some of the largest

20:35

institutional investors, right? But I

20:37

will argue, you know, if you're a

20:39

smaller firm and the quality doesn't

20:41

matter because your investment style and

20:44

uh process just doesn't require like

20:46

extreme high quality like this is

20:49

company dropped earnings, I need to know

20:50

everything right away. um then we might

20:53

not be at the right price point because

20:54

your perceived value of our data will be

20:56

lower um and there are as you pointed

21:00

out a lot of other sources.

21:02

>> Yeah. How do you think about I sort of

21:05

you know debate with myself both sides

21:07

of this sort of like best of breed stack

21:10

within an MCP pathway of like let me go

21:12

pull in the six highest quality stack

21:15

sort of pieces of the stack which makes

21:17

sense. I sort of agree with you

21:19

certainly for the tier one client

21:21

although the MCP pricing for that stack

21:23

now can be like an aggregate like 60 70k

21:26

like the numbers get pretty high when

21:28

you start to add up the MCP price versus

21:30

a number of vendors um chasing this one

21:34

pipe strategy where they're trying to be

21:36

very good but with one MCP price and

21:39

there's some creative licensing sort of

21:41

you know um vendor licensing strategies.

21:44

How do you think about where does Dupa

21:45

want to play in that? Uh and how do you

21:49

think about the the juxtaposition of

21:51

those two MCP strategies?

21:53

>> Yeah. So um because the beauty of a

21:56

large language model is that whether you

21:58

have one pipe or three pipes is

21:59

fundamentally the same to the large

22:01

language model, right? Because if let's

22:03

say I'm connected to three MCPS, which

22:05

basically means the model is connected

22:06

to three APIs and if behind every API is

22:09

three other APIs, the model is now

22:11

connected to nine endpoints, right?

22:13

versus if I had nine MCPS with only one

22:15

endpoint each. The model at the very

22:18

core is actually connected to the same

22:19

amount of resources. So it doesn't

22:21

really matter from a buyer's

22:23

perspective. Yeah, you have to deal with

22:25

more vendors and contracts and like

22:27

people. Now, uh but at least from a

22:29

product angle, you you should be getting

22:32

the same outcome. Now, from a vendor

22:34

perspective, obviously if you have more

22:37

content, you could make the argument I

22:38

can sell to more people and therefore I

22:40

can build a bigger business. Um, but I

22:42

think that the core to whether or not

22:44

you want data into your pipe or not is

22:48

whether there are synergies between the

22:50

data assets. So one example is that the

22:53

clearest example is data and documents.

22:56

Right? If all I had was numbers but I

22:58

don't have the documents then you are

23:00

losing the context for the numbers

23:01

because the number 27 in and of itself

23:04

means nothing until I can describe what

23:06

it is. I can describe the history. I can

23:08

describe the context and how the context

23:10

has shifted. So having a pipe of just

23:13

documents and having a pipe of just data

23:15

will not get you the same output as

23:16

having one pipe where they are

23:18

interconnected and woven together where

23:20

texonomy has been built. Um but said

23:23

differently, let's use like stock price

23:25

as an example, right? If I have a pipe

23:28

of just end of day stock price and I

23:30

have another pipe of just um fundamental

23:32

data, like that map is relatively simple

23:35

because end of day stock price doesn't

23:37

have a lot of like you don't have a

23:38

security master issue with end of day

23:40

stock price uh for let's say the New

23:42

York Stock Exchange tickers. So then it

23:44

works well with a large language model.

23:46

However, if you say hey end of day stock

23:48

price is not enough. I need tick by

23:50

tick, you know, L2, L3 stock price data.

23:53

Now all of a sudden you have a security

23:55

master issue, right? So now you need all

23:57

of your security pricings to get mapped.

23:59

If you throw in like option prices into

24:01

that, now you have a real security

24:03

master issue. So trying to get the LLM

24:06

to say, hey, I'm going to buy one L1

24:08

panel, one L2 panel, one L3 panel. I'm

24:10

going to buy a fundamental data panel

24:12

and magically assume that the LLM can do

24:14

the mapping in real time or in runtime.

24:17

like that's not that's not that's just

24:19

not possible, right? You will end up

24:20

with a lot of hallucination. So, there

24:23

are data sets that require the vendor to

24:25

build texonomies and mapping beforehand

24:28

and there are data sets that don't.

24:31

>> Yeah. And how do you think how do you

24:33

think about the moat of your of your

24:35

data set? I mean, I sort of would seem

24:37

that armed with agents, it would be

24:40

easier for a competitor to recreate what

24:42

you've built, but what what are the sort

24:44

of defensible elements of the the moat

24:46

of your high quality data? [snorts]

24:49

>> Yeah. So, the the mode ironically itself

24:51

is high quality, right? Because um being

24:54

able to collect data and being able to

24:56

collect data at like a 99.9% accuracy

24:59

are wholly different problems. Um I'm

25:02

going to use the car car factory

25:03

example. So when you build a car and

25:07

Toyota is famous for like you have six

25:09

sigma right when they when they build a

25:11

car the car has the same quality as

25:13

every other car down to you know six

25:15

decimal points of nine. Um that is

25:19

really really hard to do and even if

25:22

someone else has access to all the

25:24

engineers to build the factory like you

25:26

cannot replicate the quality of the

25:29

Toyota factory. Now why is that? It's

25:32

because it's it's never about just a

25:34

tool. Like giving someone a really good

25:36

hammer doesn't make him a good

25:37

carpenter. It's everything else. It's

25:40

how do you set up um your process? How

25:42

do you set up your controls? How do you

25:44

set up internal SLAs's down to how is

25:46

your company culture? How do you

25:48

incentivize and reward people? How do

25:50

you hire? Right? Every single thing that

25:53

we care about um has an angle towards

25:56

accuracy. Right? Okay, within Zupa, we

25:59

even talk about what are the personality

26:01

traits that lend to higher quality

26:04

output versus what are the personality

26:06

traits that lend to faster output and

26:08

are they the same personality traits,

26:10

right? So, we care about like

26:12

microscopic little decision making. Uh

26:15

we care about how we get incentives

26:18

aligned at the 15 minute mark of

26:20

earnings, the 30 minute mark, the 45,

26:22

the one hour mark. and how do you think

26:24

those incentives translate into actual

26:26

human outcomes and the engineers

26:28

outcomes. So you can't just go into just

26:31

like how you can't go into cloud and be

26:32

like collect all the data for me. Um

26:34

it's about how you set up your entire

26:37

factory with like very clear goals in

26:39

mind. And for us we have three things we

26:41

care about. We care about completeness,

26:42

we care about accuracy, and we care

26:44

about speed, right? And that's like

26:46

frankly the only three things we care

26:48

about. Um and you can't just go to

26:50

Claude and say and maybe you could in

26:51

five years. Who knows, right? But you

26:53

can't go to cloud and say I want perfect

26:54

data extracted from every document in a

26:56

database in real time. Go.

26:59

>> And if you look at um those those three

27:02

parameters assuming you're hitting them,

27:04

you're you're delivering at the quality

27:05

that you want and you fast forward is

27:08

the is the company's strategy to expand

27:11

into more data sets or to move more into

27:14

the application layer or is there like

27:16

what's the where do you go from here?

27:20

>> Yeah. So we we always think of a company

27:21

as made up of two two pieces. The

27:24

workflow piece and the content piece. Um

27:26

so they have this the central goal is to

27:30

solve problems for our customers, right?

27:32

So if there is a problem that requires

27:34

more data, then we will collect more

27:36

data. If there's a problem that requires

27:37

us to get better aligned with workflows,

27:39

we will do that. uh but high level the

27:41

the job of work the sorry the job of

27:44

content is to acquire data sets that can

27:47

get us deeper and deeper into our

27:49

coverage right so we cover 6,000

27:52

companies today we we constantly expand

27:54

our coverage that's just a motion that

27:56

we have um but it's also about can we

27:58

get deeper right so um let's use like uh

28:02

commodity companies as an example right

28:04

commodity companies disclose a lot of

28:06

data just stand alone um but they

28:08

actually have regulators that are not

28:11

security regulators. So, think about

28:13

like the the environmental agencies

28:15

around the world. Um, and for particular

28:18

commodities that might even be their own

28:20

like regulatory agencies and there are

28:22

filings, right? A lot of these mines

28:24

will tell you the reserve number where

28:26

their minds are at. These they file

28:28

landscaping uh with all these regulatory

28:30

agencies. These are all information that

28:34

can help you make better and more

28:36

informed decisions. And the content is

28:38

really hard to assemble, right? Some of

28:40

these guys, you literally need to call

28:42

the regulator for them to like FedEx you

28:45

the documents. Some of these guys you

28:46

need to develop a relationship. Um, and

28:49

so the content team's job is to make

28:51

sure that we get deeper and deeper. The

28:53

workflow team um the way we the way we

28:55

think about it is we call our work

28:58

basically the 13week problem work where

29:00

every quarter has 13 weeks and if I

29:02

looked at week four of every quarter,

29:04

they have a tendency to look the same.

29:06

If I look at week one, they all have a

29:07

tendency to look the same. So the

29:09

question is like what is the most

29:10

painful week? And of the most painful

29:13

week, what is the most painful problem?

29:15

And is that problem a problem we can

29:17

tackle with a workflow solution? So for

29:20

example, our first product is updating,

29:22

right? We basically said the most

29:23

painful week in a hedge fund analyst uh

29:26

quarter is the earnings week and the

29:28

most painful minute of earnings week is

29:30

literally the minute when earnings drops

29:32

because you have to update your model.

29:34

So what is the ideal workflow solution

29:36

here is to create a button where if I

29:37

hit the button I update my model. So our

29:39

workflow team's job is okay identify

29:42

this unique problem and what is the

29:44

ideal workflow outcome which is to hit

29:46

this button right so given that we have

29:48

now developed that what is the next set

29:50

of problems and what is the ideal

29:52

workflow outcome for that and usually

29:55

our framework is the best workflow is

29:58

always an invisible one so we don't want

30:00

to build into like some big UI that you

30:03

have to like tinker with and there's

30:04

like knobs to turn right we want to

30:06

build an invisible experience or an

30:08

experience experience where you

30:09

basically hit a button and it works. Um,

30:12

but the magic comes in identification of

30:14

the problem and creating a system that

30:16

is so good that it feels invisible.

30:19

>> And on that identification problem, are

30:22

you using a forward deployed model or

30:25

are your product managers is that that's

30:28

being done kind of uh in-house?

30:31

>> Yeah, so we don't do forward deploy. Um

30:34

we and maybe we will in the future, but

30:37

right now the way we like to think about

30:39

it is you need to be so integrated into

30:42

your customer that you shouldn't have to

30:45

have a forward deploy engineer, right?

30:47

We should be able to just sit down, get

30:49

a coffee with any of our customers at

30:51

any time and just like really really

30:53

dig. If you have the philosophy and the

30:56

culture of a business that is so deeply

30:58

ingrained with the problems of your

31:00

customers, you shouldn't need a proxy to

31:02

tell you what the problems are, right?

31:04

You should almost be at one with how

31:06

your customers operate,

31:09

right? So, if I went to my team today

31:10

and I asked them, hey, it's earning

31:12

season, like what's going on? I should

31:14

be getting a response be like, hey, this

31:16

is what's happening. You know, this is

31:17

the peak. Here's what we expect at 4

31:20

p.m. here's what B. Here's what miss.

31:21

Those are the high level things that we

31:23

should be able to experience internally

31:26

because we need to live like our

31:28

customers too.

31:30

>> Thomas, how are you seeing funds

31:32

integrate other sources of data, their

31:34

internal data or their alternative data

31:37

with with with the loop and how has that

31:39

changed as as more top funds have have

31:43

gone down the agent path?

31:46

>> So, they don't quote unquote integrate

31:49

into dupa. What we see a lot of is they

31:52

basically start to create these

31:53

orchestration layer internally within

31:56

the fund. Um folks have taken different

31:58

approaches. Some are building knowledge

31:59

graphs to solve that problem. So

32:01

everything feeds into the knowledge

32:02

graph. The knowledge graph feeds into um

32:05

the the large language model layer where

32:07

the agents sit. Some folks are basically

32:09

not using that. There are other ways you

32:11

can solve it. You can connect a bunch of

32:13

MCPS. You can create skills that help

32:15

you orchestrate the MCPS. So some folks

32:18

are doing that. Um the key benefit of

32:20

doing it without the knowledge graph is

32:22

access control because if let's say team

32:26

A does not have access to a certain

32:27

asset and team B does then you can't

32:30

really go to create two different

32:32

knowledge graphs. You can't create a

32:33

knowledge graph for every single person

32:35

based on their based on their you know

32:38

uh their licenses. So um it's hard to do

32:42

that when you are really really scaled

32:44

out and you're trying to build

32:45

infrastructure for everyone. Um but you

32:47

can say okay instead of building a

32:49

knowledge graph which captures context

32:51

can I use um a bunch of orchestration

32:54

skills to do that and then connect all

32:56

the MCPS. I think the most forwardinking

32:58

bite firms they also tend to be larger

33:01

um we'll work with vendors and walk them

33:04

through where the MCPs are good and not

33:07

good and where they need to work on. I

33:09

think it's easy for a vendor to look at

33:11

their MCP and say here here's the

33:13

customer's problem but your enduser's

33:15

problem and the enterprises problem

33:17

might be wholly different problems right

33:19

so like making sure that you can solve

33:21

both problems in your MCP um also really

33:25

matter can can you unpack the the

33:27

knowledge graph a little bit more some

33:29

logistics of that what does that look

33:31

like sort of in in in the specific sort

33:34

of build structure of a knowledge graph

33:37

>> so a knowledge graph is nothing but

33:38

folders and and skills, right? If you

33:41

think about like the Obsidian wiki or

33:42

whatever, um it's basically sorry,

33:45

folders and markdown files. Uh the magic

33:47

of a knowledge graph is to have all your

33:49

markdown files connect to each other and

33:51

creating a system to generate markdown

33:53

files um automatically and basically

33:56

like delete quote unquote delete

33:58

markdown files that are no longer

34:00

relevant. And so what you're trying to

34:01

mimic is basically the human brain,

34:03

right? you're trying to recreate um a

34:05

set of memory and context and analysis.

34:08

So what some firms will do is they'll

34:10

say, "Hey, we have 10 MCPS, we have 10

34:13

skills, right? If I ran 10 skills on

34:15

every ticker through every MCP, I can

34:18

create, I don't know, 10,000 markdown

34:20

files, right? If I can connect all the

34:22

markdown files together um and instead

34:25

of having the MCPS directly connect to

34:27

my large model, I connect my endpoint to

34:30

my folders with my markdown files or the

34:32

knowledge graph uh to the large language

34:35

model. So I think a lot of firms are

34:36

doing that. Um there's a tendency for

34:39

those firms to be more of a one team one

34:40

dream type setup. Um and the key benefit

34:44

of that is uh you basically have the

34:48

analysis and the context pre-done. So

34:51

the AI doesn't have to do analysis. So

34:54

let's say Amazon drops earnings. Instead

34:57

of a user prompting the system to

34:59

analyze earnings, it autoan analyzes

35:02

earnings, creates the markdown file,

35:04

connects to last quarter, Amazon

35:06

earnings, connects to Costco earnings,

35:08

connects to Nvidia earnings, connects to

35:10

all these companies as relevant to

35:12

Amazon's earnings. So when I come into

35:14

cloud and I say, "Hey, Amazon just

35:16

reported earnings. What are the macro

35:18

trends I need to take away from Amazon

35:21

and their peers and how does that

35:22

influence how I should think about next

35:24

week's earnings?" A lot of that work has

35:27

already been pre-done. So Claude's job

35:29

is really just to walk through your

35:30

knowledge graph and figure out where all

35:32

the pieces lie and assemble it together

35:34

as opposed to generating that output on

35:36

the fly which significantly improves the

35:39

quality of the output and drops your

35:41

token your token cost. step stepping

35:44

away from from data and dalupa for a

35:47

minute. Um we thought we'd fire a few

35:49

just broader AI questions at you and one

35:52

of the things that it feels like the

35:54

debate is picking up again um and in a

35:57

few of my evals some of the models have

36:00

gotten surprisingly good on the open

36:02

source like Chinese open source models.

36:04

I'm curious sort of your perspective

36:07

um on the sort of frontier models versus

36:10

open source models how you see that

36:12

debate uh evolving.

36:15

So this is a personal view. This not

36:18

dupa view. I'm a big believer in open

36:20

source. Um I mean if you look at aloopa

36:22

github we've open source every time we

36:24

create a skill or an agent we open

36:26

source it. So I'm a big believer in

36:27

that. Um my view is it's very very hard

36:32

to build something that is technical um

36:36

faster than an open-source creator can

36:39

especially when the literal contribution

36:42

um world today is everybody because

36:45

everybody can now code so everybody can

36:47

contribute to open source so all it

36:49

takes is will and will and and passion I

36:53

guess and there are just more people

36:56

who's not working in anthropic and is

36:58

working in anthropic right so it it

37:02

becomes really hard to combat open

37:04

source I think um and but in this case

37:06

we're specifically talking like open

37:08

weight models right like the K3s of the

37:10

world um I think it's really hard I

37:12

think at the bleeding edge when you

37:16

think about how you actually make a

37:18

model better there are a couple of

37:19

different elements to it right one

37:21

element and this is like way

37:23

oversimplifying it but one element is

37:26

increase parameter size, right? If if

37:28

you can build a 20 trillion parameter

37:30

model, chances are it's going to be

37:32

better than a one trillion parameter

37:33

model, right? Because you just have more

37:35

weights to play with. Um, but where you

37:38

get a lot of efficiency actually isn't

37:40

in the base model, but what you do in

37:42

post- training, right? So, if I took a

37:45

small parameter model and I really

37:48

postrain it to a very specific task,

37:50

let's say this task is building a DCF,

37:53

right? If I had $10 million to have a

37:57

model build a DCF, instead of spending

37:59

it increasing the parameter count of the

38:02

base model, I just take a small base

38:04

model and I just post train the the heck

38:07

out of the DCF piece. I f I hire a bunch

38:10

of people, build a thousand DCFs, feed

38:13

it in, do it again and again and again.

38:15

Um, you end up with a model that can

38:17

build a DCF better uh than the increased

38:21

parameter size. Obviously it taps out

38:23

like if you don't have a model with big

38:24

enough parameter size like you end up

38:26

going to these like really really deep

38:29

um targeted you know use cases but if

38:32

you think about the use cases of all the

38:35

different things you do within AI in any

38:38

industry it's actually somewhat

38:40

quantifiable of of course there's a long

38:42

tail but you know when you use AI in

38:46

Excel right there's really only like

38:48

five to 10 different things you're

38:50

trying to do with when you use it in a

38:52

hedge fund, in a chatbot, there's really

38:54

only five to 10 different things. So

38:56

there is a case to be made where the

38:58

labs can work closely because they're

39:01

are enterprises can work closely with

39:03

their customers on the post training

39:05

piece to get the output really precise

39:08

to what they need to do um in a way that

39:12

the open-source guys can't because it

39:15

would be very hard for a hedge fund to

39:18

speak to an open source model and say

39:20

I'm going to give you how I think about

39:22

the world so that you can train your

39:24

open source better, right? So, that's a

39:26

disadvantage of open source, which is

39:29

industry seekers tend to not get in. So,

39:31

post- trainining becomes difficult, but

39:33

their advantage is on the pre-training

39:35

side. Um, they have a much lower lift.

39:39

I' I've heard the I've heard the

39:40

opposite argued a bit with some of the

39:43

like local host locally hosted

39:44

post-trained Quen models where that sort

39:47

of that post- training actually lives

39:49

within the walls of the hedge fund where

39:51

there's actually a little bit of growing

39:53

skepticism about working with the

39:55

foundation labs as a large asset manager

39:57

because you're sort of leaking your

39:59

leaking your proprietary knowledge back

40:01

into the back into the foundation lab.

40:04

How would you sort of rebut that? you

40:06

sort of seem to have been observing the

40:07

opposite.

40:08

>> Yeah, I think post- training is more art

40:10

than science. I think yeah, in theory

40:12

you can go into GitHub and just download

40:14

like a post- training module and just

40:16

put all your data in there and you know

40:19

local host Quen or K3 or whatever and

40:22

run post training. I think the reality

40:24

is like when you think about what post

40:26

training really is, it comes down to a

40:28

very very simple problem. You have to

40:31

know your objective function, right? So

40:34

when when we say train a model for a

40:37

specific problem, what we are assuming

40:39

is that we can very in a very

40:42

quantifiable way write down this

40:45

problem. Right? And a lot of business

40:48

problems cannot do that. So if you go to

40:50

an analyst and you say, I'm going to

40:52

train a model that can do what you do. I

40:54

need you to write down in a few

40:56

sentences for me a measuring stick on

40:59

how I know the output is good or worse.

41:02

And the analyst will be like, I I I

41:03

don't know. Like, yes. And the final

41:05

outcome is P&L, right? Like we all know

41:08

that. But when we start to break down

41:09

what creates P&L, like what makes a

41:12

pitch better or worse? Like is a 10-page

41:15

pitch better than a onepage pitch? Like

41:17

probably not. So that's not a good

41:19

measure. Is detail of analysis better?

41:22

Maybe. But like how do you know it's

41:24

more detail? More numbers, right? So it

41:27

becomes [snorts]

41:28

very taste driven and very art driven.

41:30

So anyone can post train but what is

41:32

your objective function? That's like the

41:34

key problem. If we could take a

41:37

particular problem and say definitively

41:39

there is a quantifiable objective

41:41

function then that problem can be

41:43

post-trained on. So a very good example

41:46

is math right if you're trying to solve

41:49

math proofs there is a very quantifiable

41:52

objective function because you either

41:53

prove it or you don't and a proof is

41:55

either right or wrong. So in that world

41:58

post training becomes I wouldn't say

42:00

easy but quantifiable and therefore you

42:03

can just run cycles on but if let's say

42:06

we are writing poetry what makes one

42:08

poetry better than the other like I

42:10

don't know it's not quantifiable so if

42:12

it's not quantifiable I cannot create

42:14

that feedback loop of like this wasn't

42:15

good this is good look like this next

42:17

time

42:18

>> so that's what makes post training so

42:20

>> it almost sort of like investing or sort

42:22

of the the the craft of creating a

42:24

thesis almost is more like writing than

42:26

math obviously where it's hard to sort

42:29

of verify X anti and there's been more

42:31

sort of watching the narrative around

42:32

the latest set of models that actually

42:35

like writing is number of people

42:37

observed like writing has actually

42:38

deteriorated the writing quality has

42:39

deteriorated due to

42:41

>> sort of the um uh reinforcement learning

42:44

with verifiable rewards is sort of one

42:46

hypothesis I've I've seen. So that's

42:48

interesting. K you had a question

42:49

>> on that on that point Thomas that none

42:53

am I hearing you correctly that a lot of

42:55

knowledge work is not ver verifiable and

42:57

like discreet with like discrete metrics

43:01

does that it how does that impact how

43:04

does that what's the long-term impact of

43:07

this unverifiability

43:09

for our knowledge work like do we you

43:11

know coding went exponential because it

43:14

is verifiable you know whether Amazon is

43:17

a good buy is is hard to verify. Does

43:20

that create a natural like ceiling for

43:24

LLM usefulness or in the long term?

43:28

>> No, it does not. I don't think so. Um, I

43:30

think what it does is it creates a

43:32

ceiling for trying to post-train a model

43:34

to solve that problem, but doesn't mean

43:37

you cannot break that problem down.

43:40

>> Right? So um we know we we we know we

43:44

cannot determine if one pitch is better

43:46

than the other just by reading it right

43:48

I mean past a certain point right we

43:50

know we we we can't really discern it

43:53

becomes a judgment call but what we do

43:55

know is that within a certain pitch

43:57

there are things that are quantifiable

43:58

and verifiable like your numbers should

44:01

be right your historical numbers should

44:02

be right if you're calculating gross

44:04

margins like your your math should be

44:06

right if you're telling me that there is

44:09

some historical effect that went through

44:11

the business, right? Jeff Bezos retired

44:13

as CEO. Like those are facts that

44:16

shouldn't be questionable. So those you

44:18

can verify, right? It's the judgment

44:21

piece that you probably can't, but it

44:23

doesn't mean that there is nothing that

44:25

you can you can verify on. Um, and

44:27

that's where you can add a lot of value,

44:29

right? Say differently, when I think

44:31

about historically what a human is

44:32

doing, right? A human does three things

44:34

at an investment institution. You're

44:36

either gathering facts, you're analyzing

44:38

the facts, or you're passing judgment on

44:40

the facts. And for the most part,

44:41

anything you don't you're doing that's

44:43

not part of these three is not a good

44:45

use of your time. There's a lot of like

44:46

random stuff that comes up, but it's

44:47

probably not a good use of your time.

44:49

Um, the analysis

44:52

piece, you can very easily quantify a

44:55

lot of it today. Is this a right

44:56

analysis, the right calculation or not?

44:58

If you build a if you build a three-step

45:00

model, did it balance or did it not

45:01

balance? It's verifiable, right? But the

45:04

judgment piece is hard to verify. If

45:06

someone pitched an Amazon long and you

45:08

bought Amazon and it's down tomorrow,

45:10

like was he wrong? Like we know probably

45:13

not. But why is it that we know that?

45:15

What is the right time horizon, right?

45:17

So that becomes a human judgment piece

45:19

that I would say is very hard to do post

45:22

training on uh but on the analysis piece

45:25

is is much easier today.

45:28

I push back a little a little bit on

45:29

that and in and sort of one one world

45:32

that's sort of grown meaningfully over

45:34

the last 10 or 15 years is world of

45:36

factor investing which sort of tries to

45:38

capture that judgment like there is sort

45:39

of an X anti- table odds improvement of

45:42

a company with high free cash flow

45:43

margins versus low free cash flow

45:45

margins. So this is sort of a

45:46

demonstrated sort of tilting of the

45:49

table odds and I think a number of funds

45:50

are starting to think about this concept

45:52

of pattern recognition. How do you

45:53

codify the patterns that have been 60 40

45:56

coins versus 40 60 uh coins? Um

46:00

obviously this this realm of AI has a

46:03

much higher um sort of value quotient

46:06

than update this model. What do you see

46:09

what do you see out there? So there's a

46:12

more nuanced um ability to verify

46:16

um than hey update this model are the

46:18

numbers right? What what what are your

46:20

observations on on sort of that side of

46:22

the world that this thinking machines

46:23

Bridgewater report got a lot of uh

46:26

despite being sort of a um just a very

46:29

simple analysis got a lot of discussion

46:31

about can AI actually start to pass

46:34

better judgment and investment tests.

46:37

>> Yeah, this a good question because the

46:38

question you're really asking is what is

46:40

judgment, right? Can you actually take

46:42

this whole concept of judgment and break

46:44

it down and factors does a great job of

46:45

that. So um I think in short yes the

46:49

answer is yes like AI can get to some

46:51

piece of judgment um assuming that we

46:54

can break down what judgment is. So

46:56

let's think about factors, right?

46:57

[clears throat] Very simply, when I

46:59

think about what makes a stock move,

47:01

there's only one thing that ever makes a

47:03

stock move, which is demand and supply,

47:05

right? Every stock moves because they're

47:07

buyers and they're sellers and there's

47:08

like a higher willingness to buy than

47:11

then sell, prices go up, whatever.

47:13

Demand supply curves. Um, what we're

47:15

trying to figure out is why someone is

47:17

buying. And the reason different people

47:20

buy are very very different. The reason

47:22

the index is buying or like a big mutual

47:24

fund is buying um is very different from

47:27

the reason you know a long only hedge

47:29

fund is buying is very different from

47:31

like a quantis buying right but in the

47:33

market they all get expressed as buying

47:35

signals obviously if a big mutual fund

47:38

is buying let's say the market the S&P

47:41

500 they're probably not just buying

47:43

Amazon they're buying you know 500

47:45

different companies at the same time so

47:46

the so the that creates a market vector

47:49

move that might be different from you

47:53

know whatever a single stock investor

47:55

buying Amazon right so what factors try

47:58

to do is given any single period of time

48:00

break down the reasons for why someone

48:02

is buying by looking at what other

48:05

things are moving so instead of saying

48:07

hey whatever Amazon is the only factor

48:10

is the S&P 500 or you know the Q's or

48:12

whatever you can basically say I have a

48:14

much more intelligent way of thinking

48:16

about it right I can run PCA and I can

48:19

say hey Amazon is you know X amount you

48:22

know correlation with the first factor

48:23

in the PCA and Y amount in the second

48:25

factor and so on and so forth and that

48:27

allows you to without naming what the

48:29

PCA is like what the first order is to

48:32

say that there is a first order right

48:35

there is something you can make a guess

48:36

maybe it's momentum like I don't know

48:37

but you can make a guess right but we

48:40

know that you can statistically

48:41

quoteunquote prove that um the the

48:45

beauty of this is not that the AI can

48:48

pass judgment is that we have taken

48:50

judgement ment of what makes a stock

48:52

move and we have actually broken it down

48:54

into a language that we all agree on

48:57

right which is like sort of like this

48:59

PCA language and therefore we it's

49:02

quantified and then you can bring it to

49:04

an AI what I would actually argue is

49:06

that the stuff that is not quantifiable

49:10

today is the new set of judgment

49:13

>> yeah so let me frame it this way a like

49:16

the factor world has grown up around

49:18

observations of the present right So

49:20

it's sort of by you know by definition

49:23

looking in the rearview mirror of what's

49:25

happened. This company sort of

49:26

accumulated a high momentum loading. The

49:29

sort of intriguing element at least to

49:31

me is started thinking about actually

49:32

the the the look ahead. If I had two

49:34

companies to sort of give a simple

49:36

example one's going to raise revenue

49:38

guidance by 5% one's going to cut

49:40

revenue guidance by 5%. That's like a

49:43

sort of most like not it's not going to

49:44

be a 100% batting average, but that will

49:46

be a good batting average on the

49:49

trajectory of those two stocks, you

49:52

know, sort of classic PCA analysis

49:54

hasn't been able to sort of, you know,

49:57

um find correlations to that. But these

49:59

sort of agent the agentic digital

50:01

analyst approach fed with the right data

50:04

is getting close to having ability to do

50:06

that in some sort of reliable um uh and

50:10

that's a that's a verifi that's sort of

50:12

a more verifiable framework like you can

50:14

sort of look at batting a batting

50:15

averages of revisions. Well, the the the

50:19

thing that is really hard for a human to

50:21

do and if you look at most funds um you

50:23

don't even have the same people solving

50:25

this this problem is that the person

50:27

who's trained in fundamental analysis

50:30

really only thinks in one factor which

50:32

is fundamentals, right? So a person you

50:35

might be really good at predicting

50:37

whether a company is going to raise

50:38

guidance because of all the work that

50:40

you've done but you might not actually

50:42

know or most of the time these people

50:44

aren't in tune with the factor

50:46

environment to say sometimes raising 10%

50:50

guidance in some environments create 10%

50:52

moves for your stock sometimes it

50:54

creates 2% move to your stock right so

50:57

that people who know that tends to be

51:00

PMs risk they tend to not be close

51:03

enough to the fundamentals to be able to

51:06

marry them together. And that's always

51:08

been like a a problem for a lot of

51:10

funds, right? Which is people who

51:12

understand factors don't understand

51:13

fundamentals and vice versa. But the

51:16

stock doesn't care. The stock is a

51:17

combination of both,

51:19

>> right? The move of the stock is a

51:21

combination of every demand and supply

51:23

in the market at that moment. It doesn't

51:25

say today I live in, you know, the

51:28

momentum factor and tomorrow I live in

51:30

the fundamental factor. like everything

51:32

is influencing it at the same time. Uh

51:34

we just don't have people that can

51:35

capture that and the question is can an

51:38

AI do that? Yeah, probably.

51:40

>> Yeah. Yeah. This sort of promise of you

51:42

know quantimental which has been this

51:44

sort of you know one plus one equals

51:46

three concept for decades really which

51:48

has been really hard to like people have

51:50

sort of you know made money on

51:52

quantimental but it's more of like an

51:53

overlay strategy rather than a true

51:55

merging of the best of

51:57

>> both both um uh both practices. I don't

52:00

think there's really any scaled manager

52:02

that has brought the best of both into

52:05

but that's sort of my hypothesis is like

52:07

agent you know the agent architecture

52:09

allows allows a new way of investing

52:12

almost like systematizing the

52:14

fundamental investor

52:16

>> I mean what what what we're seeing a lot

52:18

is like there there is this race to the

52:19

medium right now which is the quan funds

52:21

are basically saying our challenge

52:23

historically has never been on

52:25

understanding factors like we got this

52:27

stuff down pretty good but We couldn't

52:30

get our hands on fundamental data and

52:32

make sense of it in a way we

52:34

systematically could do so with stock

52:35

price with security prices. And with

52:37

fundamental guys, it's we've we've

52:40

always been able to build models and

52:41

talk to companies that understand that,

52:43

but we couldn't systematically think

52:45

about factors and overlay that onto our

52:48

process the way the quants could. So you

52:50

almost see this race into the middle

52:52

where the quants are saying AI has

52:54

unlocked the ability for me to get

52:55

fundamental data, right? How do I marry

52:58

that into my process and to become a

53:00

quant fundamental shop? And the

53:02

fundamental guys are saying now clock

53:04

code has changed the way I look at the

53:06

world. I can now be a software engineer.

53:08

I can analyze security pricing, right?

53:10

How do I go from fundamental to

53:11

fundamental quant and you almost have

53:13

like this like race to the middle

53:16

>> 100%.

53:18

>> Thomas, one other question I had. I I

53:21

spent a couple weeks head heads down

53:23

head down a couple months ago um with

53:25

some of the Excel in integrations

53:27

including you know the dupa MCP into

53:30

claude XL and I was actually quite like

53:33

the simple update a quarter worked. Um,

53:36

beyond that, I was actually quite

53:38

disappointed in the ability to um to do

53:42

things I was hopeful would be um

53:45

possible like asking Claude XL to you

53:48

know rebuild my revenue you know build

53:51

for a resegmentation or handle more um

53:54

complicated issues which you know across

53:57

300 models like you know 100 are going

53:59

to be simple 200 are going to have some

54:01

sort of messy messiness to them. Um,

54:05

where do you think we're at in terms of,

54:07

you know, uh, LLM fluency with Excel?

54:10

The ability to just speak into speak

54:14

into the XL LLM and have it do more

54:16

complicated tasks like that.

54:18

>> Yeah. So, I would have given you a

54:20

different answer a year ago than what

54:22

what I'm going to give you now. So if

54:23

you asked me that a year ago, I would

54:25

have said um the dupa philosophy is we

54:28

build the best data and we pipe it into

54:30

uh whatever platform to go solve that

54:32

problem because the platform is going to

54:34

take care of that problem. Um and so we

54:37

were hopeful that cloud excel with IMCP

54:41

will be able to solve you know all the

54:43

Excel problems. Uh but as you just

54:45

pointed out it's been a year and it's

54:47

not there. And so we looked into why.

54:49

And

54:50

very simply put, the Excel problem, it's

54:54

about a set of specific business use

54:56

cases in Excel um that require a very

55:00

very fine-tuned understanding of what's

55:03

going on. So let's say someone comes

55:06

into a model and says something very

55:07

simple. I want to understand the

55:09

operating leverage of the business. The

55:11

amount of analysis that needs to happen

55:13

is actually very detailed, right? You

55:15

need to first get cash numbers. You need

55:17

to get the non-GAAP numbers out. You

55:20

need to understand everything from how

55:22

capex changes changes your revenue. How

55:25

much of that is because of opex versus

55:27

capex? Um can you actually spend on uh

55:31

can you actually cut back on capex or is

55:33

that like you know fake capex right? So

55:35

there are a lot of little little steps

55:37

that needs to happen that doesn't get

55:40

captured by the simple question um help

55:43

me understand operating leverage for

55:44

this business and then every business is

55:46

different right understanding operating

55:48

leverage for an industrial company and a

55:49

software company despite the fact that

55:52

they have different amounts of operating

55:53

leverage the way you would approach your

55:55

question is also different. So once we

55:58

realized that there are multiple layers

56:01

of depth to this problem, uh we started

56:03

asking ourselves is there an ultimate

56:06

solution of basically saying hey clot

56:09

with my MCP can solve like if the model

56:11

became better can we get there and in

56:13

the near term I think our answer is no

56:15

it cannot get there. Um and so what

56:18

we've then decided to do is we've then

56:21

decided to build our own version of an

56:23

Excel addin. It's not launched yet. Um,

56:26

but the idea is in order to solve the

56:29

Excel problem, it's not an Excel

56:31

problem. It's a very specific problem.

56:33

It's the operating leverage problem.

56:35

It's the hey, can I get the KPIs broken

56:38

down problem? Is the hey, can I

56:40

understand what each KPI represents? Can

56:42

I understand if this company needs to be

56:44

forecasted using FX neutral same store

56:47

sales or non-FX neutral same store sales

56:49

and how do I think about that? So

56:53

each of these set of problems require

56:55

very specific harnesses. Um and it is a

56:58

very very large set of harnessing that

57:00

needs to get produced. So um my view at

57:04

least as of right now is that because

57:06

the labs won't get to the level of

57:08

detail in the harnessing to be able to

57:10

solve this problem. This is actually

57:12

down to the vendor's guide like guys

57:14

like me who are willing to build into

57:16

workflow to go solve

57:17

>> and the LA labs you know have really

57:20

built their modeling corpus towards like

57:23

tuned to investment banking right which

57:25

is a very different which a very

57:27

different modeling exercise than build a

57:29

hedge fund model to forecast revenues e

57:32

>> yeah and the other thing is like is is

57:34

if you think about it this might be

57:36

something that you and I take for

57:37

granted um getting access to hedge fun

57:40

models even knowing what they look like

57:42

is not obvious, right? Because you can't

57:44

just go onto the internet and download a

57:46

model that is hedge fund grade. You can

57:48

download a model that looks like a hedge

57:49

fund grade model, but it's very

57:51

different because what makes it a high

57:53

quality model is not what it looks like.

57:55

I mean, how it looks like is important,

57:57

but it's how it's wired,

57:59

>> right? Is the thought that went through

58:01

the whole thing is why did he forecast

58:04

seasonality versus year-on-year growth?

58:06

Why use incremental costs? Why say the

58:09

company is increasing SGNA by 10 million

58:11

bucks as opposed to percentage of

58:13

revenue like why are each of these micro

58:16

decisions being made? That's what makes

58:18

a good buy model a buy model

58:21

right but very it it's it's not readily

58:24

available on the internet and I think

58:27

most product managers don't even know

58:30

like how what makes like if I showed a

58:33

really good product manager two models

58:35

uh one is a really good one and one is

58:36

an okay one and they look identical on

58:39

the formatting side it's not clear to me

58:41

that your average product person will be

58:43

able to discern it without having done

58:45

the job before.

58:47

One other question I'm I'm wrangling

58:49

with is um sort of we've been on this

58:52

pathway of abstraction with the models

58:54

like you know Claude is three from

58:56

Anthropic has talked about the system

58:58

prompt going down 80% and I'm sort of in

59:00

the process of sort of again narrowing

59:02

down my skills architecture uh with that

59:06

I've sort of going from like a chunking

59:08

like visibility uh like you many many

59:10

skills where I can see the outputs of

59:12

each skill they're more verifiable to

59:14

the model can now take sort of one shot

59:17

with that it sort of feels like eval

59:20

become much more critical because you

59:22

have less visibility. How have you

59:24

thought about building and designing

59:26

evals and managing sort of a skills you

59:29

know creation process through this trend

59:32

of abstraction and and better model

59:34

capability.

59:35

So we we've we've about a year ago maybe

59:39

a little more than a year ago we changed

59:41

the way we think about how to run

59:43

product organizations. So, historically,

59:45

you run product organizations by saying,

59:47

"Hey, here's my PRD. Here's what I'm

59:49

going to build. Here's why. Here's how

59:50

I'm going to build it." And then you go

59:51

build it. Uh, about a little over a year

59:53

ago, we changed it. We basically say,

59:55

you start every PRD with a set of evals.

59:58

So, you ask questions. Um, I care a lot

60:00

about the orthogonality of the

60:02

questions. So, you don't want to ask the

60:03

same questions five different times. You

60:05

want to make sure every question is

60:06

designed to go reflect a particular

60:09

thing a customer will do. And a lot of

60:11

times, we actually ask our customers for

60:13

these questions. So we get eval eval

60:15

from our customers and then everything

60:18

we then do is like as we build a product

60:20

we run through all the eval. So I

60:22

actually have a whole team in my

60:23

business a very large team. Um their job

60:26

is to do eval right. So you're

60:28

constantly running evals. You're

60:30

constantly testing. Sometimes an eval

60:32

takes a week and we don't have time. So

60:34

we just run a small version of it.

60:36

Sometimes we have to run a very very

60:37

broad scope one. We run it across all

60:39

the different models, across

60:40

orchestrations, across MCPs, across

60:43

token costs. So I think it's a cultural

60:46

thing. Um, when we first made the shift,

60:48

it was a little bit awkward. Uh, but now

60:50

we're like very used to it. And I think

60:53

that's just how product is going to have

60:55

to be built in the future. This is like

60:56

the new paradigm of thinking just like

60:59

how Amazon created sort of like the old

61:00

paradigm like start with the output in

61:02

mind, the press release, and then work

61:04

backwards. um today instead of starting

61:06

with the press release, you're starting

61:08

with the end evo and you're working

61:10

backwards towards that eventual like

61:13

let's say you need 100% score on all the

61:15

evos you're working towards that.

61:17

>> What's uh Thomas what's on your your

61:19

bingo card for the next uh three to nine

61:22

months of AI and investing like where

61:25

where do you think we'll be you know by

61:28

spring of 27 what will what will have

61:30

changed?

61:32

Uh I think the biggest change is going

61:34

to come in how people think about team

61:36

structure. So I think we all have a good

61:39

sense of you know the Excel problem.

61:41

Hopefully the loop is the one that that

61:43

gets the enterprisegrade Excel solution

61:45

out there. Um but you know there will be

61:48

an enterprisegrade Excel solution. Um

61:50

there will be more data vendors into the

61:52

space. Um the adoption rates will be

61:54

higher. I think all of that are

61:55

predictable. Uh I think the part that

61:57

people aren't spending enough time

61:59

thinking about is how you hire, how you

62:01

train, and how you do career lettering

62:03

in finance especially on the buy side

62:06

because when I think about the job of an

62:08

analyst, right? Um and this loosely

62:12

tracks with like the career you start

62:14

with fact gathering. Get your numbers

62:15

right, get your facts right, know where

62:17

the documents lie. If Amazon's reporting

62:19

at 4 pm, know that you know facts. And

62:22

then it goes from facts to analysis.

62:24

Right? Now you're a senior analyst. Now

62:26

you're trusted with building models like

62:29

you own the model, you own the analysis,

62:31

you own the pitch. Um to ultimately

62:33

judgment, right? What factor risk am I

62:35

taking? What stocks am I take going

62:36

long? What am I going short? For how

62:38

long? How much leverage do I take?

62:40

Right? There's a taste. I mean obviously

62:41

you get tools, but there's a huge taste

62:43

element to that. Um we are starting to

62:46

see the middle layer collapse, right?

62:49

where

62:50

a good well setup agents can run

62:53

analysis basically at the same quality

62:54

humans can. The difference is that it

62:56

can run 100 times the amount of analysis

62:58

a human can because it doesn't need to

63:00

sleep and I can open up a bunch of

63:02

different agents to go do it. So then

63:05

what that changes is the humans gets I

63:08

wouldn't say forced into but the human

63:09

now needs to spend time on how do I set

63:12

up the infrastructure for the content

63:14

side because an agent is not just going

63:16

to magically go out there and sign a

63:18

contract for you to you know purchase

63:20

fundamental data from dupa or credit

63:21

card data consumer edge or whatever you

63:23

still need to go do that how do you hold

63:25

your vendors accountable how do you hold

63:27

your internal people accountable to

63:30

creating a high quality infrastructure

63:32

and ultimately how do you spend more and

63:34

more of your time on collecting content

63:36

that cannot be easily connect collected.

63:39

So let's say calls with IR teams, right?

63:41

A good agent isn't going to help you win

63:43

more calls with IR teams or talk to the

63:45

right experts like you still have to go

63:47

do that work on your own. And then like

63:50

how do you process judgment? Can you

63:52

create frameworks for like creating more

63:55

judgments for yourself? Like how do you

63:57

increase the ability for yourself to

63:59

pass judgment? How do you even know

64:01

you've gotten better? Right? How do you

64:04

meditate on how much of a decision is

64:07

emotional versus logical, right? Has

64:10

your thesis slipped? Do you think your

64:12

thesis has slipped because you let it or

64:14

because you're because you didn't know?

64:16

So like I think there is a lot more of

64:19

the hiring training side that's going to

64:22

go into the judgment piece, but a lot

64:24

more time is going to be spent on the

64:26

infrastructure piece, whereas

64:28

historically the bulk of the time is

64:30

actually spent on the analysis piece.

64:34

Yeah. Yeah.

64:36

Great. Well, uh, thank you so much, uh,

64:38

for being with us, Thomas. Any any, uh,

64:40

any last thoughts or anything you're

64:42

you're you're wrangling with, uh, right

64:44

now?

64:45

>> I mean, the funny thing is, um, this is

64:48

a conversation we've had recently. The

64:50

single biggest challenge for for us and

64:53

a lot of peers is hiring and

64:56

specifically is in hiring engineers. And

64:58

if you think about it, it's ironic

65:00

because a year ago there was SAS

65:03

apocalypse and everyone was like

65:05

engineers are going to be out of a job

65:07

like clock code is able to code now. And

65:09

now we're sitting here saying well yes

65:13

coding is a solved problem but aligning

65:16

what you're coding with the company's

65:18

direction is not right. Communicating to

65:21

make sure that your evals represent your

65:24

customers problems like that's not

65:25

solved agentically. So while we

65:28

agentically solved the coding problem,

65:30

we did not solve the engineering

65:32

problem, right? So the engineering

65:34

problem now became bigger because we can

65:36

write more code and every single tech

65:39

company is desperately trying to hire

65:41

more engineers,

65:43

which is which I think is very ironic

65:45

but totally logical of like how we ended

65:48

up here.

65:49

>> And I wonder if that's going to be the

65:51

same thing in finance.

65:52

>> Yeah, [snorts] at least for the moment,

65:53

Jevans paradox holds. It's probably

65:56

about a year ago that Daario made his

65:58

prediction that in one to five years,

66:00

you know, white collar work will be down

66:03

uh entry- level white collar work will

66:05

be down 50%. That's thankfully looking

66:08

incredibly absurd as a prediction. I

66:11

think that's true in finance too. I mean

66:13

all the firms we talk to, the people are

66:15

busier than ever. you're maintaining

66:17

your portfolios and research in

66:19

incredibly volatile markets while trying

66:21

to sort of you know shift your process

66:24

>> um agentically and no one's even close

66:26

to the finish line of that. I think it's

66:28

still just starting. So yeah, at least

66:30

for 18 to 36 months I don't I don't see

66:33

any I've not talked to one investment

66:35

team was like wow this is so great that

66:37

we're just going to cut 30% of our

66:39

headcount because it's like we have

66:41

analysts sitting around with nothing to

66:42

do. It's like the exact opposite.

66:44

>> That's right. Yeah. I mean, if you have

66:46

analysts sitting around with nothing to

66:48

do, you are more likely to have an HR

66:50

and analyst problem than you have like a

66:53

business problem.

66:54

>> Yeah. Yeah. You know. Yeah. Awesome.

66:57

Great. Well, um yeah, excited to um

67:01

excited to get my hands on that Excel uh

67:03

thing when it when it comes out as well,

67:05

too. And uh thanks so much for for being

67:07

with us uh for being with us, Thomas.

67:09

This was a great great great discussion.

67:11

>> Thank you for bringing me on.

Interactive Summary

This episode of 'Invest with AI' features Thomas Lee, founder of Dupa, discussing the company's origin and the evolving role of AI in financial data collection and investment workflows. Lee explains how Dupa uses AI to transform manual, human-centric data collection into a high-quality 'production line' model. The discussion highlights the shift from traditional, limited data use to near-universal data consumption through agentic workflows connected to Model Context Protocols (MCPs). Furthermore, they explore the complexities of building reliable MCPs, the competitive landscape for financial data, and the future of human capital in finance as AI takes over analytical tasks while human judgment remains paramount.

Suggested questions

4 ready-made prompts