HomeVideos

Claude Opus 4 + RAG: Build an AI App with Embeddings & Knowledge Graphs | MVP Unplugged

Now Playing

Claude Opus 4 + RAG: Build an AI App with Embeddings & Knowledge Graphs | MVP Unplugged

Transcript

656 segments

0:05

This is MVP Unplugged. In this episode,

0:09

we're going to learn how to write novels

0:12

faster and more easily. We're joined by

0:15

Michael. Michael, welcome.

0:17

>> Thank you. Thank you for having me.

0:19

>> Yeah, so you have built an amazing app

0:22

called AI Story Builders that uses AI to

0:26

write novels and kind of hold together

0:28

the elements of story and narrative

0:31

verse and length and everything like

0:32

that, all the references. I'm really

0:35

excited for you to show your app for us

0:38

today. What inspired you to kind of

0:40

build this?

0:42

>> The thing that inspired me to build it

0:44

was when ChatGPT first came out. I guess

0:48

it's been 3 years now.

0:49

>> Uh-huh.

0:50

>> I wanted to come up with an application

0:53

to use that technology

0:56

that was something that I really cared

0:59

about and needed.

1:00

>> Mhm.

1:01

>> And I

1:03

really wanted to get back into writing

1:05

fictional stories. But whenever I sat

1:08

down to work on a story

1:11

I would have all these notes and, you

1:13

know, the here and papers here and I'm

1:14

using, you know, software like

1:17

Scrivener and um

1:19

I realized I'm building a database. So

1:21

initially I wanted to make a software

1:23

program that would just have a database

1:25

of my characters and that sort of thing

1:27

and then to use AI to kind of you know,

1:30

help, you know, write the story, you

1:33

know, to assist and that sort of thing.

1:35

So that's what inspired me to start

1:37

working on it. And then that led me down

1:40

a whole path.

1:42

And

1:43

you know, we can kind of get into that,

1:45

but but that's what really started me on

1:46

this

1:47

this path.

1:49

>> Mhm. Yeah. And you're a Microsoft MVP as

1:52

well. What inspired you become a

1:54

Microsoft MVP as well?

1:56

>> Um, why

1:57

>> became a Microsoft MVP because I was

1:59

involved um, in an open source uh,

2:02

uh, project called DotNetNuke.

2:04

>> Mhm.

2:05

>> And it was through working on that

2:08

um, and really

2:10

that's what made me fall in love with

2:11

open source and that's why AI Story

2:14

Builders it's fully open source, okay?

2:16

So, you have the full source code,

2:18

there's no, you know, it's MIT license,

2:20

so it it's the most permissible license

2:22

ever. And I saw that the power of open

2:25

source is when you give something back

2:27

to the community what you get back is

2:31

you know, it's measurable because the

2:34

feedback you get um,

2:36

just the support of people um, because

2:40

they realize they're all into this

2:41

together, we're trying to make something

2:43

good together.

2:44

Um, so that's what got me started in

2:46

open source and

2:48

uh, that work led to the MVP, you know,

2:51

it's a recognition of basically, you

2:54

know,

2:55

uh, service to the community, you know,

2:57

it's a recognition award, right? Um, so

3:00

that's what started that and then I've

3:02

uh,

3:03

you know,

3:04

off and on I've been an MVP for uh, I

3:07

guess almost 20 years now.

3:09

>> Mhm.

3:10

>> And um,

3:11

you know, I just love the whole you're

3:13

part of a community um, and just really

3:16

that, being part of a a dev community,

3:18

you know, you're not just coding on your

3:20

own.

3:21

So, that's why I love to stay in the

3:23

program and every year I you know, um,

3:26

you know, I hope to be uh, to receive

3:27

the award.

3:28

>> So, I'm really excited Michael to kind

3:29

of get into this and for you to share

3:31

your project. So, uh, let's let's start

3:34

there. Why don't you share AI Story

3:35

Builders?

3:36

>> Okay. So, uh, this page here, if you see

3:39

my screen um, it's just showing if if

3:42

you go to AIStoryBuilders.com

3:45

uh, you'll see this page here which kind

3:47

of, you know, it's kind of a little

3:48

landing page here. There's actually a

3:50

little video that

3:51

uh

3:53

you know, gives you an overview.

3:54

But, it basically then talks about how

3:57

to um install it. And um the way you

4:01

install it is you can actually get it at

4:03

the uh

4:05

the the Microsoft Store. Again, it's

4:07

totally free. Um but, you can just

4:09

download from the Microsoft Store.

4:10

That's probably the easiest thing to do.

4:12

And there you go. However, of course,

4:15

there's a GitHub repo. So, this is

4:17

showing the GitHub repo. And here's

4:19

where you can see all the, you know, all

4:22

the code here. Of course, still

4:23

explanations, um you know, some more

4:26

documentation about it. Um so,

4:29

you can get to it uh either way.

4:31

And uh so, when you run the application,

4:34

um it pretty much looks like this. Uh

4:36

here I already have some stories loaded.

4:39

But, when you first come in, what

4:41

happens is is that you're going to first

4:43

want to go to settings, and you're going

4:45

to want to

4:46

uh select your AI provider, either

4:48

OpenAI, Anthropic, Google. Um I will say

4:51

that the Anthropic model, especially the

4:54

latest uh Opus models, that's the one

4:57

you want to use. Um yeah, it costs a

4:59

little bit more, but the performance is

5:01

just uh immeasurable.

5:03

>> It's amazing to see the innovation in

5:05

models. It seems like every day there's

5:07

a new model coming out and the new one

5:09

on the leaderboard uh as well. Pretty

5:11

exciting.

5:12

>> Oh, of course. Yeah. And remember, I

5:14

started this uh

5:15

uh project 3 years ago. So, I was using

5:18

models that were less much less capable.

5:21

So, I had to make this tool work with

5:23

those models that were

5:25

you know, looking back at them now are

5:26

quite limited. Um yet

5:29

uh

5:30

with the newer models that are capable

5:32

of so much more,

5:33

what this application can do today, um I

5:37

feel is really night and day. Um so,

5:40

once you put that in there, you just

5:41

really want to start with the new story.

5:43

And uh let me just grab uh my little

5:46

sample here that I normally use three

5:50

little pigs.

5:52

And um

5:56

And then you can select how many

5:59

chapters you want to create. It's only

6:02

going to create the first paragraph of

6:03

each chapter, but it just kind of gives

6:05

an outline. And then of course your your

6:08

AI model and again you see I'm using

6:10

Claude Opus 4 and then you just hit

6:13

create story. So as this is going,

6:18

let me just quickly

6:20

show you the challenges I had with

6:23

creating this. So initially one would

6:27

think, "Oh, I can just send to the

6:30

prompt this is the story I'm working on.

6:34

Please either fix something or write the

6:37

next paragraph or you know, or actually

6:39

some people thought write the next

6:41

chapter." And I found that that doesn't

6:43

work. And I did I had a number of proof

6:45

of concepts I created over a period of 6

6:48

months. And I remember I didn't have the

6:50

AI coding wasn't what we have today. So

6:52

I could have made those those

6:54

proof of concepts much faster today, but

6:57

back then you got to you know, put

6:58

yourself 3 years ago.

6:59

Um

7:00

So I made a proof of concept after proof

7:02

of concept and and thing everything I

7:05

tried wasn't working.

7:07

Okay? So this is a story about failure,

7:09

all right? And and what do you do when

7:11

you just something's not working? And

7:15

what I do is I just I I try to be I try

7:19

to have a little bit of humility, right?

7:21

And just see things the way they truly

7:23

are not the way I want them to be. And

7:25

the way things truly are was

7:28

I could not get AI to write

7:31

even a page, never less a chapter. I

7:34

could not get it to write something that

7:36

was coherent.

7:38

>> [laughter]

7:38

>> You know, and again the models were less

7:40

capable of that it then. Um, but it

7:43

would write something that was well it

7:45

was coherent, but it wasn't good.

7:48

>> Yeah, yeah.

7:49

>> So,

7:50

I realized that the best I could get the

7:54

AI to do is to write one paragraph.

7:58

And for it to even write one paragraph,

8:01

I had to provide so much information to

8:04

the prompt

8:05

>> Yeah.

8:05

>> that you know, the program just needs to

8:07

keep track of that.

8:08

>> Yeah, this was Yeah, I was going to say

8:10

this was my experience as well in the

8:12

MVP program as I would have to write

8:14

huge prompts including do not do this,

8:17

do not do this, do not do this, right?

8:19

And I would like through process of

8:20

elimination, it would get better, but

8:23

enter much better models nowadays,

8:25

right?

8:26

>> Okay, enter much better models. However,

8:28

um, and hopefully [clears throat] we're

8:29

going to touch upon this uh

8:31

is

8:32

um

8:33

I found that it's still about how much

8:37

you give the prompt, you know? As we

8:39

know, AI is just an algorithm. It

8:41

doesn't really think. It's an algorithm.

8:43

It's a calculation. And therefore, what

8:45

you put in is what you get out. The base

8:47

program was was based upon this

8:49

structure that I'm showing here, which

8:51

is I realize I have to break story down

8:54

into core elements. And

8:58

it turns out that it's truly in this

9:00

order. So, this image here is just

9:02

saying this is story, right? And obvious

9:05

things, a story consists of chapters and

9:07

and characters and look, you know, and

9:10

locations and that sort of thing. So, we

9:12

know that, you know, people know a story

9:14

has characters and that you need to keep

9:15

track of the characters. Okay, that

9:16

part's obvious.

9:18

But that's actually not the first part

9:19

of story.

9:20

>> Yeah, yeah.

9:21

>> That's what I found in my experience.

9:22

The first part of story

9:24

is timeline.

9:27

When So, even in a story you have

9:29

flashbacks, right? Um, so a flashback is

9:33

is controlled by time. So, if the

9:36

character is 5 years old in the

9:37

flashback, that is controlling, okay?

9:41

Everything has to hang off of of that

9:43

timeline. So, I realized that you start

9:45

off with timeline first. So, in my

9:48

database structure, I start off with

9:50

timeline. After timeline, even before

9:53

you get to characters, is location. So,

9:55

the when and the where

9:58

>> Yeah. Yeah.

9:59

>> With with fictional stories,

10:01

um

10:01

is or actually even a true story, but

10:04

even story versus say you and I are

10:06

talking right now, story is controlled

10:09

by timeline and location.

10:12

Even if the Even if the location is

10:14

unspecified, it is still a location. You

10:18

know what I'm saying? Um for example,

10:20

it's all it's unknown to the to the to

10:22

the reader, but it is a location that is

10:24

distinct from say another location. So,

10:27

these two things are something I

10:28

discovered. Um so, timeline, location,

10:31

and then we get to character. And then

10:33

with character, you have attributes,

10:35

things like the hair color, um

10:37

their the background history, and all

10:38

that sort of thing. But, it goes in this

10:41

order. So, for example, a character's

10:43

attributes, whether they have red hair

10:44

or blue hair, is on always on a

10:47

timeline.

10:48

>> I see what I love about this, Michael,

10:49

is that you're you are defining a

10:51

combination of, you know, relational

10:54

data that is explicit, that is you're

10:56

you're telling the model this is what

10:58

you want in the story or you're helping

11:00

generate it. But, then you're allowing

11:02

the model to then bring these different

11:04

pieces together to form, you know,

11:05

paragraphs or you know, to to write

11:08

>> Right. Yeah, I feed those Right, I feed

11:09

those to the model. Um and then of

11:11

course I I

11:13

Uh you have chapters which uh compose of

11:16

paragraphs. Um another decision I did

11:18

make with this application is

11:21

um I

11:23

didn't want to have a special database

11:26

um

11:27

So, I actually use text files. So,

11:29

everything is using uh the text files to

11:32

store all these elements. Um I course

11:35

use embeddings so that I can use cosine

11:38

similarity

11:39

to surface you know things that are

11:41

related to each other. And you know,

11:44

that's pretty much the

11:46

the base of the application and then now

11:48

you can see with the three little pigs

11:50

here that we started off with. It

11:52

creates a little synopsis

11:54

and this course is fed to

11:56

the AI in the prompt. You can also have

12:00

system messages and these are things

12:02

like you know, don't have foreshadowing

12:05

maybe the the

12:07

perspective is third person limited.

12:09

Again, this sort of thing you can change

12:10

and you can of course even have world

12:12

facts things like you know, magic is

12:14

real and and that sort of thing. But

12:16

then you see it breaks things down. So

12:18

even with that little paragraph that I

12:21

fed to it, it said hey,

12:23

you know, there's different timelines.

12:25

Farewell to mother

12:27

you know, the three little pigs leave

12:28

their mother. They're constructing their

12:30

houses. The wolf arrives. We know this,

12:32

right?

12:33

Mother is a flea to safety. The wolf

12:35

fell to tax because this course has a

12:37

happy ending and the wolf's demise.

12:40

You have the locations. The pigs straw

12:42

house, the mother's forest home that

12:44

they live. The pigs stick house, the

12:46

brick house, right? These are the

12:47

locations that you would expect in the

12:49

story. Then you have the characters and

12:51

the three little pigs Joe, Tom and the

12:53

wolf. Um

12:54

and

12:56

and then you have the mother course is

12:58

off screen I guess in this first

13:00

And the course you have the chapters.

13:01

Chapters have a synopsis. Apologize if

13:04

this is hard to read but just it's there

13:07

and notice we only have the first

13:09

paragraph of each of the chapters. And

13:12

the reason for this again is is that I

13:14

found and this even with the new

13:15

frontier models

13:17

you only want

13:19

the AI to help you write one chapter at

13:22

a time. So what happens is is that you

13:25

can come in here and you can kind of

13:26

start and work on the next chapter. Um

13:29

or you can just click this AI button and

13:32

describe what you want.

13:34

Or you can just click this button and

13:38

it'll just continue from the previous

13:41

chapter.

13:42

>> I like that.

13:43

>> Right. And what's happening here is is

13:45

that I'm feeding to the AI

13:47

all the context of okay, you're working

13:51

on a chapter. Um and of course let's go

13:53

ahead and save here. Um working on a

13:55

chapter and this previous chapter

13:57

these are the characters that are in

13:58

this chapter. Um

14:00

this is the location that's this

14:01

chapter. So therefore everything that's

14:03

you know, tied to that location. Um of

14:05

course this is the timeline to that

14:07

chapter and everything is tied to that.

14:09

Um so it's all fed to the AI and then

14:11

the AI of course writes this prose.

14:13

Which again, the frontier models this

14:16

prose would be

14:17

it's better than say a uh

14:20

uh you know, a less capable model. But

14:25

if I tried to tell it to write too much

14:28

it'll write something good, but it'll

14:30

kind of get off and it's just not a

14:33

human, you know.

14:34

Um and one more quick thing. Um

14:37

updating this the database of of say the

14:39

characters and and uh you know, all

14:42

their uh

14:43

you know, elements. While that can be

14:45

time consuming, what you can do is you

14:48

can simply click um auto detect

14:51

attributes. And what happens is is that

14:54

the AI will look at this this and say,

14:56

"Hey look, there's something new and I

14:59

can just you know, add that to the

15:00

database." So that's a way for the you

15:02

to add things to the database uh

15:04

quickly and easily.

15:05

>> Yeah, I suspect

15:07

Yeah, I suspect that as you write more

15:09

chapters, the AI gets better and better

15:13

at writing chapters aligned with what

15:15

you know, what your vision is.

15:17

>> cuz you're giving it more information.

15:18

You are, yes.

15:19

>> Yeah, yeah. So like so say you write

15:21

chapter one, but then you make human

15:23

refinements to it. Say, I don't know,

15:25

say you edit it by 50% compared to what

15:27

AI did. But then the next chapter you're

15:29

going to edit it by 25% and the next

15:31

chapter you're going to edit it by 10%,

15:33

right? Because uh theoretically or or,

15:36

you know, as we think about this like,

15:37

you know, um it it understands the

15:39

context for how to make the story more

15:41

focused, especially when you give all

15:43

these attributes like timelines and

15:45

locations and characters and such, you

15:46

know.

15:49

>> And a couple of other quick features

15:50

about it. Um you can import uh an

15:53

existing story, whether it's uh you

15:54

know, Word, a PDF, or text. So, that's

15:57

why I um imported, for example, The

16:00

Great Gatsby. It's in it's in uh

16:03

you know,

16:04

uh public use. Um

16:05

>> Yeah, one of my favorite books. It's one

16:07

of my Yeah, one of my favorites.

16:08

>> Right. See, so it's all uh here and of

16:10

course it broke down, you know, these

16:12

are the characters.

16:13

Um

16:14

You You'll see there was actually a

16:15

little bit of uh

16:17

uh issues here where, you know, Daisy

16:20

and Daisy Buchanan and Daisy or it

16:22

thinks it's two different people. So, um

16:24

this is something I'm still working on

16:26

here. Um

16:27

but yeah, and these are all the

16:29

different locations and that sort of

16:30

thing. So, you can import an existing uh

16:34

uh story and of course you can um

16:38

uh export um your story to um

16:43

you know, as a Word document.

16:44

>> You know what I like about, you know,

16:46

The Great Gatsby in this case and some

16:48

of the other stories that you've

16:49

imported is uh it's one thing to write

16:52

The Three Little Pigs, which is a fairly

16:54

simple story with, you know, a few

16:56

characters, a few settings. Uh but when

16:58

you get to The Great Gatsby or, you

17:01

know, J.R.R. Tolkien or other more

17:03

complex things, it it articulates the

17:05

scale potential of AI. An AI-based

17:08

solution like this can help us keep our

17:10

own story together and uh avoid

17:13

inconsistencies in the story, which I

17:14

think is is especially as it gets more

17:16

complex.

17:17

>> Right. So, like here,

17:19

uh this right here is the last paragraph

17:21

in The Great Gatsby, and you can

17:23

basically say, "You know what?

17:25

Continue the story."

17:27

>> Yeah.

17:28

>> And and this will actually uh you know,

17:31

literally

17:33

continue the story of The Great Gatsby.

17:34

So, it's kind of wild when you kind of

17:36

think about it.

17:37

>> fan fiction.

17:39

>> Yeah, exactly. Yeah, fan fiction,

17:40

exactly.

17:41

>> [laughter]

17:41

>> So.

17:43

>> Well, you mentioned Michael a little

17:44

earlier about, you know, others, you

17:46

know, for our viewers and our listeners

17:47

out there that that you all can build

17:49

this. Um tell me a little bit more about

17:51

like how would you how did you build

17:52

this?

17:52

>> So, if you look in your hard drive, your

17:54

documents folder, you'll see a folder

17:56

called AI Story Builders, and there

17:58

here's where you'll see all the

18:00

uh folder for each story, and then

18:02

you'll see uh folders for each part of

18:04

the story. So, for example, I go into

18:06

chapters, and then you'll see folders

18:08

for the chapters, and then when I go in

18:10

there, you'll see um a text file for

18:14

each paragraph, and if you open up the

18:16

paragraph, you see something looks like

18:18

this, which is these are the characters

18:21

that are in that paragraph. This is the

18:23

prose of that paragraph, and here are

18:26

the embeddings uh

18:28

that represent that paragraph, and

18:32

the program literally

18:34

you know, opens these files up, reads

18:37

these embeddings, uses rag, uh which is

18:40

retrieval augmented generation, which

18:42

then uses a calculation called cosine

18:44

similarity to then determine what uh

18:48

paragraphs are related to each other,

18:51

and therefore it then creates the prompt

18:54

that is then sent to the AI.

18:57

Which leads us into the why I'm now

18:59

using knowledge this I'm still using

19:01

this, but I'm also now with rag using

19:04

knowledge graphs to

19:06

uh what I what in my opinion really

19:09

makes the this program 10 times better.

19:13

>> All right, so Michael, uh, tell me a

19:15

little bit more about this new graph

19:17

feature, uh, that you are building. I'm

19:20

pretty excited to see, um, you know,

19:23

what you've learned and how you're

19:24

growing and how you're improving this,

19:25

uh, this overall AI story builders.

19:27

>> The first thing I did was I,

19:30

uh, went into, you know, kind of I have

19:31

coded an app and I said, "Hey, look at

19:33

my existing AI story builder files and

19:37

create a, uh, knowledge graph

19:39

>> Okay.

19:40

>> and it created uh, something that looks

19:43

like this.

19:44

>> Yeah, so [clears throat] this is a JSON

19:45

file, um, that then is going to

19:47

aggregate all this, all this data into a

19:49

visualization.

19:51

>> Right. So, this is like the raw thing,

19:53

okay?

19:54

>> Yeah. Yeah.

19:54

>> However, I then vibe coded, uh, an app

19:58

that where I basically said, "Hey, can

20:00

you, um,

20:02

show me,

20:04

um,

20:05

let me just start here.

20:07

Uh, can you just visualize this graph?"

20:10

So, here, uh,

20:13

using the same Sherlock Holmes,

20:15

I

20:17

basically come up with something. I know

20:19

this is impossible to see, so don't

20:20

worry, I'm going to zoom in here, but

20:22

this is Sherlock Holmes, you know, that

20:25

knowledge graph I just showed you. This

20:26

is it visualized.

20:28

>> All right, so Michael, now you're, uh,

20:30

visualizing the adventures of Sherlock

20:33

Holmes. Uh, tell us a little bit more

20:35

about how, you know, an author would

20:37

utilize this to kind of keep, uh, the

20:39

story straight and understand where to

20:41

go next.

20:42

>> Right. So, this right here is just a

20:44

visualization of the graph just to get

20:46

an idea. Um, a graph, of course,

20:48

consists of entities, like these are the

20:50

characters, and then you have the edges,

20:52

which contain information that this

20:54

character interacts with this character,

20:56

um, and so on. And of course, um, we

20:59

also have everything broken down, um, by

21:01

paragraphs so that you can see what

21:03

characters interact [clears throat]

21:05

with, uh, which paragraph, uh, with with

21:07

that. So, what happens is is that this

21:10

graph is then exposed to the AI

21:13

as tools.

21:14

>> Mhm.

21:15

>> So, therefore, the AI is able to use

21:17

those tools to make um as many calls as

21:22

it needs to interrogate this graph.

21:26

So, what I've added is a chat window

21:30

that allows you to do things like

21:32

indicate the interactions between Watson

21:34

and Holmes in chapter one.

21:36

>> Yeah.

21:37

>> is it's able to come up with this um

21:40

very, very detailed uh response that I

21:43

would not what I normally would not be

21:45

able to surface using simple retrieval

21:48

augmented generation because with rag,

21:51

I'm limited by how much I can can

21:54

surface, you know, to the prompt.

21:56

Whereas here, the frontier models using

21:59

their advanced tool uh calling

22:01

capability, it can make as many calls as

22:04

it needs to that graph and therefore

22:06

surface everything that it needs. Um and

22:10

what I've also added is the way to go um

22:14

the other way, whereas you can say,

22:16

"Okay, you know, you've identified

22:18

issues. Please update um

22:22

you know, the story or update the

22:23

chapter to, you know, fix those issues."

22:26

>> Mhm.

22:27

>> So, therefore, I've exposed these tools

22:29

that allow it to update the files. So,

22:32

this

22:33

you know, so this capability is what's

22:35

being added. Um I should be releasing

22:37

that this weekend.

22:38

>> I love this cuz you're you're asking

22:39

questions to keep your story straight,

22:41

but then you're using this as context to

22:43

then write more story and have that be

22:45

more accurate as well.

22:47

>> Right.

22:47

>> Wow.

22:48

>> So, I look forward to the hopefully the

22:49

community uh will uh will like this

22:52

feature and then also can see how they

22:54

can adapt this technology to their own

22:56

applications by identifying uh things in

22:59

their applications which uh

23:02

um they can identify, you know, entities

23:04

and edges to create knowledge graphs and

23:07

then expose that knowledge graph to the

23:10

AI through tools that then allow their

23:13

applications to um you know, just go

23:17

deeper than just mere rag can can

23:20

currently provide.

23:21

>> Well, Michael, thank you so much for

23:23

sharing uh this application and sharing

23:26

your passion, being a Microsoft MVP. Uh

23:29

this is a fascinating uh project. And

23:31

so, I really appreciate you sharing it

23:32

today and thank you for being here on

23:35

MVP Unplugged.

23:37

>> Thank you for having me.

Interactive Summary

In this episode of MVP Unplugged, Michael showcases 'AI Story Builders', an open-source application he developed to assist writers in composing novels using AI. The tool leverages a unique database-like structure focusing on timelines, locations, and character attributes to guide AI in generating coherent narrative prose. Michael discusses the evolution of the software, the shift towards utilizing advanced models like Claude Opus, and the integration of Retrieval-Augmented Generation (RAG) and knowledge graphs to maintain narrative consistency in complex stories.

Suggested questions

3 ready-made prompts