Claude Opus 4 + RAG: Build an AI App with Embeddings & Knowledge Graphs | MVP Unplugged
656 segments
This is MVP Unplugged. In this episode,
we're going to learn how to write novels
faster and more easily. We're joined by
Michael. Michael, welcome.
>> Thank you. Thank you for having me.
>> Yeah, so you have built an amazing app
called AI Story Builders that uses AI to
write novels and kind of hold together
the elements of story and narrative
verse and length and everything like
that, all the references. I'm really
excited for you to show your app for us
today. What inspired you to kind of
build this?
>> The thing that inspired me to build it
was when ChatGPT first came out. I guess
it's been 3 years now.
>> Uh-huh.
>> I wanted to come up with an application
to use that technology
that was something that I really cared
about and needed.
>> Mhm.
>> And I
really wanted to get back into writing
fictional stories. But whenever I sat
down to work on a story
I would have all these notes and, you
know, the here and papers here and I'm
using, you know, software like
Scrivener and um
I realized I'm building a database. So
initially I wanted to make a software
program that would just have a database
of my characters and that sort of thing
and then to use AI to kind of you know,
help, you know, write the story, you
know, to assist and that sort of thing.
So that's what inspired me to start
working on it. And then that led me down
a whole path.
And
you know, we can kind of get into that,
but but that's what really started me on
this
this path.
>> Mhm. Yeah. And you're a Microsoft MVP as
well. What inspired you become a
Microsoft MVP as well?
>> Um, why
>> became a Microsoft MVP because I was
involved um, in an open source uh,
uh, project called DotNetNuke.
>> Mhm.
>> And it was through working on that
um, and really
that's what made me fall in love with
open source and that's why AI Story
Builders it's fully open source, okay?
So, you have the full source code,
there's no, you know, it's MIT license,
so it it's the most permissible license
ever. And I saw that the power of open
source is when you give something back
to the community what you get back is
you know, it's measurable because the
feedback you get um,
just the support of people um, because
they realize they're all into this
together, we're trying to make something
good together.
Um, so that's what got me started in
open source and
uh, that work led to the MVP, you know,
it's a recognition of basically, you
know,
uh, service to the community, you know,
it's a recognition award, right? Um, so
that's what started that and then I've
uh,
you know,
off and on I've been an MVP for uh, I
guess almost 20 years now.
>> Mhm.
>> And um,
you know, I just love the whole you're
part of a community um, and just really
that, being part of a a dev community,
you know, you're not just coding on your
own.
So, that's why I love to stay in the
program and every year I you know, um,
you know, I hope to be uh, to receive
the award.
>> So, I'm really excited Michael to kind
of get into this and for you to share
your project. So, uh, let's let's start
there. Why don't you share AI Story
Builders?
>> Okay. So, uh, this page here, if you see
my screen um, it's just showing if if
you go to AIStoryBuilders.com
uh, you'll see this page here which kind
of, you know, it's kind of a little
landing page here. There's actually a
little video that
uh
you know, gives you an overview.
But, it basically then talks about how
to um install it. And um the way you
install it is you can actually get it at
the uh
the the Microsoft Store. Again, it's
totally free. Um but, you can just
download from the Microsoft Store.
That's probably the easiest thing to do.
And there you go. However, of course,
there's a GitHub repo. So, this is
showing the GitHub repo. And here's
where you can see all the, you know, all
the code here. Of course, still
explanations, um you know, some more
documentation about it. Um so,
you can get to it uh either way.
And uh so, when you run the application,
um it pretty much looks like this. Uh
here I already have some stories loaded.
But, when you first come in, what
happens is is that you're going to first
want to go to settings, and you're going
to want to
uh select your AI provider, either
OpenAI, Anthropic, Google. Um I will say
that the Anthropic model, especially the
latest uh Opus models, that's the one
you want to use. Um yeah, it costs a
little bit more, but the performance is
just uh immeasurable.
>> It's amazing to see the innovation in
models. It seems like every day there's
a new model coming out and the new one
on the leaderboard uh as well. Pretty
exciting.
>> Oh, of course. Yeah. And remember, I
started this uh
uh project 3 years ago. So, I was using
models that were less much less capable.
So, I had to make this tool work with
those models that were
you know, looking back at them now are
quite limited. Um yet
uh
with the newer models that are capable
of so much more,
what this application can do today, um I
feel is really night and day. Um so,
once you put that in there, you just
really want to start with the new story.
And uh let me just grab uh my little
sample here that I normally use three
little pigs.
And um
And then you can select how many
chapters you want to create. It's only
going to create the first paragraph of
each chapter, but it just kind of gives
an outline. And then of course your your
AI model and again you see I'm using
Claude Opus 4 and then you just hit
create story. So as this is going,
let me just quickly
show you the challenges I had with
creating this. So initially one would
think, "Oh, I can just send to the
prompt this is the story I'm working on.
Please either fix something or write the
next paragraph or you know, or actually
some people thought write the next
chapter." And I found that that doesn't
work. And I did I had a number of proof
of concepts I created over a period of 6
months. And I remember I didn't have the
AI coding wasn't what we have today. So
I could have made those those
proof of concepts much faster today, but
back then you got to you know, put
yourself 3 years ago.
Um
So I made a proof of concept after proof
of concept and and thing everything I
tried wasn't working.
Okay? So this is a story about failure,
all right? And and what do you do when
you just something's not working? And
what I do is I just I I try to be I try
to have a little bit of humility, right?
And just see things the way they truly
are not the way I want them to be. And
the way things truly are was
I could not get AI to write
even a page, never less a chapter. I
could not get it to write something that
was coherent.
>> [laughter]
>> You know, and again the models were less
capable of that it then. Um, but it
would write something that was well it
was coherent, but it wasn't good.
>> Yeah, yeah.
>> So,
I realized that the best I could get the
AI to do is to write one paragraph.
And for it to even write one paragraph,
I had to provide so much information to
the prompt
>> Yeah.
>> that you know, the program just needs to
keep track of that.
>> Yeah, this was Yeah, I was going to say
this was my experience as well in the
MVP program as I would have to write
huge prompts including do not do this,
do not do this, do not do this, right?
And I would like through process of
elimination, it would get better, but
enter much better models nowadays,
right?
>> Okay, enter much better models. However,
um, and hopefully [clears throat] we're
going to touch upon this uh
is
um
I found that it's still about how much
you give the prompt, you know? As we
know, AI is just an algorithm. It
doesn't really think. It's an algorithm.
It's a calculation. And therefore, what
you put in is what you get out. The base
program was was based upon this
structure that I'm showing here, which
is I realize I have to break story down
into core elements. And
it turns out that it's truly in this
order. So, this image here is just
saying this is story, right? And obvious
things, a story consists of chapters and
and characters and look, you know, and
locations and that sort of thing. So, we
know that, you know, people know a story
has characters and that you need to keep
track of the characters. Okay, that
part's obvious.
But that's actually not the first part
of story.
>> Yeah, yeah.
>> That's what I found in my experience.
The first part of story
is timeline.
When So, even in a story you have
flashbacks, right? Um, so a flashback is
is controlled by time. So, if the
character is 5 years old in the
flashback, that is controlling, okay?
Everything has to hang off of of that
timeline. So, I realized that you start
off with timeline first. So, in my
database structure, I start off with
timeline. After timeline, even before
you get to characters, is location. So,
the when and the where
>> Yeah. Yeah.
>> With with fictional stories,
um
is or actually even a true story, but
even story versus say you and I are
talking right now, story is controlled
by timeline and location.
Even if the Even if the location is
unspecified, it is still a location. You
know what I'm saying? Um for example,
it's all it's unknown to the to the to
the reader, but it is a location that is
distinct from say another location. So,
these two things are something I
discovered. Um so, timeline, location,
and then we get to character. And then
with character, you have attributes,
things like the hair color, um
their the background history, and all
that sort of thing. But, it goes in this
order. So, for example, a character's
attributes, whether they have red hair
or blue hair, is on always on a
timeline.
>> I see what I love about this, Michael,
is that you're you are defining a
combination of, you know, relational
data that is explicit, that is you're
you're telling the model this is what
you want in the story or you're helping
generate it. But, then you're allowing
the model to then bring these different
pieces together to form, you know,
paragraphs or you know, to to write
>> Right. Yeah, I feed those Right, I feed
those to the model. Um and then of
course I I
Uh you have chapters which uh compose of
paragraphs. Um another decision I did
make with this application is
um I
didn't want to have a special database
um
So, I actually use text files. So,
everything is using uh the text files to
store all these elements. Um I course
use embeddings so that I can use cosine
similarity
to surface you know things that are
related to each other. And you know,
that's pretty much the
the base of the application and then now
you can see with the three little pigs
here that we started off with. It
creates a little synopsis
and this course is fed to
the AI in the prompt. You can also have
system messages and these are things
like you know, don't have foreshadowing
maybe the the
perspective is third person limited.
Again, this sort of thing you can change
and you can of course even have world
facts things like you know, magic is
real and and that sort of thing. But
then you see it breaks things down. So
even with that little paragraph that I
fed to it, it said hey,
you know, there's different timelines.
Farewell to mother
you know, the three little pigs leave
their mother. They're constructing their
houses. The wolf arrives. We know this,
right?
Mother is a flea to safety. The wolf
fell to tax because this course has a
happy ending and the wolf's demise.
You have the locations. The pigs straw
house, the mother's forest home that
they live. The pigs stick house, the
brick house, right? These are the
locations that you would expect in the
story. Then you have the characters and
the three little pigs Joe, Tom and the
wolf. Um
and
and then you have the mother course is
off screen I guess in this first
And the course you have the chapters.
Chapters have a synopsis. Apologize if
this is hard to read but just it's there
and notice we only have the first
paragraph of each of the chapters. And
the reason for this again is is that I
found and this even with the new
frontier models
you only want
the AI to help you write one chapter at
a time. So what happens is is that you
can come in here and you can kind of
start and work on the next chapter. Um
or you can just click this AI button and
describe what you want.
Or you can just click this button and
it'll just continue from the previous
chapter.
>> I like that.
>> Right. And what's happening here is is
that I'm feeding to the AI
all the context of okay, you're working
on a chapter. Um and of course let's go
ahead and save here. Um working on a
chapter and this previous chapter
these are the characters that are in
this chapter. Um
this is the location that's this
chapter. So therefore everything that's
you know, tied to that location. Um of
course this is the timeline to that
chapter and everything is tied to that.
Um so it's all fed to the AI and then
the AI of course writes this prose.
Which again, the frontier models this
prose would be
it's better than say a uh
uh you know, a less capable model. But
if I tried to tell it to write too much
it'll write something good, but it'll
kind of get off and it's just not a
human, you know.
Um and one more quick thing. Um
updating this the database of of say the
characters and and uh you know, all
their uh
you know, elements. While that can be
time consuming, what you can do is you
can simply click um auto detect
attributes. And what happens is is that
the AI will look at this this and say,
"Hey look, there's something new and I
can just you know, add that to the
database." So that's a way for the you
to add things to the database uh
quickly and easily.
>> Yeah, I suspect
Yeah, I suspect that as you write more
chapters, the AI gets better and better
at writing chapters aligned with what
you know, what your vision is.
>> cuz you're giving it more information.
You are, yes.
>> Yeah, yeah. So like so say you write
chapter one, but then you make human
refinements to it. Say, I don't know,
say you edit it by 50% compared to what
AI did. But then the next chapter you're
going to edit it by 25% and the next
chapter you're going to edit it by 10%,
right? Because uh theoretically or or,
you know, as we think about this like,
you know, um it it understands the
context for how to make the story more
focused, especially when you give all
these attributes like timelines and
locations and characters and such, you
know.
>> And a couple of other quick features
about it. Um you can import uh an
existing story, whether it's uh you
know, Word, a PDF, or text. So, that's
why I um imported, for example, The
Great Gatsby. It's in it's in uh
you know,
uh public use. Um
>> Yeah, one of my favorite books. It's one
of my Yeah, one of my favorites.
>> Right. See, so it's all uh here and of
course it broke down, you know, these
are the characters.
Um
You You'll see there was actually a
little bit of uh
uh issues here where, you know, Daisy
and Daisy Buchanan and Daisy or it
thinks it's two different people. So, um
this is something I'm still working on
here. Um
but yeah, and these are all the
different locations and that sort of
thing. So, you can import an existing uh
uh story and of course you can um
uh export um your story to um
you know, as a Word document.
>> You know what I like about, you know,
The Great Gatsby in this case and some
of the other stories that you've
imported is uh it's one thing to write
The Three Little Pigs, which is a fairly
simple story with, you know, a few
characters, a few settings. Uh but when
you get to The Great Gatsby or, you
know, J.R.R. Tolkien or other more
complex things, it it articulates the
scale potential of AI. An AI-based
solution like this can help us keep our
own story together and uh avoid
inconsistencies in the story, which I
think is is especially as it gets more
complex.
>> Right. So, like here,
uh this right here is the last paragraph
in The Great Gatsby, and you can
basically say, "You know what?
Continue the story."
>> Yeah.
>> And and this will actually uh you know,
literally
continue the story of The Great Gatsby.
So, it's kind of wild when you kind of
think about it.
>> fan fiction.
>> Yeah, exactly. Yeah, fan fiction,
exactly.
>> [laughter]
>> So.
>> Well, you mentioned Michael a little
earlier about, you know, others, you
know, for our viewers and our listeners
out there that that you all can build
this. Um tell me a little bit more about
like how would you how did you build
this?
>> So, if you look in your hard drive, your
documents folder, you'll see a folder
called AI Story Builders, and there
here's where you'll see all the
uh folder for each story, and then
you'll see uh folders for each part of
the story. So, for example, I go into
chapters, and then you'll see folders
for the chapters, and then when I go in
there, you'll see um a text file for
each paragraph, and if you open up the
paragraph, you see something looks like
this, which is these are the characters
that are in that paragraph. This is the
prose of that paragraph, and here are
the embeddings uh
that represent that paragraph, and
the program literally
you know, opens these files up, reads
these embeddings, uses rag, uh which is
retrieval augmented generation, which
then uses a calculation called cosine
similarity to then determine what uh
paragraphs are related to each other,
and therefore it then creates the prompt
that is then sent to the AI.
Which leads us into the why I'm now
using knowledge this I'm still using
this, but I'm also now with rag using
knowledge graphs to
uh what I what in my opinion really
makes the this program 10 times better.
>> All right, so Michael, uh, tell me a
little bit more about this new graph
feature, uh, that you are building. I'm
pretty excited to see, um, you know,
what you've learned and how you're
growing and how you're improving this,
uh, this overall AI story builders.
>> The first thing I did was I,
uh, went into, you know, kind of I have
coded an app and I said, "Hey, look at
my existing AI story builder files and
create a, uh, knowledge graph
>> Okay.
>> and it created uh, something that looks
like this.
>> Yeah, so [clears throat] this is a JSON
file, um, that then is going to
aggregate all this, all this data into a
visualization.
>> Right. So, this is like the raw thing,
okay?
>> Yeah. Yeah.
>> However, I then vibe coded, uh, an app
that where I basically said, "Hey, can
you, um,
show me,
um,
let me just start here.
Uh, can you just visualize this graph?"
So, here, uh,
using the same Sherlock Holmes,
I
basically come up with something. I know
this is impossible to see, so don't
worry, I'm going to zoom in here, but
this is Sherlock Holmes, you know, that
knowledge graph I just showed you. This
is it visualized.
>> All right, so Michael, now you're, uh,
visualizing the adventures of Sherlock
Holmes. Uh, tell us a little bit more
about how, you know, an author would
utilize this to kind of keep, uh, the
story straight and understand where to
go next.
>> Right. So, this right here is just a
visualization of the graph just to get
an idea. Um, a graph, of course,
consists of entities, like these are the
characters, and then you have the edges,
which contain information that this
character interacts with this character,
um, and so on. And of course, um, we
also have everything broken down, um, by
paragraphs so that you can see what
characters interact [clears throat]
with, uh, which paragraph, uh, with with
that. So, what happens is is that this
graph is then exposed to the AI
as tools.
>> Mhm.
>> So, therefore, the AI is able to use
those tools to make um as many calls as
it needs to interrogate this graph.
So, what I've added is a chat window
that allows you to do things like
indicate the interactions between Watson
and Holmes in chapter one.
>> Yeah.
>> is it's able to come up with this um
very, very detailed uh response that I
would not what I normally would not be
able to surface using simple retrieval
augmented generation because with rag,
I'm limited by how much I can can
surface, you know, to the prompt.
Whereas here, the frontier models using
their advanced tool uh calling
capability, it can make as many calls as
it needs to that graph and therefore
surface everything that it needs. Um and
what I've also added is the way to go um
the other way, whereas you can say,
"Okay, you know, you've identified
issues. Please update um
you know, the story or update the
chapter to, you know, fix those issues."
>> Mhm.
>> So, therefore, I've exposed these tools
that allow it to update the files. So,
this
you know, so this capability is what's
being added. Um I should be releasing
that this weekend.
>> I love this cuz you're you're asking
questions to keep your story straight,
but then you're using this as context to
then write more story and have that be
more accurate as well.
>> Right.
>> Wow.
>> So, I look forward to the hopefully the
community uh will uh will like this
feature and then also can see how they
can adapt this technology to their own
applications by identifying uh things in
their applications which uh
um they can identify, you know, entities
and edges to create knowledge graphs and
then expose that knowledge graph to the
AI through tools that then allow their
applications to um you know, just go
deeper than just mere rag can can
currently provide.
>> Well, Michael, thank you so much for
sharing uh this application and sharing
your passion, being a Microsoft MVP. Uh
this is a fascinating uh project. And
so, I really appreciate you sharing it
today and thank you for being here on
MVP Unplugged.
>> Thank you for having me.
Ask follow-up questions or revisit key timestamps.
In this episode of MVP Unplugged, Michael showcases 'AI Story Builders', an open-source application he developed to assist writers in composing novels using AI. The tool leverages a unique database-like structure focusing on timelines, locations, and character attributes to guide AI in generating coherent narrative prose. Michael discusses the evolution of the software, the shift towards utilizing advanced models like Claude Opus, and the integration of Retrieval-Augmented Generation (RAG) and knowledge graphs to maintain narrative consistency in complex stories.
Videos recently processed by our community