HomeVideos

Running AI Agents Locally = Safe...? Think Again

Now Playing

Running AI Agents Locally = Safe...? Think Again

Transcript

490 segments

0:00

Running AI on your machine does

0:01

not make it private.

0:02

Most people think

0:03

local means safe,

0:04

but if your machine is

0:04

connected to the internet,

0:06

you basically

0:06

bought your own house

0:07

and left every window open.

0:09

Here are the seven ways that

0:10

your private AI

0:11

is still leaking data and

0:13

exactly how to lock it down.

0:14

In my previous business,

0:15

I used to overcharge

0:16

people for this stuff

0:18

and made me just sick

0:19

of it, so I had to quit.

0:20

Now I'll just show you how to

0:21

build it yourself.

0:21

My last video on local AI setup

0:23

got a ton of comments

0:24

and the same questions just

0:25

kept coming up.

0:26

Is local still local

0:27

even when my machine

0:28

is connected to the internet?

0:30

Let's put it this way.

0:31

Using cloud AI like

0:32

ChatGPT or Claude

0:34

through their

0:34

website, their platform,

0:35

is like renting an apartment.

0:37

Your landlord has a

0:38

copy of your keys.

0:39

They can walk into your unit

0:40

whenever they feel the need to (ex. emergency..)

0:42

They might look

0:42

through your stuff.

0:43

You have to trust

0:44

that they won't.

0:45

Running AI locally is buying

0:46

your own house.

0:48

No more landlords,

0:49

no more shared keys.

0:50

That is real.

0:51

That matters.

0:52

But here's the

0:52

part most people skip.

0:53

Owning a house does

0:54

not mean it is secured.

0:56

If you leave the windows open,

0:57

the back door unlocked,

0:59

and your mail gets forwarded to

1:00

your old address,

1:01

you are exposed.

1:02

Different kind of

1:03

exposed than renting,

1:04

but you are still exposed.

1:05

And right now, there are three

1:06

levels of owning your house

1:08

that most people mix up.

1:10

Level one, local.

1:11

You own the house, but it's

1:13

still connected to the street.

1:14

Internet is on.

1:15

Updates are flowing.

1:16

Plugins are active.

1:17

This is where most of you are.

1:19

Level two, offline.

1:20

You own the house and you cut

1:21

the phone line.

1:22

No internet during use,

1:23

but you still go online

1:24

sometimes to

1:25

download new models

1:27

or get updates.

1:28

level three, air-gapped

1:30

You own the house and it is on

1:31

its own island.

1:33

No internet connection ever.

1:34

everything gets carried in by

1:35

hand on a USB drive.

1:37

Most people think that they are

1:38

at level three.

1:40

They are at level one and level

1:41

They are at level one and level

1:41

one has seven open windows.

1:43

Let me show you each one and

1:44

how you can close it.

1:47

The first one is the scariest

1:48

in January 2026 two months ago.

1:50

in January 2026 two months ago.

1:51

Security researchers scan the

1:53

internet and found over 175,000

1:56

local AI servers completely

1:58

exposed to the public internet.

2:00

That is very bad.

2:01

Not cloud servers,

2:03

personal

2:03

machines, people running

2:05

Olama, LM studio, VLLM, their

2:07

private AI was just wide open

2:10

for anyone on the

2:11

internet to use.

2:12

But how?

2:13

One setting.

2:14

when you run a local AI tool,

2:15

it creates a little

2:17

server on your computer.

2:18

Think of it like a door by

2:19

only opens inward.

2:22

Only you can use it.

2:23

That is called

2:24

binding to local host.

2:26

But a lot of people change one

2:27

setting so they can access

2:29

their AI from their phone or

2:30

another computer

2:31

on their Wi-Fi.

2:32

Now it faces the street.

2:33

And there's no lock on it.

2:35

Attackers can just walk in, run

2:36

prompts on your machine,

2:37

download your models, and in

2:39

some cases, run code directly

2:40

on your computer.

2:41

Here's what you

2:42

should do instead.

2:43

Keep your AI server

2:44

bound to local host.

2:45

If you need access

2:46

from other devices,

2:47

put a lock gate in front of it.

2:49

That means a reverse

2:50

proxy with a password.

2:52

and add a firewall rule that blocks

2:54

anyone outside

2:54

your home network.

2:56

Just five minutes, that window

2:57

is closed and sealed.

3:00

This one is happening right now.

3:01

In March 2026,

3:03

which is this week

3:03

Microsoft published a security

3:05

report about browser extensions

3:07

that were specifically built to

3:08

steal AI chat histories.

3:11

Here's how it works.

3:12

You are running your local AI

3:13

through a web

3:13

interface in your browser.

3:15

You think it is private because

3:16

the AI is just on your machine,

3:18

but you have browser

3:19

extensions installed.

3:20

Some of those extensions have

3:21

permission to read everything

3:23

on every page you visit.

3:24

That includes your local AI

3:26

chat window as well.

3:27

The extension reads your

3:28

prompts and your AI's response

3:29

right off the screen.

3:30

else's server in

3:32

the background.

3:33

You just never see it happen.

3:34

Think of it this way.

3:35

Your house is locked, but you

3:37

have a mail slot in your front

3:38

door and someone is just

3:39

sitting outside,

3:41

reaching through the slot and

3:42

reading every letter before you

3:44

even get a chance to.

3:46

You can create a separate

3:47

browser profile just for AI.

3:50

Zero extensions, there's

3:51

nothing installed.

3:52

Use the profile only when

3:54

you're working

3:54

with your local AI.

3:56

This takes just two minutes to

3:57

set up and it

3:58

completely kills this attack.

4:00

those first two, you can fix it

4:01

in five minutes.

4:02

The next ones are sneakier

4:04

because you didn't

4:05

actually set them up.

4:06

They were already running

4:07

before you even

4:08

installed anything.

4:10

Here's the one that makes

4:11

people's stomachs drop.

4:13

When your local AI runs, it

4:14

saves your conversations, chat

4:16

logs, prompt history, sometimes

4:18

full documents you fed into it.

4:20

Those files set in

4:21

folders on your computer.

4:22

It's pretty normal so far.

4:23

But most computers have

4:24

cloud sync

4:25

turned on by default.

4:27

Your iCloud, OneDrive, Google

4:29

Drive, Dropbox, these services

4:32

automatically copy your files

4:33

to their servers.

4:34

desktop and documents folder

4:37

without asking.

4:38

So your local private AI

4:40

conversations might already be

4:42

sitting on Apple servers or

4:43

Microsoft's or

4:44

Google's right now.

4:45

You bought your own house to

4:46

get away from the landlord, but

4:47

your filing cabinet is still

4:49

automatically faxing copies of

4:51

everything to your old landlord's office.

4:53

Create a dedicated folder for

4:56

all your AI work.

4:58

Call it something like AI

4:59

workspace or whatever.

5:01

Then go into the sync settings

5:03

and exclude that folder.

5:05

Make sure it never sinks.

5:06

Keep your prompts, your chat

5:08

logs, your RAG documents and

5:10

your model files

5:11

in that folder only.

5:12

Nothing in there should ever

5:13

touch the cloud.

5:16

This one is about the

5:16

AI models themselves.

5:18

When you download a model to

5:19

run locally, you're downloading

5:20

from your downloading a you're

5:21

downloading a file from the

5:22

Internet, usually from a site

5:23

called like Hugging Face.

5:25

It is like an app store for AI

5:27

models, open source models.

5:29

Most of those models are safe,

5:31

but researchers have found that

5:32

some models files have been

5:34

tempered with a bad model.

5:35

A bad model file can

5:36

do two things here.

5:38

First, it can contain hidden

5:39

sneaky instructions baked into

5:41

the model's template.

5:42

When you load it, those

5:43

instructions silently change

5:45

how the AI behaves

5:46

without you even knowing.

5:47

Second, and I

5:48

think this is worse.

5:49

Some model file formats can run

5:50

code on your computer the

5:52

moment you just opened them.

5:54

When you chat with the model,

5:55

when you load it, there are

5:56

real documented

5:57

vulnerabilities for this.

5:58

Think of it this way. You

5:59

bought your new lock.

6:00

for your front door

6:01

But you bought it from a

6:02

complete

6:02

stranger on the street.

6:04

And that stranger might have

6:05

kept a spare key.

6:06

Here's what you should do

6:07

instead. Now you're going to

6:09

only download models from

6:10

verified trusted publishers.

6:13

Learn to prefer the safetensor

6:14

file format over older formats.

6:16

Safetensors was specifically

6:17

designed to

6:18

prevent code execution.

6:20

And always check the file hash.

6:22

This is like a fingerprint that

6:23

proves the file has not been

6:25

tempered with during download.

6:29

Now this one is about your

6:30

operating system, your OS.

6:32

Windows, Mac OS, they all

6:34

collect data about how

6:35

you use your computer.

6:36

They usually just call it

6:37

diagnostic and telemetry

6:39

Some of are just basic

6:40

But some of it goes deeper than

6:42

you think. On Windows, crash

6:42

reports can include a snapshot

6:43

reports can include a snapshot

6:46

of what was in your computer's

6:47

memory at the

6:48

time of the crash.

6:49

If your AI was running and it

6:51

crashed, or your system

6:51

crashed, or your system

6:52

crashed, fragments of your

6:54

prompts could be

6:54

inside that crash report.

6:56

and that report

6:57

gets sent to Microsoft.

6:58

Windows also has a feature call

7:00

recall that takes regular

7:01

screenshots of your screen,

7:03

if that is on

7:04

you're using your local AI.

7:07

Microsoft is essentially taking

7:08

photos of your

7:09

private conversation.

7:11

So it's basically... So it's

7:11

basically your house has

7:12

cameras you did not even

7:14

install. And they are

7:15

pointed at your desk.

7:16

So here's what you should do

7:17

instead. Here's what you should

7:17

do instead. I'm going to need

7:18

you to go into your privacy

7:19

settings and

7:20

minimize your diagnostic.

7:22

Turn off the optional

7:23

telemetry. Turn off recall if

7:26

it is even on. On Mac OS, on

7:27

Mac OS, check your analytic

7:29

settings and avoid...

7:30

and always avoid

7:31

submitting crash reports during

7:33

sensitive AI session.

7:36

Here is the last one. I think this is less

7:37

technical, but I do think it

7:38

technical, but I do think it

7:39

can trip off a lot

7:40

of business owners.

7:41

If you handle

7:42

customer data, client files,

7:44

or anything with

7:44

personal information,

7:46

moving your AI to

7:47

local does not erase

7:49

your legal obligations.

7:51

In fact, I was also not aware

7:52

that this could

7:53

be this twisted.

7:54

GDPR still applies...

7:56

HIPPA still applies...

7:57

CCPA still applies...

7:58

The rules follow the data,

7:59

not where your computer sits.

8:01

in fact, here's the twist.

8:02

When you use Cloud AI,

8:03

the provider shares some of the

8:05

liability with you.

8:06

They have security team

8:07

compliance program,

8:09

encryption standards.

8:11

When you go to

8:11

the local though,

8:12

all of their

8:13

liability shifts to you.

8:15

You're now the entire

8:16

security department.

8:18

Every access control,

8:19

every audit log,

8:20

every encryption standard

8:22

is your responsibility.

8:24

So, owning your

8:24

house does not actually

8:26

change the building code.

8:27

You still have to follow them.

8:28

And now there's no building

8:29

management company

8:30

handling that for you.

8:31

There's no more Strata.

8:33

So, here's what I

8:33

think you could do.

8:34

I think you should

8:35

treat your local AI

8:36

as a production server,

8:38

almost, not a toy.

8:40

So, that's starting with

8:40

encrypting your drives,

8:42

setting up the access controls,

8:44

also knowing what

8:45

logs you have to keep

8:47

and what's in them.

8:48

if you handle regulated

8:49

sensitive data,

8:51

you gotta get clear on what

8:52

specific obligations are.

8:53

So, look them up.

8:55

This is not legal advice,

8:57

but it's a real conversation

8:58

you need to have.

9:01

Seven windows open.

9:02

Here's how to close all of them.

9:03

this weekend, 10 steps.

9:05

One, bind your AI

9:06

server to local host.

9:07

Never 0, 0, 0, 0.

9:09

Two, add two,

9:09

add a firewall rule

9:10

that blocks outside connection

9:12

to your AI port.

9:13

Three, create a dedicated

9:15

browser profile for AI,

9:16

no extensions.

9:18

Four, create a no sync folder

9:19

for all AI files that you have

9:21

excluded from iCloud, OneDrive,

9:24

and Dropbox, and much more.

9:26

Five, only download models from

9:28

verified publishers,

9:30

and learn to prefer

9:30

safetensor format.

9:32

Six, go into your OS

9:33

privacy settings,

9:34

minimize the diagnostics,

9:36

disable the optional telemetry.

9:37

Seven, you gotta

9:38

encrypt your hard drive,

9:40

and always have backup.

9:40

full disk encryption,

9:41

turn it on if it's not already.

9:43

Eight, turn off any

9:44

plug-ins for connectors

9:45

you are not actively

9:46

using in your AI tools.

9:47

Nine, nine, nine,

9:48

check your AI tool setting for

9:49

telemetry or analytic toggles.

9:50

Analytics toggles.

9:51

Opt out of everything you can.

9:52

Ten

9:53

if you handle client data,

9:54

write down your AI data policy.

9:56

The data processing document,

9:57

where data goes,

9:58

who has access,

9:59

how long you keep it,

10:01

and what exactly happens if

10:03

something leaks.

10:04

That's it. Ten steps.

10:05

One weekend, now your

10:06

house is fully locked.

10:07

Obviously, local AI is much

10:08

better than renting.

10:09

I believe that.

10:10

But local is not a

10:11

magic shield yet.

10:11

It is a better

10:12

starting position.

10:13

You still have to lock the

10:14

doors, close the windows,

10:15

and check who has the keys.

10:17

By the way, I put that full

10:18

ten-step checklist in a PDF.

10:19

You can download it for free.

10:20

There's no gatekeeping.

10:21

Link in the description.

10:22

You can just print it out.

10:26

Although I do feel like only two out of ten people

10:27

two out of 10 people are going

10:28

to actually do this,

10:29

but if this changed how you

10:31

think about your set up,

10:32

why don't you drop a comment

10:33

and tell me which

10:33

which windows surprised you

10:34

the most. And if there's

10:35

if there's any windows that you

10:36

think I may have missed,

10:38

please let me know in the

10:38

comments as well

10:39

I'm curious.

10:39

I read everyone.

10:40

I still have to get back to the

10:42

comments that I got

10:43

from the other videos.

10:44

I will do that today

10:45

I will see you in the next one

10:46

Bye

Interactive Summary

Running AI locally does not automatically guarantee privacy if the machine is connected to the internet. This video explains why local AI setups are often exposed to vulnerabilities and provides a 10-step guide to properly secure a local AI environment, emphasizing that owning the hardware requires taking full responsibility for security, data management, and legal compliance.

Suggested questions

4 ready-made prompts