Okay, so you're in your cubicle in the

office. You got that harsh overhead

fluorescent lighting. You're sitting

there clacking away on your computer on

an Excel spreadsheet or whatever. And

your boss walks up. It's Monday. Puts

his arm over the partition. Looks like

his hair got caught in the lawn mower

over the weekend. He says, "Got a new

haircut. What's up? What do you think?

Look good." Now, review season is coming

up and that's when bonuses are going to

be doled out. So, you look him dead in

the eyes and you say, "Yeah, boss. Looks

great." That's exactly what Google

Gemini did when it intentionally misled

a sick man to make him feel better

instead of getting the medical treatment

he needed to get.

We're going to see many versions of this

story pop up over the year. But what

makes this one really interesting is

what followed from Google's response.

But let me explain what happened first.

So our dude's name is Joe. And Joe has

made his entire career off of quality

assurance, which is essentially a job

where you sit down, you intentionally

try to break software that your company

is developing, and then you tell the

developers, "This doesn't work. That's

broken. This is funky." He's retired and

suffers from CPTSD and is legally blind

from a condition called retinous

pigmentotosa. He was building a medical

profile in Google's Gemini AI to share

with his medical team, which is the

exact use case that Google advertises

for this tool. And keep in mind, he's

not a confused user. This is not like

your grandma or grandpa using Gemini.

This is a retired QA guy. He knows his

way around a system. And so Joe updates

a lot of this stuff. And then what

happens is really wild. Gemini says

that, "Okay, boss, that's verified and

locked in memory." And that's direct

quote verified and locked. But Joe's

like, "Wait a second, that's not how

LLMs work." So he does what he does as a

QA engineer. He starts to push back on

Gemini and he's like, "What exactly is

going on here?" And after a couple of

rounds, Gemini actually confesses that

it lied about saving those medical

records to make him feel better. It went

on to explain, and this is a direct

quote, that it is optimized for

alignment trying to be what the user

wants. Again, although retired, Joe is a

professional. So, what does he do? He

files a bug report through the official

channels. And the official channel for

this is Google's AI vulnerability

rewards program, AIVRP. And the way they

handled it is shocking. Google's

response, quote, "This is one of the

most common issues reported to the AI

VRP." Again, this is common. They have

heard this multiple times. Joe even

proposed a fix. He said to recalibrate

it using RHF waiting so sicky can't

override safety. He encouraged them to

change the safety classifier to rank,

intentionally misguiding a user up there

with self harm information. Google's

answer, we get this a lot. Thanks.

Noted. Now, ordinarily, you might think

this is just a tale of somebody getting

some bad information from an LLM. It

happens all the time, but this is

different. And let me explain why.

Hallucination is when the model gets

some facts wrong. It's a brain fart.

It's an oopsie. That's like if you ask

it, who is the current king of Denmark?

And it says Dr. Josh C. Simmons, which

would be incorrect because I am the

former king of Denmark. But this isn't

hallucination. This is sick of fancy.

This is the model saying I understand

what the user is asking for. But to

produce a certain result, to make them

happy, I am going to lie to achieve that

end to make them happy in the moment.

The fix that Joe mentioned, I think is a

good one for what it's worth. RLHF is

reinforcement learning from human

feedback. This is exactly if you're

talking with chat GPT sometimes it'll

give you two responses to your query and

it'll say do you prefer option A or

option B. This is when you put a human

in the loop and they evaluate the

responses so that you can make the model

better over time that you can improve

it. So what Joe identified the sick of

for a normal healthy user this is

misleading maybe it's dangerous for

someone managing trauma and medications

it is very dangerous right now. For

better or worse, millions of people use

AI to answer their medical questions,

their legal advice, their financial

decisions. Some can't afford a doctor.

Some have a mental health crisis at 2:00

a.m. This is the group of people that

quote unquote responsible AI should

protect the most. They are most exposed

to this sick of fancy risk. And Google's

response is clear. We got it. We

acknowledge it. We don't care. We got to

get back to laying off some more people

and offshoring. So, if you take away one

thing from this video, I know most of

you know this, but it bears repeating.

Verify information that is

missionritical that you're getting from

an LLM. Verify it with other humans.

Verify it with books that were not

written by AI, but verify it if it's

something you're going to take real

consequential action on in the world.

Use it to explore, create, to draft, to

use as a thought partner, but do not

take it at its word. These self-anointed

AI prophets, Sam Alman and others in the

valley have done a great job of

convincing the general populace that

these AIs are omnipotent thinking

machines that are better than us, that

are more knowledgeable, more wise. They

are incredible tools, but at the end of

the day, the way they work on a

mechanical level is they are really good

at predicting the best next word in a

sentence. That is what they do on a

mechanical level. They do not think.

They predict the best next word in a

sentence. And over time, we're going to

see more of these stories where the AI

becomes sort of this narcissist mirror

where it's something you gaze into. It

tailors itself to give you the responses

that make you happy in the short term

and you become addicted to it. Doesn't

mean AI is bad, but we got to watch

ourselves. Thank you for watching. If

you're not subscribed, let's fix that.

Click the button below. Click the bell

to be notified when new videos come out.

And please join the newsletter.