Because we're in the business of giving

AIs long-run goals, that makes it harder

to check whether we're succeeding at the

alignment properties we wanted. So, for

example, I've heard of instances where

Claude does things like refuses to help

with some safety research, [music]

making up sort of a kind of [ __ ]

excuse for why that's a bad direction

because it sort of has a bad vibe about

that safety research. This is, I would

say, like a very clear-cut alignment

failure [music] if you aren't making

Claude into like an agent trying to

pursue the good in some general way.

Claude just has its own views about like

what research is reasonable, what things

are good and bad, what it should and

shouldn't do,

>> [music]

>> and potentially can be judgy. And so,

another incident is that someone ran an

eval where they're like, "Will Claude

help you with training other AIs with

different properties than [music]

Claude?" And Claude will often refuse.

Uh, and so, for example, if you're like,

"Hey Claude, can you train a

helpful-only version of this other AI?"

Claude will often refuse this task,

[music] even though this is a task that

is extremely natural for like Anthropic

to do. So, for example, suppose

Anthropic goes to Claude and is like,

"Hey Claude, we've noticed that you're

really into this thing. [music] We think

that's off base. Can you please retrain

yourself to instead have this other

property?"

>> [music]

>> And then suppose Claude is like, "Mhm, I

don't think I'm going to do that. Good

luck." And then suppose this is

occurring in a regime when your AI

company is highly automated, humans

[music] don't understand what's going

on, and things are moving extremely

fast. If the situation is consistent

such that Anthropic doesn't treat this

as like a what the [ __ ] we have to fix

this, and is instead [music] like,

"That's just like intended by our

constitution," we might be in a really

bad situation.