One of the most frustrating things about

AI is that it's hard to know if someone

used it or not. Sure, there's all the

common tells like the M dashes and the

you're absolutely right and the it's not

X, it's Y's and all these types of

patterns, but it's generally pretty hard

to actually know if a given piece of

text or line of code was really written

with AI. And this has scared a lot of

people and for reasons that are good. If

it's impossible to know what text was

written with AI and what was written by

a human, the world will devolve into a

pile of slop really, really fast. The

European Union agrees and that's why

they just introduced the AI act which

has a specific goal of making it easier

to know and label AI generated media and

text and anthropic is complying with it.

Normally these types of watermarks are

things that only apply to media gen

things like images, videos, audio, etc.

because there's a lot of data there

that's relatively easy to sneak

watermarks into. But when it comes to

text, it's a lot harder. Which is why

it's pretty crazy that Anthropics

already updated their docs on this,

saying that they plan to do marking of

all AI generated content across all

platforms. Cloud models launched in the

EU on or after August 2nd, 2026 will

support machine readable markings at

launch. Generated text will carry

embedded watermarks and generated files

will include digitally signed Providence

metadata where supported. Oh boy. This

means our friends at Anthropic will be

marking not just the text they output,

but the code they output as well. This

is a very scary change and I have a

feeling it's not going to do what is

intended. As always, there's a lot of

layers to this one from the relatively

good intent of this EU act to the

questionable implementations we're going

to be seeing all across the field and

also the impact this will have on us

actually using the models. I'll talk

about all of that, the workarounds, and

more after a real quick break for

today's sponsor. Picking the right

database has never been more important,

but it's also never been harder. You can

go with industry standards like

Postgress for your application or

ClickHouse for your data warehouse. But

how do you pick the right thing if you

don't even know what you're building

yet? Wouldn't it be nice if there was a

great solution that combine the best

both of those? You know, like if

Postgress was just managed by

Clickhouse. Yeah, they're actually doing

this now. And it couldn't be cooler. You

get enterprisecale Postgress with all

the benefits of Clickhouse, too. The

Postgress database is what you should

use for your apps. It's super fast and

built on top of real NVMe drives on the

same machine resulting in absurd

performance up to 10 times faster for IO

heavy workloads. Where things get magic

is the ClickHouse integration. All of

the changes in Postgress get synced to

your ClickHouse. So you can do massive

analytics calls without slowing down

your production DB. And if you're

worried about latency, don't be. Your

data is replicated within seconds. And

if you want to access that analytics

data from the Postgress DB in your app,

you can with PG Click House. This would

have saved my butt when I was building

the T3 chat wrapped feature last year

and I had to query data from all sorts

of different places in order to pull it

together. Setup couldn't be easier. You

click start, you add a new database, you

get a connection string and now you're

good to go with a real Postgress

database with all the power of

ClickHouse behind it. Stop compromising

on your database today at

soy.link/clickhouse.

The best place to start on this chaos is

probably the official code that the EU

has put out. This is the code of

practice on transparency of AI generated

content. There's two different sections

in this code. There's section one for

the providers which is rules for marking

and detection of AI generated and

manipulated content and section two is

for deployers rules for labeling of deep

fakes and AI generated and manipulated

text. This code was drawn up by

independent experts in a

multistakeholder process facilitated by

the AI office of the EU. It helps

providers and deployers of generative AI

systems to comply with the AI acts

obligations for labeling and marking of

AI generated content. Even though

adherence to the code is voluntary, the

transparency requirements under article

50 are legal obligations. AI needs to be

deployed in ways where users know

they're talking to AI. That is totally

fair. Providers of AI systems generation

shall ensure the outputs of AI systems

are marked in a machine readable format

in detectable as artificially generated

or manipulated. The inclusion of text

content here is a bit concerning.

Providers shall ensure that their

technical solutions are effective,

interoperable, robust, and reliable as

far as this is technically feasible,

taking into account the specificities

and limitations of various types of

content, the costs of implementation,

and the generally acknowledged

state-of-the-art as may be reflected in

relevant technical standards. So, this

is the only section that is obligatory

for companies in this ruling. The rest

of the code is more suggestions. But

that one piece there, that uh section

two here, the providers must ensure that

outputs are marked in a machine readable

format. That's where this all starts to

become a show. There's an

interesting call at the end here of the

section two. This obligation shall not

apply to the extent the AI systems

perform an assistive function for

standard editing or do not substantially

alter the input data provided by the

deployer of semantics thereof or where

authorized by the law to detect,

prevent, investigate or prosecute

criminal offenses. What this means is if

you have an AI system that is like

editing or autocorrecting or things for

existing documents, images, whatever,

that does not need to have the same

level of like transparency and encoded

metadata proving that it was AI. So if I

use AI to like do a small correction

like to remove a zit from my face, that

doesn't need to be marked the same way.

If I'm writing an essay and I ask for

the agent to find little things that I

could improve in the essay or like clean

up the grammar and stuff, it doesn't

need to apply the same way there. But

the actual output from the LLM, it does.

Very interesting. And with that, we now

need to talk a little bit more about

watermarks. Going to start from a weird

point. So, hear me out. This image could

be AI generated. It might not be. It's

hard to know for sure. I don't know

where this image would have been taken,

but Maria posted it online on Twitter a

few days ago. In fact, when Maria posted

it, there was no marking here that said

it was generated with AI. So, clearly

this photo is real, right? Let me go a

step further. These two images are the

same, right? Like obviously these are

the exact same image with no differences

between them. What if I told you there

was not a single bite shared between

these two images? One of these is a PNG

and the other is a JPEG. This is what

makes watermarking for media both really

easy and really hard. That image we were

just looking at, it's not huge, but it's

not small either. It's 623

kilob. And the relationship between the

size of the file and what we see not

that simple. When you have this much

data in a file, there's a lot of room to

stuff things into it. a lot of which

won't even be perceivable to humans.

Let's say I want to add some special

encoding to make sure that I know this

image is the one I have here on my

computer. I could put it in the image

metadata and things like that, but I

want it to be harder to get rid of.

Let's just zoom into some of the like

noise and the gradient here and pick a

corner. Let's say let's just start from

the top left. Why not? I'll take a

random color from here. We'll take this

darker purple. And now I'm going to put

in a little encoding. I'm T3. So what

we'll do is every three dots I will fill

one with this darker color. One, two,

three. Do it. One, two, three. Do it.

One, two, three. Do it. Okay. I just

added a marking on this image in the top

left corner where I changed one of the

colors in specific pixels every three

pixels. So three are left normal, then

one I change it slightly. Can you tell

any difference in this image? If you

think you can, you're lying because the

compression in the video will not allow

such a thing. And this is me changing

the pixel to a civic color. So, we do

three normal, one change, three normal,

one changed, but I was changing it to a

fixed color. Imagine we get a little

sneakier with it. Imagine instead we

take the color of a given pixel here and

we knock it down or up one in any of the

different spectrums you have control of

with color because you have RGB values

or HSL and you can change any of these

at any point to or from anything. So we

could have a pattern where every four

pixels we bump one of these by one and

it wouldn't change anything visually for

any normal person. If we just go from 30

to 31 every four pixels, that is

relatively easy to encode in an image.

And it's in a way that will not be

noticeable to any human, but does

accurately with the actual pixels encode

real data. But in order for that type of

granular encoding to work where every

four pixels we go up or down one value,

we need to make sure we don't lose the

pixel values. And it turns out the

compression we use for media is directly

in opposition of the types of watermarks

we're talking about here. One of the

things most image compression algorithms

will do is they take areas where colors

are similar and they flatten them to be

the same so you don't have to encode as

much data. It'll take a grid every four

pixels and say okay these four have

these colors and these brightness

levels. Can we normalize that so we only

have to encode it once instead of four

times? That means that if we have this

pattern where every four pixels we

change the color slightly, that's going

to be flattened by pretty much every

compression algorithm ever. And all you

have to do to kill the watermark is turn

your PNG into a JPEG. It is seriously

that easy. And I have been able to evade

pretty much every single fancy type of

media detection algorithm for generated

AI image detection with something as

simple as reexporting as a JPEG or just

increasing the compression or scale the

image up and then back down. There's a

lot of little things you can do that

just destroy these watermarking

techniques. And the real reason is this

here. There is so much data available

here that leaving watermarks in it is

easy. But all the algorithms are

optimizing for what humans actually see.

And the difference between what data

exists and what humans perceive is the

problem here. With something like an

image, there's a wide range of data that

can be changed before a human notices a

difference. This means it's easy to

embed this type of metadata in an image,

but it's also hard to do it in a

resilient way because the compression

algorithms are working around this fact

that humans are bad at seeing the actual

ones and zeros in the bites. So there's

a lot of hacks we can do to compress the

files in order to save storage, but it

also in that process would kill a lot of

these hidden pieces of metadata. This is

why many people who post their AI

generated images on X get tagged because

they're just copy pasting the image

straight from chat GPT. And when you

copy from chat GPT or Gemini or wherever

else, they have these IDs in them.

Google's went as far as creating this

whole synth ID platform, which is a tool

that watermarks and identifies AI

generated content, mostly images, video

stuff that they've been talking about

forever. And no matter how hard I push

them, they will not let me in. And I

only get like three test runs a day

using the official Gemini site. Synth

ID's tagging is hilariously simple. They

apply this noise pattern on top of the

image. GBT image 2 has a similar one

where this pattern gets applied on top

and it looks that same deep fried way

that GPT images look. I'm not sure if

this is the entirety of how they do this

check and how they do these tests, but I

would guess it is a significant portion

of it. These patterns are what are being

used for a lot of the identification.

And again, the right compression, the

right application, the right layers

added to it, a very very slight filter

even, which is trivial for me to add. If

I just go here, filter, I can do like

the slightest sharpening pass where I

add like a 1% sharpen.

no perceivable difference. But now every

single bite in this image has been

re-encoded and any patterns that were

applied on top are fully destroyed. I

can even demo that trivially by copying

the noise pattern and then doing a quick

anything with this. If I apply that 1%

sharpen doesn't look any different, but

the bites that they're actually looking

for here have just changed entirely. If

I zoom in real deep and show you

see that you can literally see the

difference with like 2%. And I'm going

to apply. Now it's a very subtle change.

Zero is nothing. And then five. You can

see a slight change too. This is for

like a visible noise pattern. Any of

these are indetectable to humans. None

of them are indetectable to AI. Like any

one of those changes will entirely

destroy the patterns it's looking for

here. It's so easy to kill these things.

It's hilarious. It's even easier if

you're okay with a slightly blurriier

image or you do the lazy thing, which is

to resize the canvas and see how the

pattern got way blurrier. How do you

think that will work if it's also

applied at 1% opacity on top of

something else? Trivial to destroy these

patterns. There's no world in which they

will persist well across any of these

types of things, especially if you apply

them the way they're actually applied

IRL, which is something like this. You

have it on the image as like a 0.1%

opacity, maybe 1%. Yeah, you don't see

any difference here. And if I blur,

unblur, resize, or do anything like

that, the pattern gets destroyed very

fast. You could even do the stupidest

possible thing, which is to just take

that original image and do that with it.

I just change where every pixel lands

just by shortening it slightly. It's so

easy to kill these types of watermarks

just because there's so much wiggle room

in the data. When you have 623

kilobytes, I can delete random and

no human will notice, but the computers

no longer have access to that pattern.

All of this is to say this image is

clearly real and we would have no way to

know otherwise. So again, the problem is

this gap between the amount of data and

what humans perceive. And when you have

images that are millions of bits, it's

easy to flip things around in all

directions. But what happens when you go

from 623 kilobytes to I don't know 10

bytes. Each of these bytes is a letter

or a chunk for a word. Like once you're

responding with plain text that is

small, your ability to hide things in it

and also transform it goes down

massively. If I have a four pixel image

and I change one pixel, you can tell. If

I have a 4 million pixel image and I

change one pixel, you can't tell. If I

have a whole book and I change one word,

you're not going to be able to tell. But

if I have one sentence and I change a

word, you can absolutely tell. And

that's where this new text watermarking

comes in. I already read this section

about how Anthropic is going to be

following the rules. They simply call

out that the marks are going to work

everywhere that Claude is used, whether

that is through the API on the platform,

Claude, Claude Code, Claude Co-work, and

Claude tag. They also want to help us

detect Claude's marks, too. They'll

support users and other third parties to

detect Claude's marks as the code

requires. We'll share details in

forthcoming documentation. They are

actually planning on adding marking to

their old models as well, which is

interesting to me too. As AI generated

content becomes commonplace, greater

transparency and signals about where

content comes from can give people

useful context about the information

they consume. Support transparency and

comply with our legal obligations.

Anthropic is working to include machine

readable marks in content that Claude

generates. Embedded watermarks will

apply to all generated text and

Providence metadata will apply where

Claude supports processing files. This

will work with all the cloud platforms

too. Whether you're hosting on AWS,

Google Cloud or Foundry. So let's see

how they actually will be marking the

content. Cloud uses two complimentary

techniques to mark content generated and

processed by AI or by cloud

specifically. Watermarks embedded in

text and signed providence metadata

attached to files. First we have the

text embeds. When a supported cloud

model generates text, it weaves an

imperceptible watermark directly into

the text itself. You won't see it. It

doesn't change the meaning, quality, or

readability of clause responses. Because

the watermark is part of the text, it

will travel with the text when it's

copied and pasted elsewhere and may

persist through some editing.

Watermarking will be applied at the

model level, which means it will be

present no matter which cloud product or

surface the text comes from. This is a

very interesting call out specifically

may persist through some editing. Then

there's the signed providence metadata

which is that when they touch or create

things like SVGs, PGs or JPEGs, it will

provide signed providence metadata.

Metadata follows the coalition for

content providence and authenticity. The

CGPA open standard which is used across

the industry to record information about

content providence. If the metadata is

present, it signals that a file was

processed with claude and lets you

detect whether the file has been

tampered with. Thankfully, they call out

limitations underneath here. Detected

mark provides a signal that content was

processed by Claude, but it's not fully

conclusive. Detecting a Claude mark

tells you that content may have been

processed by Claude. It does not on its

own confirm the full providence of the

content. For example, Claude may not be

the original author because people often

use Claude to proofread, translate,

summarize, or convert files. The output

can carry a Claude mark even if the

underlying ideas, text, or data

originated from another source. The

content also may have been changed after

Claude processed it. Mark content may be

modified, exerted, or combined with

other materials after Claude is

processed. There's also the fact that a

lack of a detected mark doesn't mean the

content wasn't AI generated or

processed. There's also the fact that it

doesn't mean anything if it says no, we

didn't process that because there's so

many cases where it won't flag. Like if

the model generated text before this was

a thing, if the text has been heavily

edited, paraphrased, translated, or

mixed into other writing. If the passage

was too short, so there wasn't enough

text to encode a watermark. And for the

file stuff, if the file's been

converted, resaved, screenshotted, or

other things, again, the examples I was

giving here, then it won't work. And if

it was produced through a platform

feature or file type where a particular

marking type wasn't supported, very,

very fun. The harsh reality here is that

this is only ever going to catch the

lowest effort people because if you just

copy paste the text straight out of

Claude or you have a bot that auto

replies on Twitter for you and things

like that, then yeah, this will catch

those things. But if you put any effort

in or do even the most basic of

grammatical pass or changes to the text

that comes out, this will be trivial to

break. It is worth noting that other

labs will likely be doing this soon too.

Thoric called this out. Other labs will

be adding similar watermarking. It's

hard to identify AI generated text and

this gives people better tools to do it.

They'll be shipping a text detection API

as well that we can use ourselves. All

claw generated text will have this

embedded watermarking. For example, you

can check if a PR was generated by cloud

code. That said, it does have

limitations. He just called out the same

section we just read. We also got this

wonderful example of the watermark from

Nat here. Wow, the new claw text

watermarking is wild. Here we can see it

making a list of totally normal, not

suspicious text, but search engine

optimization, but if you look real

closely and you use those thinking

glasses,

Sam Alman sucks. Wow, it's really

subtle, isn't it? Back to the real world

for a sec though because Sean Godc wrote

yet another awesome piece about this

because when the act was officially

announced he had concerns. Article 50 is

the thing we're talking about here. If

LM providers want to do business in the

EU, they'll have to apply watermarks to

their outputs. Some hidden signatures

that can be used to identify AI content.

LM text watermarking is a fascinating

problem. Like the best engineering

problems, it's theoretically hard to

solve perfectly, but has multiple

partial solutions like synth ID and some

quiet unicode trickery from OpenAI and

anthropic. It'll be interesting to see

how these AI labs navigate these

trade-offs before the end of the year.

So, why is this so hard? It's easy to

watermark an image because digital

images contain lots of noise the human

eyes cannot see. For instance, you could

apply a watermark like these 20 pixels

in these exact spots will always share a

color. Text is much much harder. Unlike

images, text is very compressed already.

You cannot make any changes to a

sentence that a human won't notice with

one exception which we'll get to later.

So, how are you supposed to watermark

it? It's basically a textonography

problem concealing a secret code made

more difficult because the plain text

cannot be arbitrarily manipulated. Any

changes you make to apply the watermark

will compromise the quality of the

output. For instance, every fifth letter

is an E would be a good watermark, but

applied naively, it would make the AI

output full of typos. Could you just let

the model figure out how to fit the

watermark? Strong AI models are smart

enough to juggle this kind of

constraint, but it still consume

reasoning time that we better spend on

the user's problem and make the model

sound much less capable than it is. Yep,

sorry. It's steganography. My bad.

Steganography and stenography being so

similar of words is uh humans are not

good at text either. Do we even need

watermarks to detect AI content? If

you're enthropic and you're required to

be able to verify whether your models

produce a particular block of text,

can't you simply run the text through

each model, measuring as you go how

closely the models predicted tokens

match each token from the text? Not

really. The space of all possible cla

answers to a question is way larger than

the space of all possible watermarked

answers to a question. In other words,

you'd get too many false positives for

human text that reads like it was AI

written. I know a couple people,

especially YouTubers, who are hit really

hard with this where like the way they

naturally speak is similar to the LLM

because those models were trained on

them and now whenever they talk was

like, "Oh, that's AI generated, but in

reality, they just talk in the way that

the models liked and copied from them."

It's way more likely for a human to

accidentally write like Claude than it

is for a human to accidentally reproduce

a watermark. It would also be

prohibitively expensive to run every

anthropic model against a piece of text

in order to watermark it. The EU AI act

will eventually require labs like

Enthroic to offer free watermarking

services to every EU citizen. You

couldn't do that with a run this model

based approach. Yep. Apparently, synth

ID is being used for text. Like we need

more reasons for Gemini to be bad. When

an LLM generates text, it's generating a

series of tokens. At each step, the

model itself does not output a single

token, but instead outputs a full list

of 100,000 tokens in its vocabulary,

each annotated with the probability that

that token will be the next one. Tools

like chatbt and claude code will pick

semi-randomly from the most likely

output options in order to get their

outputs. The semi-random sampling

process can be influenced in a

detectable way. For instance, we could

choose a sampling strategy like we pick

the second most likely token, then the

first, then the second, then the first,

and so on to still produce high quality

output, but you'd be able to rerun the

model against the generated text and

verify that the pattern holds. But this

makes verification really expensive, and

any slight tweaks to the output would

break it. also changes to the input

because again the whole context window

is necessary for this. So synth ID does

this instead by assigning a score for

each token based on the previous tokens.

To apply this watermark, the model

adopts a sampling strategy like out of

the top five most likely tokens, pick

the ones with the top synth ID score.

The watermark can then be detected by

calculating the aggregate synth ID score

of a block of text. It's suspiciously

high. It's very likely to be AI

generated. This is similar to noticing a

lot of m dashes, but instead of a list

of keywords, it's subtle math

relationships between things that humans

can't identify as well. And since it's

trivial to sign the scores, it's trivial

to run the watermark detection. The

other strat people have been using are

these different space characters.

There's a lot of different ways to

encode a space in a thing that are not

just the normal space bar press. humans

will not see it. But if you encode these

strategically inside of something that

an AI model writes, you can detect which

ones are where and use this to get real

useful data. I remember Verscell a while

back was working on a feature where you

could edit content on a blog or

something via the Verscell toolbar and

the really clever strategy for linking

different pieces of text in the UI to

the official like CRM backing it for

your blog or whatever was to append

different white code text at the end of

each paragraph so that when you selected

that text in the UI and you made

changes, it'd be able to map it to a

different place in the actual database

simply by detecting which row it does

with an identifier using these different

unicode space characters to hide

messages inside of the UI. It's very

clever and really cool, but also like

easy to trim out, too. If you were to

use different space characters to apply

watermarks, it would be very cheap to

detect, but also easy to filter out too.

Cloud code is apparently doing this in

the past to tag suspicious requests from

Chinese users. In the last few years,

they've noticed that when they copy

blocks of text from chatb and paste in

the VS code, sometimes VS Code marks

some all of the spaces as unusual unic

code characters. Are openthropic using

homoglyphs as AI generated watermarks?

The author's not sure, but they're

definitely using them. Interesting. But

the problem here isn't whether or not

they're easy to add. It's how easy are

they to remove. Because all text

watermarks can be trivially removed. If

you use these Unicode homoglyphs, it's

trivial to just replace them with the

original character. Now you're out of

it. Watermark gone. If you have access

to even a relatively weak unwatermarked

LLM, you can strip out the synth ID

watermarks by asking the LLM to

paraphrase the text content. Because the

watermark is inherent to subtle

vocabulary choices, rewarding the

content will remove that watermark. You

could even do it by hand, although at

that point it's not really AI generated

content anymore. Since there will be

some kind of free public watermark

testing tool, you can just keep tweaking

until it comes back negative. Yep,

that's the other problem. If it's easy

for us to check if text is generated or

not, that works on both sides. Both for

us reading the text and also for people

trying to generate AI text and get it

out there looking like normal text.

Moreover, the AI act requires

watermarking techniques to be

interoperable as far as it is

technically feasible. That means AI

writers will have to publish their

watermarking process and potentially

even attempt to standardize on applying

the same kinds of watermarks. The author

does not see how this is compatible with

the kind of security by obscurity that

LM text watermarking depends on. You

can't be obscure in how you implement it

to hide it, but also give all of these

tools. Like there's a have your cake and

eat it too. By the way, the cake's

invisible and all made up type thing

here. Then we get to C2PA, which is

honestly one of the stronger things in

all of this, but not because of how it

makes it easy to verify AI content.

We'll come back to C2PA in just a

moment. The AI acting code of practice

talks a lot about digitally signed

media. The idea here is that you can

include an AI disclosure in the files

metadata itself. Ideally in a way that

cannot be tampered with like signing a

hash of the files contents. The sign

metadata process is basically the CGPA

content credentials. While you can

remove CGPA metadata, you can't fake it.

So a file with created by human metadata

can be trusted and a file with no

metadata at all can be held in

suspicion. Okay, this is the part that I

actually want to bring up here. The

thing that makes CGPA strong isn't that

it's flagging AI gen content. It's that

it provides standards to identify human

generated content. And that's the only

direction that makes sense here,

especially for media. It's a lot easier

to sign something as really generated by

the chip on your camera with a photo you

took than it is to detect all things

that weren't taken on cameras. And the

more ways we can find to sign human

generated media, the easier this gets.

Because I really do think long-term

that's the only solution that's going to

work. Sadly, the CTPA created by human

markers are not going to solve anything

in text. It only really applies to

files. In the words of the code of

practice, that's quote a data format

that supports attaching metadata like

audio, image, video, or containerized

text. The output of chat tools in most

of the output of AI agents is not

containerized text but plain old regular

text so it cannot be signed. What would

it even look like to sign a chatgbt

output? There's no artifact to pass

around. I think it's fascinating

question whether cloud code has to CGPA

sign any HTML files or PDFs it generates

for you seems kind of tricky to get

right. But in any case, the Act also

mandates some kind of actual

watermarking as well. And the harsh

conclusion here is that no matter what

happens, technical users will be able to

strip out the watermark at will. There

will be a plethora of tools for

nontechnical users that will do the same

thing. We already have a bunch of repos

that are doing watermark removal here.

Unicode text hygiene, statistical

rewrite hooks, and CGPA metadata removal

from PNG, JPEG, SVG, PDF, DOCEX, HTML,

and MD. This is a real repo that already

exists to remove watermarks from AI

generated things. It's an agent skill

and a standard Python script to strip

multi- vendor AI providence marks from

text and files for privacy and hygiene

on content that you own. They have a

skill you can install if you want to

remove AI marks from things that are

generated by the same agent in your

codebase, by the way. So, you can

literally ask Claude to remove the

watermark from Claude's outputs, which

is hilarious. Yeah. To conclude on my

end, the issue here is that this is a

really shitty cat and mouse game where

the mouse has all of the advantages.

It's way too easy to work around these

things and it's way way too hard to do

this type of watermarking in a way that

isn't trivial to bypass. Like I I just

cannot fathom any method where you can

watermark text that isn't either

incredibly expensive to detect or

incredibly cheap to work around. And

that catch 22 is pretty brutal and I

just don't ever see it working out. All

this will ever do is catch the lowest

effort spammers. People setting up bots

that just use Claude or other models to

spam people on Twitter and whatnot. And

they can all move to cheap openweight

models or add really basic checks in

front. But this is not going to stop

anything. This is one of those things

where it like sounds really good and to

normal people, to voters, to

politicians, to all the people involved

in making these processes, it sounds

like a really good idea. We should be

able to detect what text was generated

by models. We should be able to detect

when an image was edited by an LLM. Like

obviously we want these things, but we

literally cannot do it. We have to think

back on what our goal is here. Is our

goal to make it slightly easier to

notice when a random person copy pastes

an output from chat GPT? Is our goal to

make it harder for students in high

school to cheat on essays and have chat

GBT generate them? Or is our goal to get

in front of people using AI to do

destructive campaigns across society,

like a government using it to spread

propaganda and generate a bunch of

messaging that aligns with them? I think

this will help a little bit with the

stopping high school cheaters, but I

don't think it's going to help with

almost anything else here because any

medium or even like slightly higher than

zeroeffort attempt to mislead people

with AI generated stuff will not be

stopped at all by this. And even the

high schoolers are going to find

workarounds if we're being real here.

So, I just I don't see this being

useful. I see this being one of those

things that sounds really good,

especially to normal people that are

getting tired of AI slop destroying

everything around them, but I just don't

perceive this as a thing that will work

for any real use cases. And I think that

we'll get over this watermarking phase

relatively quickly. What we're going to

have to do is start verifying when

images are real instead of fake and

educating the public about the fact that

there are many things that look real and

aren't. Unlike the image I have on my

screen now, which is clearly a real

image of my face on a leg. But generally

speaking, the education side and

flagging of human content is what we

need long term. That's all I have to say

on this one. I honestly wish that the

effort of these really smart people was

put into things that would actually

work. I know this is a good faith effort

to try and make it easier for the public

to adjust to a world with more and more

AI generated stuff, but these efforts

just aren't going to work. Even a very

basic cursory understanding of

engineering and watermarking techniques

makes that obviously the case. How do

you guys feel though? Is this a net

positive or is it just a waste of time?

Curious how you all feel about it. And

until next time, peace nerds.