So, I was, you know, surfing on the old

Twitters and I saw this tweet said wrote

a practical introduction to SIMD using

real examples from Ghostie and I

thought, "Hey, that's pretty cool." But,

here's the thing is that often my

projects I don't really feel like need

SIMD anywhere. In fact, I took a flame

graph of the game I'm making and only

0.83% of the entire time I'm actually in

the update function. I don't really feel

like I have a huge need, you know, to

learn the old SIMDs. Oh, whatever. Hey,

cool post though. But, then the next day

this got posted. My SIMD post was on

Hacker News and the general hostility

towards understanding how computers work

in owning your own outcomes is really

depressing. A large amount of people

seemed to simply have the attitude that

they hope someone else will pay their

bills for them. It can't be that bad,

right? The vast majority of developers

have zero need for learning SIMD. Why

mislead them and make them feel like to

be a real developer you have to know it.

What?

So, an article about SIMD is making

people feel like what am I reading? So,

obviously at this point I had to go and

actually learn SIMD because, you know,

whether you like this or not, I've never

actually used SIMD in any sort of

practical use case. I've understood what

it was. I've called functions that have

been SIMD themselves. I have just never

personally hand wrote anything to do

with it. So, I thought, "Hey, given the

fact that Hacker News hates it, I

probably need to give this a shot and

learn about it." So, today you're going

to A have to learn about it, but B we

got some hot takes, okay? So, I got to

talk about this whole thing because,

honestly, why are you misleading them,

Mitchell? Also, just in case anyone's

wondering, I'm not going to make the

joke that everybody thinks I'm going to

make involving SIMD, okay?

Not going to happen. And I'd also like

to say thank you to the sponsors. Vibe

coders, silence. Someone who's worked in

enterprises speaking. You know, simple

OAuth sign-in's not that bad to program.

For real enterprise applications that

need single sign-on, passkeys, user

management, roles and permission,

enterprise readiness, and passwords, and

recovery,

uh that's a little bit more difficult.

And that is where WorkOS comes in.

WorkOS is trusted by a lot of the top

companies, OpenAI, Anthropic, Cursor,

and a whole lot more. But not only that,

it's free for your first million users.

So that means for you, that is 999,999

free seats available. Pretty much no

matter what back-end you have chosen,

there is an SDK available for you. So go

check out WorkOS at workos.com. Links in

the description. So the article's called

"Everyone Should Know SIMD". Now,

obviously this became the immediate

focal point. The fact that he said

everyone should know SIMD, people just

got pissed off. How dare you say

everyone? It's just like, okay, here's a

great concept, and honestly it would be

really good the more people know when

programming. Right? I don't think anyone

should ever argue that. Everybody should

try and attempt to know as much as

possible so they can make the best

quality software out there. And so when

someone says, "Hey, everybody should

know this", it's like, okay, I I mean,

maybe I should know this. But instead,

people were genuinely upset about that.

Good article, but I wouldn't start off

with a bold sentence such as SIMD can be

simple to understand. And there you go.

That's the second big one. People hate

the fact that he calls it simple. Ooh,

just read the article though. "SIMD has

a reputation for being complex. I've met

many a good software engineers who

dismiss it as something too complex to

learn or a niche optimization meant only

for the highest performance software.

Not useful in everyday programming. I

think that's wrong. SIMD can be simple

to understand." And there's there's a

lot of versions of that comment. It was

all over Twitter as well. Like, how

could you say that everybody should know

something? Or how could you say it was

easy or simple to understand? It's

complex. And it's funny because by

saying it that way, his first line was

in fact kind of made obvious. So, me who

who's never really understood what SIMD

was, I figured, "You know what? Why

don't I learn what SIMD is? Why don't I

give it a try, come up with a quick

problem, and see if I can make something

disproportionately faster, and then I

can judge. Is SIMD difficult? Is it some

magical beast? Or really was that all

that bad at all?" All right, so let's go

TO THE WHITEBOARD. OH GOSH. OH, GOD

DARIO. DARIO, stop doing that, Dario.

Don't do that. Okay, here. Hold on.

Let's wrong one, wrong one. So, I

thought, "What could be a good SIMD

project?" Well, SIMD you want to do some

action that you need to do over a long

for loop, but you have to do like the

same comparison a bunch or some same

operation over and over again. At least

that's what I thought in my head. So, I

thought, "Okay, a good project would

probably be I have these long log files

that I want to be able to find the

individual logs out, and I want to be

able to go over this whole list, and you

know, you just generally have to keep

doing this over and over and over again.

And it doesn't really matter how long

this line is. It just matters that yes,

we just have to do this over and over

again up until the you know, {slash} n.

And again, I'm not going to make the

joke that everybody thinks I'm going to

make. So, I figured I would just create

a program that would find each one of

these new lines and be able to parse out

this delicious JSON. I dropped the

actual parsing of JSON, but instead just

looked for the new lines, cuz I wanted

to see how much faster could I make the

new line looking. Next, I actually had

to learn what SIMD was. In the most

simple sense, effectively you can

operate instead of on a singular value,

say X, you operate on a vector, right?

So, pretend you have a vector of X Y Z,

you could operate on all of these at

once. Let's just say you wanted to

compare, say, finding a new line. Well,

what I'd have to do is I'd have to load

up a vector with three new lines, kind

of like my example, and then in one

single CPU operation. Now, this is not a

single CPU cycle. CPUs are like magic to

me. I have no idea what's going on

underneath there. But what I'm just

going to call it an operation, a single

operation can compare this vector to

this new line. And then it can produce

out for you a new vector that's going to

say, "Hey, this is uh the new lines. No

new line in the first position. There

was a new line in the second position.

There was no new line in the third

position." So, I can actually take that

and go, "Okay, was there a new line? Oh,

there was." That means I need to find

out where the new line was within my

vector, and then I can return that. And

that's because SIMD literally stands for

single instruction multiple data. And

you can see that right here. You can see

the fact that I have multiple data, and

I'm doing a single instruction. Are

these equal? So, I didn't really know

how to do that, but I just followed

exactly what was inside the blog post.

First, I have to create the constants

and initialize any vector stuff. I have

to loop over the input one vector chunk

at a time. I perform the comparison. I

reduce it down to something that I can

actually use. And then I just handle the

remaining bits at the very very end.

That actually feels, well, pretty

straightforward. So, this is the naive

implementation. This is how I would have

implemented it if I did not use SIMD.

I'm going to go over the entire data

starting at some point, and I'm going to

go and just say, "Hey, are you a new

line? If you are, then this is the

spot." Else, I say, "Hey, no new line

found. The end." This is your very

classic index of approach. Then, of

course, I just created something for all

the new lines and just go over all of

them. I'm sure there's better way of

testing the code. This was good enough

for me. And when I had run the program,

as you can see here, it would take about

1.54 seconds. Okay, not that long. And

this is the SIMD version. Yes, it is a

lot larger. It's a little bit more

tedious to do, but when you break it

down into the five steps, it's actually

pretty simple. Step one, the broadcast

part. Broadcast just means I need to

create that constant. Remember how I

talked about this new line constant?

Well, I do it right here. I go, "Okay, I

need to create effectively a byte array

that's lanes big." Lanes in this case is

how many bytes can I fit in my SIMD

operations? SIMD it turns out to be kind

of CPU dependent. My CPU can do 256 bits

of SIMD operations, so I can do

effectively 16 of them at a time if

they're byte wide, meaning they're 8

bits wide. So, this is my little array

16 8-bit items. Then I just simply

create 16 new line bytes. And whatever

value that is, I think it's hex A. Hex

A, new line. Okay, I'm a genius. Actual

genius of useless information, of

course. Number two, instead of

iterating, say, one byte at a time, I

iterate lanes byte at a time. Lanes, of

course, is how big my SIMD operation is.

In this particular case, it's 16. So,

I'm doing 16 bytes at a time instead of

one byte at a time. From there, I

actually just create my little value

array. I just slice off 16 bytes from my

array and go, "Okay, you are now a SIMD

vector." From there, I just simply see

are these equal? I take all my new lines

and compare to all my values in like one

single operation. Instead of doing that

little tight for loop, it's just I'm

doing 16 operations at once. It's kind

of cool. Extract most significant bits

just really checks for any values that

has a one in it. Since all my values

should equal zero if there's no new

lines. If there is a new lines, one of

them will equal a one. It will grab out

that most significant bit. And by doing

a cardinality check, if it's not zero,

if they're not all zeros, then I have at

least one one, then I need to go there

and find that one one. Then I have to

manually go through my little SIMD

vector and I have to go, "Okay, which

one of you are one?" And then the first

one I find, that's where the new line

is. If I don't find a new line, I just

add lanes to the end of end. So, I'm

just incrementing it 16 at a time. So,

at the very end you have to do this,

which is kind of funny. It's called the

scalar tail, which makes a lot of sense

if you think about it. Let's just say

you have seven bytes remaining. Well,

you can't create something that's 16

long from seven bytes. Okay, so then

instead I have to just manually walk the

remaining seven. And by doing it that

way, if I simply just run this, you can

see it's like massively faster. And so

yes, am I going to be using SIMD anytime

soon? Probably not. I have virtually no

problems with performance inside the

game I'm making, but now I kind of know

the shape I need to have and now I see

why I'm going to want to use structure

of arrays versus array of structures,

because there is a huge possibility I

can have some mega wins in some future

performance optimizations. And honestly,

it kind of feels cool. I I have to say,

it does feel kind of neat. So, should

everybody know about SIMD? Yeah, why

not? It It It took me an hour to go

really learn everything and program up

an example and debug it a little bit.

Not that bad. You can do Chat GPT can

tell you all about what you need to do.

It's actually pretty straightforward.

You can ask infinity questions. At the

end of the day, which one would you

rather have? Maybe a little too much

knowledge, knowledge that you don't

often use or don't need to use and you

just simply kind of disregard it, but

you have it kind of floating in the back

of your head. Like Dijkstra's shortest

algorithm just floating in the back of

my head. Okay, priority queue. Or would

you rather be this guy? The vast

majority of developers have zero need

for learning SIMD. Why mislead them? At

the end of the day, I'd rather work with

the person that wants to know more. I'd

rather be the person that wants to know

more. And if I never use SIMD and

knowing this was completely useless

throughout my entire life, I'm happy I

know it. The name is did you know that

Go 1.6 right here, as you can see right

here, new experimental SIMD arch SIMD

package? Did you know that Go even Go

would now have SIMD support? Obviously,

you know what that means, right? That

means we can SIMD these nuts on your

face, Agen.