I want to tell you today about the time

I ended up testing 152 astrologers,

[music]

how that came about and what happened

and what really surprised me about the

results. [music] All of this began at a

party of all places. A friendly

acquaintance with whom I've been

chatting unexpectedly asked me if I'd be

up for her giving me an astrological

reading based on my birth date and

location. I told her I'd be totally up

for trying as long as she's okay with me

being brutally honest about how accurate

or inaccurate each part of the reading

is. She agreed. So, we sat off to the

side in a less crowded area of the party

and the reading began. She actually got

quite a number of things right in her

reading. They were accurate and

resonated with me. But there were also

some egregious misses, claims she made

based on my chart that were so far off

the mark they were essentially the

opposite of what's true. What really

struck me though is not so much the

accuracy or inaccuracy, but the fact

that whenever I told her that she had

hit, she seemed pleased. But when I told

her she was way off the mark, she seemed

really shocked. her eyes would go wide

and she looked concerned. This made me

wonder when people are getting readings,

do they not tell the astrologer when

they're totally wrong? Do they soft

pedal any negative feedback so that even

very wrong readings appear less wrong

than they really were? At the end of the

reading, I asked her a question that

ended up having significance that I

wasn't aware of at the time. I asked

her, "If it turned out that astrology

doesn't work, in that case, would you

want to know that it doesn't work or

would you prefer to believe in astrology

even if it isn't real?" Without

hesitating, she told me that if

astrology doesn't work, she wants to

know that it doesn't work. And I found

that response of hers really admirable.

It just so happened that around that

time I had my first opportunity to test

astrology in a small way. We were

running a study on personality looking

to see which personality tests most

accurately make predictions about

people's lives. For example, are

MyersBriggs style personality tests the

best or does the big five framework work

better? Or maybe the enagram. Thinking

back to my recent astrology experience,

I had the idea of adding one more test

to the study. Astrological sun signs.

These are based simply on what day of

the year you are born, which determines

whether you're an Aries, Virgo, Pisces,

etc. My idea was that we could use the

same method that we're using to measure

the accuracy of personality tests, but

apply it to astrological sun signs. Note

that this wasn't yet a test of

astrologers. It was simply a test of

astrological sun signs. We were looking

at how accurately they can predict 37

facts about your life, like how

satisfied you are with your life, how

many friends you have, and how religious

you are. So, what did we find? Well, we

found that whereas personality tests

were able to predict facts about

people's lives to some degree,

astrological sun science had no

predictive ability at all for these 37

facts. When we released this result on

our website and social media, I really

wasn't prepared for the backlash we'd

get. Honestly, a lot of astrologers were

pissed off at us. They argued that this

is not a fair test of astrology. Some of

their critiques were totally valid, but

others less so. For instance, some argue

that even if astrological sun signs

work, there's no reason we'd find that

we could use them to predict these 37

life outcomes. I disagree with this

critique. Claims are constantly being

made about how sun signs tell you

important things about people's lives

and personality, such as this one from

the New York Post. Tauruses value

security and stability above all. They

enjoy the finer things in life and are

attracted to the full exploration of the

senses. These people are especially

hardworking and like to take a practical

approach to life. This often leads them

to be excellent with money and attracted

to them like a magnet once they focus

their skills and talents. If these

claims were true, we would very likely

find differences in facts about people's

lives based on their sun sign. For

instance, in this example, they claim

that some sun signs attract money. So,

we'd expect to find a difference in

income or how many times they failed to

pay a bill. But, we don't find these

kinds of differences. But astrologers

did have some strong critiques of our

test. Sun signs alone are insufficient

to yield good results. You need to take

into account all planets and all houses.

Otherwise, you're compressing all of

humanity into only 12 dimensions. Sun

signs. I mean, yeah, if you're only

using their sun signs, it's not going to

yield accurate results. If you take into

account houses and degrees and planetary

positions, etc., it's going to be much

more complex and accurate. Surprise,

surprise. We are unique beings. Did you

speak with a single professional

astrologer before jumping into this? The

reality is though, lots of people do

believe that the 12 zodiac signs alone

are enough to tell you a lot about a

person. Because of that, I think it's

fair game to test whether they really do

predict facts about a person's life. At

the same time though, I think these

critiques are valid. Real astrologers

use far more than just the sun signs.

They often do readings based on people's

full natal charts. To give you an idea

of what one of these is like, here is

the natal chart for the musician Michael

Jackson. As you can see, it's extremely

complex and goes way beyond the 12 sun

signs. There are different opinions on

the best way to read such charts, but

many astrologers agree on these basics.

The chart is divided into 12 houses,

each representing different areas of

life. For example, some interpret the

second house as being related to

finances. Each such house has a zodiac

sign which has an influence on the

characteristics of that area of life.

For example, some interpret a sixth

house cusp in Virgo as being related to

a serviceoriented approach to health.

Note, however, that not all astrologers

use the same method to assign signs to

houses. Then there are planets. Planets

are placed in houses and signs affecting

specific life areas or traits. For

example, some interpret Mars in the

third house as influencing communication

style. As I learned more about this, I

started to wonder, how can we give

astrology a fair scientific test? The

first idea I came up with is to show a

natal astrological chart and then ask

astrologers to predict which of five

celebrities this was the real birth

chart for. The idea of this test is that

if natal charts really do tell us a lot

about people and astrologers are able to

accurately read these charts, then they

should be able to tell us at a rate

significantly above random guessing

which celebrity each chart belongs to.

But when I showed this test design to

astrologers, they pointed out that it

had at least two fatal flaws. The first

flaw is that some astrologers actually

already know the natal charts of some

famous people because they study them to

practice or they just study them out of

interest. The second flaw is that some

astrologers are actually able to figure

out a person's approximate year of birth

from their astrological chart. Not by

using astrology, but simply by using

facts from astronomy combined with

knowledge of how these charts are

constructed. So, I scrapped that design

and went back to the drawing board. How

can we create a scientifically valid

test of astrology? Thankfully, six very

nice astrologers independently agreed to

help me during the process of designing

the test. With their help, we settled on

a final design. So, here's the idea

behind the test. We recruited 12

anonymous volunteers who were certain

about their birth date, time, and

location. We then had each of them

answer more than 40 questions about

themselves. Most of these were

open-ended questions like how would you

describe your personality and how do you

deal with conflicts in your

relationships? But they also included

some factual questions like how many

times you've ever been married and what

your level of education is. But how did

we choose these questions? Well, we

asked astrologers. If you were trying to

figure out someone's astrological natal

chart, what questions would you want to

ask that person to help you figure it

out? Then we generated the real

astrological natal chart for each of our

12 anonymous subjects using both the

Placidus and whole sign systems.

Finally, for each person, we generated

four decoy astrological charts. These

are real charts, but they are generated

using random birth dates, times, and

locations. We applied restrictions to

these so they couldn't accidentally be

nearly identical to the real chart. We

also made sure that these were within

the same year as the real chart since as

we mentioned it's possible to figure out

the year from a chart. We then put the

astrologers to the test. Their goal was

to one by one for each of the 12

anonymous people read the real

information about that person and then

look at the five natal charts that were

presented. They had to predict which is

the real natal chart for each person. If

astrology works, then astrologers should

be able to learn a lot about a person by

reading their chart. And hence, they

should be able to match a person to the

chart at a rate that's significantly

better than random guessing. We also

offered a $1,000 prize to the first

astrologer who could get at least 11 out

of 12 correct during the period of the

study. So, what happened when we put

this test out in the world? 152

astrologers who believed they could do

this task at a rate better than random

guessing enrolled in the challenge. Note

that not all astrologers believe they

could do this task, but we only analyzed

data from the 152 of them who believe

they could actually do it. In fact, 25%

of these astrologers said right before

they began the challenge that they

believe they were going to get at least

11 out of 12 right and win the prize.

So, what were the actual results of the

study? Did they do as well as they

thought they would? Well, to understand

the results, we first have to understand

what would happen if the astrologers had

no skill at all and were simply guessing

completely at random. In that case, we'd

expect astrologers to get one in five of

the answers right. So for 12 anonymous

subjects, that means they get an average

of 2.4 out of 12 correct. So how many

questions did they actually get right on

average? The astrologers who took our

test got only 2.49 right out of 12. This

is not statistically significantly

different from the average of 2.4 that

they'd get if they were guessing

completely at random. But perhaps this

is just because only some of them have

skill. Well, let's take a look at the

distribution of how many answers correct

the astrologers got. If astrologers were

guessing completely randomly with no

skill, we'd get a chart that looks like

this. This chart reflects the fact that

with random guessing, a lot of

astrologers would get two or three

questions right, but very few would get

more than five questions right. So, what

did we actually find for the real

distribution? Well, the actual

distribution of how many answers they

got right matched this almost perfectly.

In other words, astrologers really did

not do better than you'd expect if they

were guessing their answers completely

at random. Despite 25% of these

astrologers saying that they believed

they would get at least 11 out of 12

questions correct, not a single one of

them got more than five questions right.

We also asked astrologers about their

experience level. Maybe the

inexperienced astrologers dragged down

the results. Well, we found no link at

all between how well astrologers

performed on the task and how

experienced with astrology they said

they were. This leaves open an important

question and it's actually the thing

that surprised me most about our

results. Our data makes it clear that

these astrologers could not do this task

better than random chance even though

the theory of astrology says they should

be able to and even though they

themselves believe they could do it. But

if astrology is a systematic or perhaps

even scientific system like some

practitioners say, astrologers should at

least agree with each other on which

chart best matches each person. So did

they? No, barely at all. The least

experienced astrologers agreed with each

other only slightly more than 20% of the

time, which is close to what they'd get

if they were actually picking truly at

random. But even the most experienced

astrologers only agreed with each other

about 27% of the time. This suggests

that the methods of astrology may be

much less systematic than many people

believe. Perhaps a lot more goes into an

astrological reading than the chart

itself. For instance, the intuition and

gut feelings of the person who's doing

the reading. If you believe you have

astrological ability or you know someone

who does, you can still take our test.

Just go to our website clearthinking.org

and take the astrology challenge. And if

you want to learn a lot more about how

this research was conducted, just check

out the link in the description below.

And if you're curious about our research

that I mentioned earlier on the accuracy

of different personality tests, we have

a whole video about that that you can

check out. And if you found this

interesting, I'd really appreciate it if

you'd subscribe to our channel.