There's Something Odd About this Test

Harvard's Project Implicit has an interesting set of tests online, which are meant to show you where your implicit biases may be.  You may be strongly or moderately biased toward light skinned people, for example -- if their tests are accurate, 56% of people are.

However, there's something strange about the methodology that I can't quite place.  It's based on how fast you can process words and images.  I took the religion test, and it tells me that my biases work out this way:

Strongest positive bias:  Islam
Moderate positive bias:  Christianity/Judiasm (tie)
Less positive bias:  Hinduism

Now, while I make no bones about having some biases in this department, I'm pretty sure that isn't an accurate picture of how my biases actually shake out.  I can understand how this method would lead to a bad result on Hinduism:  of course there will be a processing delay there, because Hindu concepts aren't something my brain uses often.  I have to take the instant to remember what "karma" is before I can sort the word, and the concept is packed back in the back of the brain.  

The other religions have concepts I use regularly, so naturally they would come out on top.  If you were to ask me, though, I would think I had the strongest bias toward Christianity.  That suggests there is something odd about the method of determining biases; otherwise, I'd have to accept the existence of an unconscious bias in favor of Islam over my own faith.  I think I do have a stronger pro-Islam bias than many Americans, having known some very brave Muslims that I liked and admired, but still, that seems unlikely.

12 comments:

E Hines said...

One problem they have is in their questionnaire. Using the IAT sampler as my initial example, they ask for my highest degree, and my major in that highest degree. I have three Master's degrees--which one will I choose? And my preferred major wasn't on the list....

Strongest position on social issues? For me it varies with the categories they offer.

Essentially, their questionnaire begins with flawed assumptions.

In the practice test, I came out significantly preferring middle-aged adults; I would have expected little difference.

On registering, they continue to operate from poor assumptions (listed above) or from nebulous ones: what's a liberal or a conservative? They don't list my ethnicity; although the choices they include subsume mine, but that's pretty coarse.

Having gotten logged in, they shunted me to an "Event Likelihood Task," a test they used to assess my emotional state and how I view my emotional state. Apparently we don't get to pick our test.

Some of the tasks in the test tried to prejudice my answers by being all negative descriptions first, then all positive, then the offered answers; other tasks reversed that order; others still had no deliberate positive/negative prejudicing.

I score moderately more calm than anxious.

Part of the problem I have with this (aside from their assumptions) is their assumption of what will prejudice me. Until I understand their thinking on this, I don't buy the validity of their test outcomes--I can only accept that they're using these surveys to collect data that aren't yet ready for actual use or interpretation.

Eric Hines

Grim said...

If you go through 'demonstration' rather than registering, you can play with several of the tests.

Regarding their assumptions about what prejudices we might have, that is clearly a flaw in the model. For example, their weapon test is clearly meant to play off an emotional reaction against pictures of weapons. Weapons carry no negative associations for me, so it was merely a sorting game (one that turned out the opposite way, with me, than they clearly expected; but there was only a slight difference).

BillT said...

The test gives no results if you go slow -- Please try to go as fast as possible.
Expect to make a few mistakes because of going fast. That's OK.


Translation: "We expect you to make some mistakes, particularly if you're mildly dyslexic, but we're still going to count them as valid answers -- 'cuz that's *our* implied bias."

That was enough to make me pull the plug. I can get a more accurate personality reading by taking a "Which Dungeons and Dragons Character Are You?" quiz...

Grim said...

Hey, that was a really great quiz! It said I was a Human Paladin.

MikeD said...

And how very appropriate that is for you! I got Human Wizard.

BillT said...

But Paladin *was* human.

Bonus points for those of you who twig the reference...

bthun said...

"But Paladin *was* human.

Bonus points for those of you who twig the reference..."

Have Gun,
Will Amble, more or less

Grim said...

Bthun, you haven't stuck your head up in far too long.

Bill, an extra credit question like that, in this audience, is just curving the grade.

BillT said...

Lured Bthun away from the kegerator in the garage, didn't it?

bthun said...

Howdy Grim, Bill, kind ladies and gentlemen about The Hall...

My head has remained below the trench line these last several months mostly due to the fates, but Sir Grim's last observation in this piece, specifically the "I'm also increasingly thinking that we just need less social tolerance for vocal minorities." observation, might also account for why I've been in read only mode these past several months.

Well, that and concern that my opinions on our currently elected/appointed leadership, if vocalized with the vigor those opions arouse, will surely result in that $#@^^^ed black SUV, if not a drone leased courtesy of our Iranian brethren, being stationed on the Hun hovel once again.

One man with a six shooter, aka Paladin, is one thing. Hordes of Three Stooges like gub'ment bureaucrats rootin' in and around the hovel is quite another.

douglas said...

It is good to see you Bthun!

I took the religion test also, and also got Islam above Christianity and Judaism with Hindu below, clustered in the middle of the scale. I see the problem with this test largely being familiarity with the religious terms going in. If you're measuring reaction time, and correlating positive/negative outcomes on that, you need a control for familiarity. Just knowing the testers own religion or educational background isn't going to suffice.

They probably get government grants for this stuff, too. If we're going to keep expanding government, perhaps grant auditors would be a good place.

bthun said...

Howdy and thanks Douglas!