DOJ Thinks Firefighter Math is Racist

Now, here's a subject I do know something about: firefighter examinations in North Carolina. I've taken several of the certification exams in the firefighter series as part of my training as a volunteer; I'm also a fully certified Technical Rescuer in rope rescue, water rescue (Swiftwater and still water), and wilderness rescue, and those certifications come out of the same department (the Department of Insurance, whose elected head is ex officio State Fire Marshal as well).

[T]he lawsuits... claim that tests are racist because blacks fail them at a higher percentage than whites, and require cash awards to be paid to those blacks who failed them. Most blacks generally pass the tests, and the lawsuits do not explain how the tests can be racist against only some blacks. Blacks who passed the tests are excluded from the financial payouts.

Last week, Durham, North Carolina settled with the DOJ, saying blacks failed the tests required to become a firefighter more often, and “Employers should identify and eliminate practices that have a disparate impact based on race.” It said the Durham Fire Department must pay nearly a million dollars to people who failed the test, and hire up to 16 of them.

While the DOJ said the tests were not relevant to actually being a good firefighter, an online practice test suggests that it is directly relevant, that people could die if such firefighters were hired. One question asks if a building is 350 feet away, how many 60-foot hoses would be needed.

The tests are of uneven quality at best. The way they work is that the questions are drawn at random from a large database of test questions that were approved in advance, and the instructor has no way of knowing which questions will appear. The test is administered by a proctor, so the instructor doesn't even see the test on the day of the test. 

The intent is to foster honest testing, but the effects entail that there is no guarantee that the firefighters will have been taught the material on which they are tested. The testing database is only reviewed occasionally, so there are reasonable odds that a question might appear that is out of date. Two examples: in the helicopter rescue operations test we were asked one question about an outdated practice dating to pilots who would have been trained on Medievac in Vietnam; on another occasion, we were asked multiple technical questions about standards for Type I Harnesses, which no longer exist because that entire type was disqualified as acceptable by the NFPA standard a long time ago.

I've also seen test answers that were outright violations of logic. For example, once we received a question about how much heat a device could be exposed to before needing to be replaced. The answers were, I believe, 100 degrees, 200 degrees, 240 degrees, and 300 degrees. Logically only 300 could be correct given that only one answer was acceptable, because if you were exposed to 300 degrees you were also exposed to at least 240, 200, and 100 as well. However, the correct answer was (IIRC) 240, even though being exposed to 300 degrees would exceed that standard also.

So there's a lot of cramming and memorization, just stuffing your head with the exact technical figures that are likely to turn up on a test. Almost none of it is relevant to an emergency, as the decisions about what equipment you will have on the occasion were made long ago when the stuff was purchased -- and the purchases were made by people who had ample leisure to check the technical standards and be sure they were correct. 

I don't think the tests are racist, unless there's a racial disparity in the ability to memorize trivia. I do think they're not the most useful way to test qualifications. As for the one question they ask in the article, maybe it's helpful to know that you will need six hose sections; but probably you aren't going to have an exact measurement of the distance, and you'll just keep adding hose until you get there. I have trouble imagining an occasion when you'd park the apparatus 350 feet away from the fire you wanted to fight anyway; more likely you'll be parked a lot closer, and using the hoses to link multiple apparatus together to boost pressure (and, in rural environments without fireplugs, to increase your water supply or to enable tankers to tie in and out as they go to get more).

The whole system could usefully be rethought. The DOJ's effort, however, is not likely to improve it because it isn't aimed at the parts that don't work well.