Ouija Science

Maggie's Farm sent me to a list of "Eight Warning Signs for Junk Science," including:

1. Science by press release.
2. Rhetoric that mixes science with the tropes of eschatological panic (“a terrible catastrophe looms over us if theory X is true, therefore we cannot risk disbelieving it”).
3. Rhetoric that mixes science with the tropes of moral panic (“only bad/sinful/uncaring people disbelieve theory X”).
4. Consignment of failed predictions to the memory hole (also known as "moving the goal posts").
5. Computer models replete with bugger factors that aren’t causally justified (if you don’t have a generative account that makes falsifiable predictions, you’re not doing science, you’re doing numerology.
6. Convenience (if a ‘scientific’ theory seems tailor-made for the needs of politicians or advocacy organizations, it probably has been).
7. Bad reps (past purveyers of junk science do not change their spots).
8. Refusal to make primary data sets available for inspection.

The comments section delivered several valuable similar links: Seven Warning Signs of Quack Science, Six Symptoms of Pathological Science (includes "criticisms are met by ad hoc excuses made up on the spur of the moment"), and Ten Signs a Claimed Mathematical Breakthrough Is Wrong. I didn't understand most of that last one, but I enjoyed the author's conclusion that these are only warning signs, not disproofs. Even if a paper fails most or all of these tests, he cautions,"there might be nothing left to do except to roll up your sleeves, brew some coffee, and tell your graduate student to read the paper and report back to you."

One of the comments quoted this observation by my hero, Richard Feynman (in "Cargo Cult Science"): "When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition." His advice on the occasion of the Challenger disaster is apropos as well: "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled."