Why that study about psychic porn was totally bogus

Late last year, a Cornell psychologist claimed to have discovered evidence of precognition, specifically an ability to detect the future presence of erotic photos. We were skeptical about all this, and now it appears we can officially dismiss this study.

Before we get to the debunking, let's look at the original claims. Cornell professor emeritus Daryl Bem set up a series of experiments to test the presence of psi, or "anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or biological mechanisms" - in other words, evidence of extrasensory perception.

In one of the experiments, 50 female and 50 male students played a computer game where they were shown two curtains on the screen, and then they had to guess which one had a picture hiding behind it. Some of the photos were neutral in content, while others were erotic. When an erotic photo was behind on the curtains, the participants guessed its presence 53.4% of the time. That's a statistically significant result - hardly overwhelming evidence of anything, but it can't be immediately dismissed as random chance either. (For a more complete overview, check out this earlier post.)

This isn't the first study to claim statistically significant evidence for some form of precognition - Bem calls it "retroactive influence", while American's greatest living newsman Stephen Colbert dubbed it "extrasensory pornception." But all those previous studies proved to be impossible to reproduce, featuring serious methodological flaws and biases on the part of the experimenter. And it's starting to look like that's the case here as well.

In breaking down the problems with Bem's research, skeptic Michael Shermer explains some of the fundamental issues that have already come to light with the study:

Experimental inconsistencies plague such research. [Paranormal research expert Ray] Hyman notes that in Bem's first experiment, the first 40 subjects were exposed to equal numbers of erotic, neutral and negative pictures. Then he changed the experiment midstream and, for the remaining subjects, just compared erotic images with an unspecified mix of all types of pictures. Plus, Bem's fifth experiment was conducted before his first, which raises the possibility that there might be a post hoc bias either in running the experiments or in reporting the results.

Moreover, Bem notes that "most of the pictures" were selected from the International Affective Picture System, but he does not tell us which ones were not, why or why not, or what procedure he employed to classify images as erotic, neutral or negative. Hyman's list of flaws numbers in the dozens. "I've been a peer reviewer for more than 50 years," Hyman told me, "and I can't think of another reviewer who would have let this paper through peer review. They were irresponsible."

There's plenty of other problems. None of Bem's other eight experiments showed any signs of such a precognitive effect, which doesn't really make sense - if precognition does exist, even as a very subtle effect, then surely it should at least be consistent in its subtlety. There's also what Ray Hyman calls "the patchwork quilt problem", which is the fact that evidence for paranormal effects can only ever be positive - there can be data proving the existence of psi, but never anything disproving it, and paranormal researchers can claim any unexplained effect as evidence of psi.

The simple fact of the matter is that we have no real evidence to support the existence of precognition, and none of the studies purporting to provide proof have stood up to even the mildest scrutiny. While I suppose we can't completely rule out the possibility of some form of precognition - although I'd be utterly shocked if anyone ever finds anything even remotely convincing - it doesn't appear that this study has much of value to add to the discussion.

Via Scientific American. Image via.