Designing Tests for the Paranormal

There’s nothing wrong with testing paranormal claims.

The desire to test is often misinterpreted as an attack. It’s seen as equivalent to the statement, “I don’t believe you.” But in my opinion you honor something immensely by testing it. You give it the chance to reveal its full glory to you and the rest of the world. You imbue it with the strength and weight it deserves. A well-designed test is never a threat to something that’s true, only to something that’s false.

By testing a claim — especially a claim of your own — you can only come to a better and more authentic understanding of it. This is only a problem when you’re more interested in what you want to be true than in what is true. If it’s the latter you value, testing is always a win-win.

I’ve often had people try to sell me on an idea, and after talking to them I discover that they haven’t really tested the idea at all. They simply trusted the source of it, or liked the sound of it, and that was enough for them. If that’s really all they want it’s not my place to criticize, but I eagerly await the day when I come upon a claim that has already been tested more thoroughly by the bearer of it than I would have done myself. Then I know I’m dealing with someone serious.

Test, test, test. It’s ok. Really. This is what science is all about.

I’ve tried to come up with various tests that will most effectively prevent ambiguous results. Ambiguity can create just enough mystery to lure people into believing things which they would have otherwise been unimpressed by. It leaves gaps for people to hide in who are still clinging to shaky ideologies. It’s where wishful thinking flourishes.

There are ambiguities that we’ll never be relieved of, but some are surprisingly easy to wipe out with a little worthwhile effort. Let’s focus for now on the easy stuff.

You can make some excellent progress by sticking to this simple principle:

Every test should be as easy as possible for a genuine participant to pass, and as hard as possible for a fraudulent participant to pass, simultaneously.

Say you’re confronted by a blind person who claims that he can see. How could you test his claim quickly and easily in a way that would remove ambiguity?

You could try asking him what color the sky is. Our guiding principle shows us, however, that this is a pretty useless test. It is indeed easy for a sighted person to pass, but it’s also easy for a blind person to pass. A blind person has probably learned that if the sun is beating down on him, blue is a very good guess. If not, it might be better to go with white. If tested at night, he knows he should answer black. There’s huge potential here for a false positive.

So instead you try asking him how many fingers you’re holding up. This is still extremely easy for a sighted person, but there’s only a 10% chance of a false positive. Or a 9% chance if you’re clever enough to consider not holding up any fingers at all.

We can still do much better. We want the chance of a false positive or a false negative to be astronomically low — effectively zero, in other words.

You might consider writing a word on a piece of paper and asking him to read it aloud. A false positive is now incredibly unlikely, but what about a false negative? Could the person be able to see, but unable to read? Could he be dyslexic and have trouble with certain words? Do you have really sloppy handwriting? There isn’t that much danger of a false negative here, but if you can improve the test even further it’s worth doing.

You could return to the finger counting game, but instead of counting them once, you make him count them ten separate times. Now the odds are even greater against a false positive than in the word test (one in 26 billion), and you won’t find many sighted people out there who can’t count to ten.

And none of these tests take more than a minute to do.

We could improve on them further; I just want to give you the gist of the process.

By following this principle you manage to weed out a great deal of the ambiguity that will haunt you later, while keeping things very easy for the person being tested so he has little reason to object. The attitude is more likely to be, “That’s all I have to do? Piece of cake!” rather than, “Why are you bothering me with all this?” You don’t want to chase away the people who actually have something to show you, if you can help it.

The process isn’t rigorous or scientific, but done well it can give you a solid idea of someone’s true abilities in a very short period of time. When the truth of someone’s claim could alter your whole understanding of the very universe — as most supernatural claims would — it seems crazy not to take these very simple steps on a regular basis.

In the future I’ll try to give examples of tests that I think work well for specific claims that I’ve run into. The first is coming up next.

About this entry