Do extraordinary claims require extraordinary evidence?

Pseudo-skepticism and the issue of burden of proof

A UFO over Manitoba, Canada? It certainly is an extraordinary claim… Image from cbc.ca.

You might have heard the catch phrase “Extraordinary claims require extraordinary evidence,” which is often uttered by skeptics against anyone who purports to believe in extra sensorial perception, UFOs, unlikely conspiracy theories, god, and a large variety of other dubious notions.

It is often attributed to one of the founding fathers of the American skeptical movement in the 1970s, the astronomer Carl Sagan, who did in fact use it in a Cosmos/PBS segment aired on 14 December 1980. Sagan, however, borrowed the phrase from Marcello Truzzi, an American (and Danish born) sociologist who co-founded (together with the legendary Paul Kurtz) what was known as the Committee for the Scientific Investigation of Claims of the Paranormal (CSICOP) in 1976. (Nowadays CSICOP is known as CSI, the Committee for Skeptical Inquiry.)

Truzzi’s original phrase was “An extraordinary claim requires extraordinary proof,” as he put it in the first issue of his journal, Zetetic Scholar, published in 1978. In turn, Truzzi himself borrowed the sentiment from the 18th French astronomer Pierre-Simone de Laplace, who said that “the more extraordinary a fact is, the more it needs to be supported by strong evidence.” The Scottish Enlightenment skeptic David Hume said something similar when he wrote, in his essay On Miracles, that “A wise man proportions his belief to the evidence” and should “always reject the greater miracle.”

Regardless of whether we refer to the notion as Sagan’s standard or Hume’s dictum (I preferred the latter), the idea was interpreted very differently by Truzzi on the one hand and by Paul Kurtz and Carl Sagan on the other, leading to a major schism early on in the history of modern skepticism and—most importantly—is at the root of what it means to be a skeptic, or a reasonable person. So it’s worth digging in a little deeper into the whole affair.

Truzzi was once characterized by Kurtz as “the skeptic’s skeptic,” which, knowing Paul’s sarcastic sense of humor, was probably intended as a very backhanded compliment. Here is the story, in brief, before we get to the epistemological meat of the matter.

Truzzi had founded a skeptically-inclined journal known as Explorations, but agreed to make it the official magazine of CSICOP under the title The Zetetic. “Zetetic” is a Greek word that is essentially synonymous with skeptic, both of them meaning inquirer, though zetesis more specifically means research while skepsis indicates thorough investigation. Six of one, half a dozen of the other, as we would say today.

Truzzi served as editor of the magazine during its first year of publication, until August 1977, but soon found himself at odds with the rest of the board of CSICOP. Why? Because Truzzi thought that true skepticism implied a stance of complete neutrality, of agnosticism, toward extraordinary claims, and that therefore it would be intellectually honest to invite authors who supported notions like the paranormal, UFOs, etc. to contribute to CSICOP’s effort. One can sympathize with this “let a thousand flowers bloom” attitude, though I would argue below that Truzzi was wrong about this, in a very interesting and revealing way.

The majority of the CSICOP board thought there were already too many publications devoted to astrology, ESP, and the like, and so they rejected Truzzi’s suggestion. As Sagan put it on a different occasion (in his 1996 book, The Demon-Haunted World): “Keeping an open mind is a virtue—but, as the space engineer James Oberg once said, not so open that your brains fall out.” (Interestingly, this quote also has a long history, going back to 1940.) The upshot was that my friend Ken Frazier took over the publication, now renamed Skeptical Inquirer, which, somewhat redundantly, means “Inquirer Inquirer.”

(Full disclosure: I have often written for Skeptical Inquirer, and I currently publish a regular column on philosophy of science with them.)

Undaunted, Truzzi started another journal, confusingly entitled Zetetic Scholar. He wanted to promote “zeteticism,” a term that obviously never caught on, as an alternative to what he began to refer to as “pseudo-skeptics.” As he wrote in his magazine in 1987:

“In science, the burden of proof falls upon the claimant; and the more extraordinary a claim, the heavier is the burden of proof demanded. The true skeptic takes an agnostic position, one that says the claim is not proved rather than disproved. He asserts that the claimant has not borne the burden of proof and that science must continue to build its cognitive map of reality without incorporating the extraordinary claim as a new ‘fact.’ Since the true skeptic does not assert a claim, he has no burden to prove anything. He just goes on using the established theories of ‘conventional science’ as usual. But if a critic asserts that there is evidence for disproof, that he has a negative hypothesis—saying, for instance, that a seeming psi result was actually due to an artifact—he is making a claim and therefore also has to bear a burden of proof.”

He was not the only one within the skeptic community to feel that way. Former paranormalist-turned-skeptic Susan Blackmore (the author of an interesting, if ultimately misguided book on the concept of memes) wrote:

“There are some members of the skeptics’ groups who clearly believe they know the right answer prior to inquiry. They appear not to be interested in weighing alternatives, investigating strange claims, or trying out psychic experiences or altered states for themselves (heaven forbid!), but only in promoting their own particular belief structure and cohesion. … I have to say it—most of these people are men. Indeed, I have not met a single woman of this type.” (Women skeptics, in: Women and Parapsychology, edited by L. Coly & R. White for the Parapsychology Foundation, 1994, pp. 234-236)

More recently, science journalist John Horgan also took on the skeptic movement (he was invited to a skeptic conference after my suggestion), though his point was different, as he criticizes us for only going after the easy targets (e.g., homeopathy and Big Foot) while staying clear of the difficult ones (e.g., the lack of efficacy of mammograms, the alleged natural roots of war, the lack of empirical foundation for the multiverse, the sheer silliness of the singularity, and more).

John Horgan, lookin buff and bashing skeptics. Image from johnhorgan.org.

Of the above, I think Horgan’s attack is the most serious one, so I’ll leave it for last. The least tenable is Blackmore’s claim that she has never met a skeptic woman who didn’t think a priori that a lot of extraordinary claims are nonsense. This is very odd, as I could name a long list of the kind of women she claims does not exist. As for trying out hallucinogenic substances I’m not sure how that would be scientific or skeptical. The point of science is to study phenomena under controlled conditions, not to “experiment” on oneself. Indeed, doing the latter is generally considered bad science.

Let’s get back to Truzzi’s more substantive objection against what he termed pseudo-skeptics. He does have a point. Sometimes skeptics jump to conclusions because they think they just know that something cannot be the case. When they do so, they make fools of themselves. I describe in detail one such case in my own contribution to the skeptical literature, Nonsense on Stilts: How to Tell Science from Bunk.

In chapter 4 of that book I talk about the famous UFO incident of Campeche, Mexico, which occurred on 5 March 2004. A Mexican military aircraft reported and photographed strange unidentified lights clearly visible over the horizon that night. A number of (pseudo) skeptics jumped in to confidently proffer an plethora of explanations. These included meteor fragments (according to Jose de la Herrin, a local astronomer), electrical flares in the atmosphere (courtesy of Julio Herrera of Mexico’s National Autonomous University), sparks of plasma energy (Rafael Navarro, also of the National Autonomous University), and a group of weather balloons (the explanation offered by the Urania Astronomical Society of Morelos). What these explanations have in common is that they are all dead wrong and that they were all advanced without much thought by someone who simply knew there could not be any unidentified flying object at all.

It took a real skeptical investigator, CSICOP’s own Robert Sheaffer, to actually look into the matter and identify the UFOs. It turns out that the Campeche sightings were not caused by flying objects but by a phenomenon on the ground: they were flares erupting from a group of oil wells (hence their bright infrared appearance recorded by the military aircraft). The crew of the Mexican airplane became confused as to what was moving relative to what else because of the distance, the unfavorable atmospheric conditions, and the unfamiliarity with the phenomenon.

So, turns out Truzzi is right, then? Not exactly. Let us unpack his statement from above a bit more carefully. “In science, the burden of proof falls upon the claimant; and the more extraordinary a claim, the heavier is the burden of proof demanded.” That is completely correct, and as we have seen it is a principle that goes back at least to Hume and Laplace, though Cicero argues in a similar way in De Divinatione, where he criticizes the Stoics for believing in the pseudoscience of divination (he didn’t use that term, but that’s clearly what he meant).

Truzzi continues: “If a critic asserts that there is evidence for disproof, that he has a negative hypothesis—saying, for instance, that a seeming psi result was actually due to an artifact—he is making a claim and therefore also has to bear a burden of proof.” Here too he is right, and the various “skeptics” who proffered unsubstantiated explanations of the Campeche UFOs clearly run afoul precisely on these grounds.

However, Truzzi also crucially says that “the true skeptic takes an agnostic position, one that says the claim is not proved rather than disproved.” Well, now we are getting into an interesting discussion about what proper skepticism is supposed to be. There are two major alternatives: Pyrrhonism and Academic Skepticism, both having roots in the ancient Greco-Roman world.

Marcello Truzzi, the skeptic’s skeptic. Image from Wikimedia.

Pyrrhonism is the root of Truzzi’s brand of skepticism, whether he realized it or not. The creed of a Pyrrhonist is that we ought to suspend judgment on all “non-evident” matters because for any argument and evidence in favor of proposition X it is possible to articulate a counter-argument, or to bring up counter-evidence that denies X. (There are other interesting arguments brought up by Pyrrhonists, see here.) As a result, the rational thing to do is to suspend judgment—i.e., to behave agnostically, as Truzzi puts it—a course of action that, according to Pyrrhonists, has the advantage of bringing about ataraxia, or tranquillity of mind.

There are various problems with Pyrrhonism, beginning with the fact that it isn’t exactly clear where they draw the distinction between evident and non-evident matters, though we can rest assured that the sort of issues we are talking about would counts as non-evident. (On evident matters Pyrrhonists simply follow common opinions, their natural drives, and recognized expertise.)

The big problem is that in many cases—contra the Pyrrhonist claim—it is not at all reasonable to think that we can articulate an equally convincing counter-argument against a given position X. Let’s consider two sharply contrasting examples.

First, if I were to strongly argue that the number of stars visible in the sky from Rome during a clear night in mid-summer is even you could very easily come up with a counter-argument aiming at demonstrating that the number is actually odd. In this case, obviously, agnosticism truly is the only viable option, and one would be a fool to take any other position.

Second, however, suppose that I’m arguing that quantum mechanics is a highly reliable and empirically predictive scientific theory. Now it would be next to impossible for you to come up with reasonable arguments to the contrary. Agnosticism in this case would look foolish, because the “priors,” as Bayesians call them, i.e., our expectations of one hypothesis being true, are much, much, much higher than the corresponding priors in favor of the alternative hypothesis. Suspension of judgment here is not a reasonable option at all.

Ah, you might say, but those are clearcut cases. Aren’t a lot of situation somewhere in the middle? They are, which is why the second kind of skepticism, the Academic variety (so-called because it characterized a period of Plato’s Academy) is the way to go.

An Academic Skeptic like Carneades or Cicero would say that statements, hypotheses, and theories all come with varying degrees of persuasiveness (pithanon, translated by Cicero into Latin as probabilis), so that in some cases agnosticism is the way to go, in other cases we ought to strongly lean toward one possibility over its counter, and in yet other cases our confidence in position X will be somewhere in the middle.

I argue that the majority of modern skeptics are—again whether they realize it or not—Academic Skeptics, not Pyrrhonists. As they should be! In practice this means that we are perfectly justified in thinking that, say, yet another UFO sighting, or yet another report of telepathy, or yet another psychic, and so forth are very likely bunk and do not actually need to be investigated. Not until those making the claim are prepared to provide us with extraordinary evidence to support it. This means that agnosticism is far too conservative a position in many cases and that, contra Truzzi, we don’t need to give more space to purveyors of (likely) nonsense, who are dangerous enough as it is. However, we are not thereby authorized—as Truzzi pointed out—to just throw out there random “explanations” for things we have not investigated. That truly is intellectually dishonest.

Let me now come back to Horgan’s objections to modern skepticism. I think his criticisms ought to be welcome and taken seriously by the community. But I also think he goes a bit overboard in his effort to apply a correction. He seems somewhat contemptuous of the skeptical effort as a whole, and accuses us of not doing things that we, in fact, do.

Skeptics have written multiple times about several of the topics Horgan is concerned about. The entire Science-Based Medicine site is devoted to criticism not just of homeopathy, but also of a number of other medical claims, whether mainstream or not, that do not measure up to the available evidence. And speaking of homeopathy, the fact is that billions are made every year by people who sell water and sugar, and as a consequence countless patients forgo empirically sound health care to their peril. This is the sort of betrayal of public trust that is most definitely worth pointing out, regardless of how much of a low hanging fruit Horgan thinks it is.

I am on record criticizing the facile and scientistic approach to ethics of Sam Harris and, to some extent, of Michael Shermer—both fellow skeptics, and the latter a friend. I have certainly denounced the notion of a singularity as pseudoscience, and commented on the worrisome tendency of some fundamental physics to go “post-empirical.”

That said, some of the topics raised by Horgan require a solid technical-scientific background in order to be tackled them with confidence, so many skeptics are rightly cautious before getting in a debate with a Neil deGrasse Tyson or a Brian Greene.

Could the skeptical movement use more self-criticism and an expanded horizon of targets? Absolutely. But let’s not throw the baby out with the bath water. Skepticism has been for two and a half millennia a torn in the side of all sorts of fuzzy thinkers. It needs to be strengthened and encouraged, not bashed. We need more of it, done better. And people like Truzzi, Horgan, and Blackmore can certainly help!