Notes

Chapter 1. As a Matter of Fact

1. Especially in Chapter 13, addressing what psychologists call “motivated reasoning.”

2. Those of us who know little about either vaccination or autism can assert that vaccination does not cause autism because experts who do know about vaccination and autism have concluded that vaccination does not cause autism. My evidence is the fact of the conclusion of the experts, even though their evidence comes from controlled experiments, data analysis, systematic observation, and other methods of scientific inquiry. See Frank DeStefano, Cristofer S. Price, and Eric S. Weintraub, “Increasing Exposure to Antibody-Stimulating Proteins and Polysaccharides in Vaccines Is Not Associated with Rise of Autism,” Journal of Pediatrics 163 (2013): 561–567; Dennis K. Flaherty, “The Vaccine-Autism Connection: A Public Health Crisis Caused by Unethical Medical Practices and Fraudulent Science,” Annals of Pharmacotherapy 45 (2011): 1302–1304. Treating expert conclusions as evidence is the focus of Chapters 9, 10, and 11, and the vaccination–autism example reappears in Chapter 9.

3. See “FDA Cautions against Use of Hydroxychloroquine or Chloroquine for COVID-19 Outside of the Hospital Setting or a Clinical Trial Due to Risk of Heart Rhythm Problems,” at www.fda.gov (July 1, 2020), referencing Pharmacovigilance Memorandum of May 19, 2020, prepared by the Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, at www.accessdata.fda.gov; Recovery Collaborative Group, “Effect of Hydroxychloroquine in Hospitalized Patients with Covid-19,” New England Journal of Medicine 383 (Nov. 2020): 2030–2040; Adam Clark Estes, “Hydroxychloroquine Conspiracies Are Back, but Trump’s the Patient Now,” at www.vox.com (Oct. 7, 2020).

4. Daniel P. Moynihan, “More than Social Security Was at Stake,” Washington Post, Jan. 18, 1983, A17. The phrase and the idea are now associated with Moynihan, but the website Quote Investigator (www.quoteinvestigator.com) reports that much the same phrase was first used by the financier Bernard Baruch in the 1940s and by then–secretary of defense James Schlesinger in the 1970s.

5. David Hume, A Treatise of Human Nature, ed. L. A. Selby-Bigge, 2nd ed., revised by P. H. Nidditch (Oxford: Clarendon Press, 1975) (orig. pub. 1739), bk. 3, pt. 1, sec. 1, par. 27. Ironically, questions of evidence now surround Hume himself. The University of Edinburgh recently renamed (to 40 George Square) what was previously the David Hume Tower, basing the renaming on racist statements Hume made in some of his writings, especially in a footnote in “Of National Characters,” in David Hume, Essays Moral, Political Literary, ed. Eugene F. Miller (Indianapolis: Liberty Fund, 1987) (orig. pub. 1777), 197–215, at 207n10. As a result of the renaming, questions of evidence about Hume, and not just in the Hume oeuvre, are now prominent. Although questions whether to rename buildings or take down statues are normative, lurking behind the normative questions are a host of preliminary factual ones. For Hume, such factual questions—questions whose answers demand evidence—include questions such as what Hume did or did not write, what opinions Hume did or did not hold, what actions Hume did or did not take, and what beliefs Hume’s contemporaries did or did not possess. As long as we accept some version of Hume’s distinction between the is and the ought, these factual questions are unavoidable prerequisites to any decision about whether and how to celebrate Hume and whether, how, and how much to condemn him.

6. More accurately, yellowish-appearing skin is evidence of jaundice, and jaundice is evidence of hepatitis. Although it may be more difficult to detect the change of skin tone for many African Americans than for most whites, the same phenomenon exists for both.

7. See Adam Liptak, “Amy Coney Barrett, Trump’s Supreme Court Pick, Signed Anti-Abortion Ad,” New York Times, Oct. 1, 2020, A1. The link from a judge’s current policy or moral beliefs to that judge’s current judicial decision making is uniformly resisted by nominees themselves, as it was by Justice Barrett in her confirmation hearings in October 2020. And it is typically resisted by many—perhaps most—legal professionals. At least with respect to Supreme Court justices, however, the conclusion that current moral and policy views are strong evidence of likely votes as a justice is supported by decades of research by empirical political scientists. See Saul Brenner and Harold J. Spaeth, Stare Indecisis: The Alteration of Precedent on the Supreme Court, 1946–1992 (New York: Cambridge University Press, 1995); Jeffrey J. Segal and Harold J. Spaeth, The Supreme Court and the Attitudinal Model Revisited (New York: Cambridge University Press, 2002). The attitudinal model exemplified in these books has been challenged and refined in recent years, but the basic idea that pre-legal attitudes are the chief (but not the only) determinant of Supreme Court votes remains the “holy grail” of empirical research about Supreme Court decision making. Nancy Scherer, “Testing the Court: Decision Making under the Microscope,” Tulsa Law Review 50 (2015): 659–668, at 661.

8. See CNN Opinion, “Who Won the Debate,” www.cnn.com, Oct. 23, 2020; Jennifer Agiesta, CNN Poll, “Biden Wins Final Presidential Debate,” www.cnn.com, Oct. 23, 2020.

9. Admirably and fortunately, not always. See Dave Boucher, “Michigan Board Votes to Certify Election Results despite GOP Calls to Delay,” Detroit Free Press, Nov. 23, 2020; Brad Raffensperger, “I Have Fought to Uphold the Integrity of Elections in Georgia. It Doesn’t Matter if the Attacks Come from the Guy I Voted For,” USA Today, Nov. 25, 2020.

10. “Sir, I had rather be right than president,” speech in the United States Senate, Feb. 7, 1839. See Robert Seager III, “Henry Clay and the Politics of Compromise and Non-Compromise,” Register of the Kentucky Historical Society 85 (1987): 1–28.

11. Philippa Foot, “Moral Arguments,” Mind 67 (1958): 502–513. A valuable overview is Pekka Väyrynen, “Thick Ethical Concepts,” Stanford Encyclopedia of Philosophy, www.plato.stanford.edu (2016); Bernard Williams, Ethics and the Limits of Philosophy (Cambridge: Cambridge University Press, 1985), 141–144.

12. For background see John Hooper, Fatal Voyage: The Wrecking of the Costa Concordia (Amazon / Kindle, 2015); Marco Imarisio and Fiorenza Sarzanini, Concordia: The True Story (Milan: Corriere Della Sera, 2012); Edward Jones, Reckless Abandon: The Costa Concordia Disaster (Amazon / Kindle, 2012); “Captain of Ship That Capsized off Italy in ’12 Is Convicted,” New York Times, Feb. 12, 2015, A4.

13. See Gregory Korte, “ ‘Crooks’ Versus ‘Socialists’: Ads Frame Georgia Bitter Runoffs,” Bloomberg News, Dec. 31, 2020, www.bloomberg.com.

14. See Dan Berman, “Barrett Ducks Questions on Presidential Power and Systemic Racism in New Answers to Senate,” CNN Politics, at www.cnn.com (Oct. 21, 2020).

15. One of the first to treat evidence as a legal subject in its own right was Lord Geoffrey Gilbert, whose treatise The Law of Evidence was first published in London in 1754, twenty-eight years after Gilbert’s death. On the origins of evidence law in the common-law world generally, see John H. Langbein, “Historical Foundations of the Law of Evidence: A View from the Ryder Sources,” Columbia Law Review 96 (1996): 1168–1202.

16. This is not to suggest that common-law legal systems, which emerged from English law, are more valuable sources of enlightenment than the legal systems of the civil-law world. Rather, the contrast arises because most civil-law jurisdictions have traditionally employed something resembling a “free proof” approach tending to avoid much use of rules and instead evaluating evidence on a case-by-case basis in the same way that ordinary people evaluate evidence in their everyday lives. See Mirjan R. Damaška, Evidence Law Adrift (New Haven, CT: Yale University Press, 1997); Damaška, “Free Proof and Its Detractors,” American Journal of Comparative Law 43 (1995): 343–357; Frederick Schauer, “The Role of Rules in the Law of Evidence,” in Philosophical Foundations of the Law of Evidence, ed. Christian Dahlman, Alex Stein, and Giovanni Tuzet (Oxford: Oxford University Press, forthcoming 2021). There may be much to be said for the free-proof idea, and Jeremy Bentham, although living and writing in a common-law country, said a great deal of it with characteristic vitriol. Jeremy Bentham, Rationale of Judicial Evidence, in The Works of Jeremy Bentham, vol. 6, ed. John Bowring (Edinburgh: William Tait, 1843). But the nonsystematic approach of the free-proof model as compared to the rule-based common-law approach that makes the former less incrementally valuable in contributing to how we might think about evidence.

17. Oliver Wendell Holmes, “The Path of the Law,” Harvard Law Review 10 (1897) 457–476, at 459, written when Holmes, not yet on the Supreme Court, was a justice of the Massachusetts Supreme Judicial Court.

18. Letter from Oliver Wendell Holmes Jr. to Harold J. Laski (Jan. 5, 1921), in The Holmes-Laski Letters, vol. 1, ed. Mark De Wolfe Howe (Cambridge, MA: Harvard University Press, 1953), 300.

19. Steven Brint, In an Age of Experts: The Changing Role of Professionals in Politics and Public Life (Princeton, NJ: Princeton University Press, 1994).

Chapter 2. Zebras, Horses, and the Nature of Inference

1. In fact, Dr. Woodward was nominated for (but did not win) a Nobel prize in 1948, but for his work on a cure for typhoid fever and not for the “horses, not zebras” maxim.

2. Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable (New York: Random House, 2007).

3. Barbara Spellman reminds me that the problem with the “great minds think alike” adage is that it is wrong. Great minds are great precisely because they depart from the crowd (or the herd, as Aristotle put it) and from each other.

4. See Centers for Disease Control and Prevention, “Lyme Disease,” at www.cdc.gov.

5. Just how those probabilities come into play implicates long-standing debates about “inference to the best explanation,” Bayesian updating, and so-called probabilism, all to be addressed later in this chapter.

6. Ian Hacking, An Introduction to Probability and Inductive Logic (Cambridge: Cambridge University Press, 2001), 11.

7. See D. J. Friedland et al., Evidence-Based Medicine: A Framework for Clinical Practice (New York: McGraw-Hill / Lange, 1998); David M. Eddy, “Evidence-Based Medicine: A Unified Approach,” Health Affairs 24 (2005), 9–17; Evidence-Based Medicine Working Group, “Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine,” Journal of the American Medical Association 268 (1992): 2420–2425.

8. Evidence-based medicine is explicitly described as a “movement” in Desmond J. Sheridan and Desmond G. Julian, “Achievements and Limitations of Evidence-Based Medicine,” Journal of the American College of Cardiology 68 (2016): 204–213, at 205.

9. D. L. Sackett, W. M. Rosenberg, and J. A. Gray, et al., “Evidence Based Medicine: What It Is and What It Isn’t,” British Medical Journal 312 (1996): 71–72.

10. See the summary in Matthew Herper and Helen Branswell, “Moderna Covid-19 Vaccine Is Strongly Effective, Early Look at Data Shows,” www.statnews.com (Nov. 16, 2020). Pfizer’s similarly designed study used 43,000 participants and yielded a 95 percent effectiveness rate; www.cnn.com (Nov. 18, 2020). For the full protocols of the Moderna and Pfizer tests, see https://www.modernatx.com/sites/default/files/mRNA-1273-p301-Protocol.pdf, and https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_clinical_protocol.pdf. Obviously much depends on the “otherwise identical” configuration of the placebo and treatment groups. If we are dealing with humans, it is easy to grasp that no two groups are truly identical. But so too with lab mice, for no two groups of mice are identical, although the differences among mice may be less apparent to us. Still, the larger the treatment and control (placebo) groups, the more likely it is that the array of characteristics in one group will be highly similar to the array in another group drawn at random from the same population and selected in the same way.

11. On how qualitative or case-based inquiry can be (and should be) as theoretically informed and rigorous as quantitative inquiry using large data sets, see Gary King, Robert O. Keohane, and Sidney Verba, Designing Social Inquiry: Scientific Inference in Qualitative Research (Princeton, NJ: Princeton University Press, 1994).

12. “Wants us to believe …” is not a claim about the motivations of the evidence-based medicine movement. It is a claim about language, based on the idea that assertions are made only when there is some point in doing the asserting. And that point is usually the plausibility of the assertion’s negation. As the philosopher John Searle puts it, “no remark without remarkableness.” John R. Searle, Speech Acts: An Essay in the Philosophy of Language (Cambridge: Cambridge University Press, 1969), 144. If I observe, correctly, that my colleague Sam is sober today, I have raised the inference that there are days on which he is not, just as the signs on the Massachusetts Turnpike warning drivers not to back up on the high-speed limited access highway if they have missed their exit suggest, alarmingly, that there are Massachusetts drivers who would actually do that. So too with “evidence-based medicine,” a practice whose label raises the inference that there is another kind of medicine, one not based on evidence.

13. This is especially true when an experiment is designed and conducted (or supervised) by researchers with no financial interest in the outcome. To minimize this risk, both Moderna and Pfizer took pains to stress that their tests were conducted in conjunction with health-focused government agencies such as the National Institute of Allergy and Infectious Diseases, a part of the National Institutes for Health.

14. On the QAnon conspiracy theory, see Kevin Roose, “What Is QAnon, the Viral Pro-Trump Conspiracy Theory?,” New York Times, Oct. 19, 2020.

15. Donald J. Trump for President, Inc. v. Boockvar, Case 4:20-cv-02078-MWB (M.D. Pa., Nov. 21, 2020).

16. On circumstantial evidence, see Zechariah Chafee Jr., “La Critique du Témoignage” (book review), Harvard Law Review 42 (1929): 839–843, at 840, describing but not endorsing the “widely prevalent belief” that there is something wrong with circumstantial evidence. But the law wisely knows no such category. That a defendant with motive to kill the victim was in the vicinity at the time and place of the murder may be stronger evidence than an eyewitness identification by a witness with poor eyesight from a great distance on a foggy night, and stronger than a partial fingerprint taken by an inexperienced police investigator. Evidence can be stronger or weaker, but the strong–weak scale maps poorly onto one that puts eyewitness identification or physical evidence at one pole and everything else, including circumstantial evidence, at the other. It was earlier argued that the direct / circumstantial distinction reflects the difference between evidence that is itself evidence of something that matters in some trial, and evidence that requires an additional inferential step, the latter being systematically weaker. Arthur P. Will, A Treatise on the Law of Circumstantial Evidence (Philadelphia: T and J. W. Johnson, 1896). Under this view, there is a difference between the direct evidence offered by someone standing nearby when the defendant says “Hand over the money or I’ll shoot you” and the circumstantial evidence of someone who did not see the robbery but saw the defendant running out of the bank carrying a bag just after the robbery was alleged to have taken place. But the strength of the inference from the circumstantial evidence is often, as here, very strong, which is why the law refuses to treat the category of circumstantial evidence as categorically less valuable just because it is circumstantial. See, among thousands of decisions, State v. Quinet 752 A.2d 490 (Conn. 2000); State v. Jenks, 574 N.E.2d 492 (Ohio, 1991); State v. Derouchie, 440 A.2d 146 (Vt. 1981).

17. In The Adventures of Silver Blaze (1892), Sherlock Holmes inferred from the absence of a dog’s barking the non-occurrence of an event that would have been expected to cause the dog to bark. And although on November 4, 2020, the day after the presidential election, the charges of electoral fraud made by President Trump and his allies were properly labeled as having been made without evidence, the subsequent failure over the ensuing months by those alleging fraud to provide documentary, statistical, testimonial, or circumstantial evidence of such fraud can be understood as evidence of the absence of fraud. With every day that passed without evidence having been offered by those with an interest in there being such evidence, what started as no evidence of fraud increasingly turned into evidence of no fraud.

18. A valuable book by Beth A. Bechky about crime labs and forensic evidence is entitled Blood, Powder, and Residue: How Crime Labs Translate Evidence into Proof (Princeton, NJ: Princeton University Press, 2021). The title implies that at some point individual pieces of evidence prove some conclusion. This usage is common, but it risks suggesting, misleadingly, that evidence is useful only when it is sufficient to establish criminal guilt beyond a reasonable doubt.

19. I say “typically” in order to leave to philosophers—and maybe neuroscientists—questions about what, if anything, we can know a priori (that is, without evidence). It is important to recognize, though, that even the things we understand as direct perceptions—what we see, smell, hear, taste, and touch—are themselves based on evidence. That I think that something I am eating is salty, for example, is based on my having a sensation on my tongue that I have come to learn is produced by sodium chloride contacting certain of my taste buds.

20. See https://www.monticello.org/site/blog-and-community/monticello-affirms-thomas-jefferson-fathered-children-sally-hemings#footnoteref1-m550xjt. See also Annette Gordon-Reed, Thomas Jefferson and Sally Hemings: An American Controversy (Charlottesville: University of Virginia Press, 1997); Annette Gordon-Reed, The Hemingses of Monticello: An American Family (New York: W. W. Norton, 2008). For a dissenting view, see the claims of the Thomas Jefferson Heritage Society, www.tjheritage.org, and Robert F. Turner, ed., The Jefferson-Hemings Controversy: Report of the Scholars Commission (Durham, NC: Carolina Academic Press, 2011). Although the Thomas Jefferson Foundation admirably relied explicitly on Bayes’ theorem to support the conclusions they reached from multiple and successive items of evidence, it is important to recognize that the .50 figure that the Foundation used as its prior probability—prior to its statistical analysis of the likelihood that Jefferson was the father of all six of Hemings’s children—is itself a qualitative rough estimate of the evidence available prior to the statistical analysis. The .50 figure seems reasonable, but attaching a hard number to an impressionistic assumption, albeit one based on evidence, suggests greater accuracy than in fact exists. It is also worth noting that a common error, although not one that the Foundation commits, is setting the prior probability at .50 in the absence of any evidence. Doing so is often based on the assumption that in the absence of evidence, one could reach either of two opposing conclusions. But a proper assessment of prior probability comes from either evidence or a plausible assumption, and it would hardly have been plausible to assume a 50 percent probability that a slave owner named Thomas Jefferson impregnated an enslaved person in his household named Sally Hemings. So the true prior probability was very low. According to the Foundation’s report, the DNA evidence and various pieces of documentary evidence then raised the posterior probability to .50. Then that posterior probability was used as the prior probability for the further evidence supplied by the statistical analysis.

21. Evidence that “merely” confirms what we already knew will ordinarily still be evidence whenever the probability prior to receiving the confirming evidence is less than 100 percent. If three witnesses observed a red Buick at the corner of First Avenue and Elm Street at 11 a.m. on October 6, then the observation of a car with the same description on the same day at the same time by a fourth witness makes the previously highly probably conclusion that that car was in that place at that time even more highly probable.

22. See Rudolf Carnap, Logical Foundations of Probability, 2nd ed. (Chicago: University of Chicago Press, 1962), 463; Mary Hesse, The Structure of Scientific Evidence (Berkeley: University of California Press, 1974), 134.

23. One example is Peter Achinstein, “Concepts of Evidence,” Mind 87 (1978): 22–45.

24. Van Meegeren’s forgeries have been the subject of several books, including Edward Dolnick, The Forger’s Spell: The True Story of Vermeer, Nazis, and the Greatest Art Hoax of the Twentieth Century (New York: HarperCollins, 2008). Those taken in by van Meegeren’s forgeries included prominent art experts, prominent museums, and Hermann Goering, which undoubtedly contributes to the continuing fascination.

25. After the end of the Second World War, van Meegeren was charged with collaborating with the Nazis for having sold several important paintings, including a Vermeer, to Hermann Goering, whose eagerness to “acquire” great art was without limit. Van Meegeren wisely preferred to be punished as a forger than as a traitor, and his confession to the forgery resulted in the dropping of charges against him for selling part of the Dutch patrimony to the Nazis. But van Meegeren died in 1947, within a month of his confession, and he never served any of his eventual one-year sentence for forgery. For the full story, see Dolnick, The Forger’s Spell; Jonathan Janson, “Essential Vermeer 3.0,” www.essentialvermeer.com (2020); Beneditta Ricca, “The Art of Forgery—Art Forgers Who Duped the World,” Artland, April 17, 2020, https://magazine.artland.com/the-art-of-forgery-art-forgers-duped-world/.

It is worth noting that confessions, although ordinarily strong evidence of what the confessor has confessed to, are still only evidence of that conclusion, and in some contexts may be less than reliable. For various reasons, people sometimes confess to doing things they did not do, even when the confession exposes them to criminal punishment. So although a confession is evidence, it is not conclusive. Saul M. Kassin, “False Confessions: Causes, Consequences, and Implications for Reform,” Current Directions in Psychological Science 17 (2008): 249–253; Saul Kassin and Katherine L. Kiechel, “The Social Psychology of False Confessions: Compliance, Internalization, and Confabulation,” Psychological Science 7 (1996): 125–128; Richard A. Leo, “False Confessions: Causes, Consequences, and Implications,” Journal of the American Academy of Psychiatry and Law 37 (2009): 332–343.

26. A recent overview is Nick Chater et al., “Probabilistic Biases Meet the Bayesian Brain,” Current Directions in Psychological Science 29 (2020): 506–512.

27. For “beliefs come in degrees,” see Edward Elliott, “ ‘Ramseyfying’ Probabilistic Comparison,” Philosophy of Science 87 (2020): 727–754.

28. On the use (and misuse) of generalizations in various contexts, see Frederick Schauer, Profiles, Probabilities, and Stereotypes (Cambridge, MA: Harvard University Press, 2003). On the concept of a generalization as being central to the idea of a rule, see Schauer, Playing by the Rules: A Philosophical Examination of Rule-Based Decision-Making in Law and in Life (Oxford: Clarendon Press, 1991). Also relevant is the idea of a generic, as theorized prominently by the philosopher Sarah-Jane Leslie. See Sarah-Jane Leslie, “Generics: Cognition and Acquisition,” Philosophical Review 117 (2008): 1–47; Leslie, “Generics and the Structure of the Mind,” Philosophical Perspectives 21 (2007): 375–403; Sarah-Jane Leslie and Adam Lerner, “Generic Generalizations,” Stanford Encyclopedia of Philosophy, https://plato.stanford.edu/archives/win2016/entries/generics/ (2016). “Volvos are reliable” is a generic statement because most Volvos are reliable, but “ticks cause Lyme disease,” to use one of Leslie’s examples, is also a generic even though only about 1 percent of ticks cause Lyme disease. And some of why this is so depends on the category with which the generic is being compared, making the idea of a meaningful generic statement very close to the idea of incremental relevance as discussed in the text.

29. One accessible analysis applicable beyond the law is George F. James, “Relevancy, Probability, and the Law,” California Law Review 29 (1941): 688–705.

30. “The Examination (Audit) Process,” FS-2006-10 (Jan. 2006), at www.irs.gov.

31. See Karen Hube, “Game of Chance?,” Wall Street Journal, Apr. 19, 1999.

32. Gilbert Harman, “The Inference to the Best Explanation,” Philosophical Review 74 (1965): 88–95; Peter Lipton, Inference to the Best Explanation, 2nd ed. (London: Routledge, 2004).

33. The most comprehensive discussion of holistic reasoning, both in the legal system and generally, is Amalia Amaya, The Tapestry of Reason: An Inquiry into the Nature of Coherence and Its Role in Legal Argument (Oxford: Hart, 2015). As Amaya makes clear, inference to the best explanation is closely related (but not identical) to the idea of abduction as developed in the nineteenth century by the pragmatist philosopher Charles Sanders Peirce. On the differences between abduction in the modern sense (inference to best explanation) and abduction as understood by Peirce, see Igor Douven, “Abduction,” Stanford Encyclopedia of Philosophy, at www.plato.stanford.edu (2017). On inference to the best explanation generally, see also Gloria Hon and Sam S. Rakover, eds., Explanation: Theoretical Approaches and Applications (Dordrecht: Springer, 2001); Peter Achinstein, “Inference to the Best Explanation: Or, Who Won the Mill-Whewell Debate?,” Studies in the History and Philosophy of Science 23 (1992): 349–364; Philip Kitcher, “Explanatory Unification and the Causal Structure of the World,” in Scientific Explanation, ed. Philip Kitcher and Wesley Salmon (Minneapolis: University of Minnesota Press, 1989), 410–505; Bas C. van Fraassen, Laws and Symmetry (Oxford: Oxford University Press, 1989).

34. In addition to sources cited earlier, see Stathis Psillos, “Inference to the Best Explanation and Bayesianism,” in Induction and Deduction in the Sciences, ed. Friedrich Stadler (vol. 11 of the Institute of the Vienna Circle Yearbook) (Dordrecht: Springer, 2004), 83–92; Samir Okasha, “Van Fraassen’s Critique of Inference to the Best Explanation,” Studies in the History and Philosophy of Science 31 (2000): 691–710.

35. See Ronald J. Allen, “Factual Ambiguity and the Theory of Evidence,” Northwestern University Law Review 88 (1994): 604–640; Ronald J. Allen and Michael S. Pardo, “Relative Plausibility and Its Critics,” International Journal of Evidence and Proof 23 (2019): 5–59; Michael S. Pardo and Ronald J. Allen, “Judicial Proof and the Best Explanation,” Law and Philosophy 27 (2008): 223–268; Nancy Pennington and Reid Hastie, “Explaining the Evidence: Tests of the Story Model for Juror Decision Making,” Journal of Personality and Social Psychology 62 (1992): 189–206; Pennington and Hastie, “A Cognitive Theory of Juror Decision Making: The Story Model,” Cardozo Law Review 13 (1991): 519–558; Pennington and Hastie, “Explanation-Based Decision Making: Effects of Memory Structure on Judgment,” Journal of Experimental Psychology: Learning, Memory, and Cognition 14 (1988): 521–533.

36. Susan Haack, “Proving Causation: The Holism of Warrant and the Atomism of Daubert,” Journal of Health and Biomedical Law 4 (2008): 253–289.

37. Lipton, Inference to the Best Explanation, at 1.

Chapter 3. The Burden of Proof

1. To the contrary is Richard H. Gaskins, Burdens of Proof in Modern Discourse (New Haven, CT: Yale University Press, 1992), 1–6, 108–114, equating ignorance with uncertainty.

2. For some philosophers, to know something is to be in such a state of confidence that it would be redundant to say “know for certain,” precisely because “know” just means “know for certain.” You only know something, under this view, if it would be inconceivable for it to be otherwise—if all contrary possibilities can be definitively ruled out. Descartes is the canonical source, and generations of philosophy students have been schooled in this absolutist conception of knowledge. See John Hospers, An Introduction to Philosophical Analysis, rev. 2nd ed. (London: Routledge and Kegan Paul, 1967), 153; Matthias Steup and Ram Neta, “Epistemology,” Stanford Encyclopedia of Philosophy, https://plato.stanford.edu/archves/fall2020/entries/epistemology/ (2020). This conception pervades the professional philosophical landscape—see Alvin I. Goldman, Knowledge in a Social World (Oxford: Clarendon Press, 1999), 23, who describes this as knowledge in a “strong” sense—but it is not dominant in nonphilosophical discourse, and is not the conception I use in this book. Rather, references to knowledge here and in most of the world are to what Goldman sensibly calls W-knowledge (knowledge in the “weak” sense), and which Goldman uses in his book. I know that it is usually cold in Vermont in January, that my car is a 2019 Subaru Impreza, that my class meets at 10:10 a.m. on Mondays and Wednesdays, that I was born on January 15, and that Kamala Harris is now vice president of the United States. Such knowledge may not count as knowledge for some philosophers for some philosophical purposes, but it is noteworthy that experimental research has found that the philosophers’ conception of knowledge differs not only from that of laypeople, but also from the usage in most other academic disciplines. Nat Hansen, J. D. Porter and Kathryn Francis, “A Corpus Study of ‘Know’: On the Verification of Philosophers’ Frequency Claims about Language,” Episteme 18 (2021): 242–268; Christina Starmans and Ori Friedman, “Expert or Esoteric? Philosophers Attribute Knowledge Differently than All Other Academics,” Cognitive Science 44 (2020). See also Michael S. Pardo, “Epistemology, Psychology, and Standards of Proof: An Essay on Risinger’s ‘Surprise’ Theory,” Seton Hall Law Review 48 (2018): 1039–1055; Pardo, “The Gettier Problem and Legal Proof,” Legal Theory 16 (2010): 37–58.

3. The legal system’s historical aversion to using numbers to portray or operationalize “beyond a reasonable doubt” is defended in United States v. Hall, 854 F.2d 1036, 1043 (7th Cir. 1988) (Richard A. Posner J.); Ronald J. Allen and Alex Stein, “Evidence, Probability, and the Burden of Proof,” Arizona Law Review 55 (2013): 557–602; Laurence H. Tribe, “Trial by Mathematics: Precision and Ritual in the Legal Process,” Harvard Law Review 84 (1971): 1329–1393. Contrary arguments, favoring translating such imprecise ideas into more precise formulas, whether in the legal system or otherwise, include Jennifer Rose Carr, “Imprecise Evidence without Imprecise Credences,” Philosophical Studies 177 (2020): 2735–2758; Joe Fore, “ ‘A Court Would Probably Find …’: Defining Probability Expressions in Predictive Legal Analysis,” Legal Communication and Rhetoric: JALWD 16 (2019): 49–84; Frederick Mosteller and Cleo Youtz, “Quantifying Probabilistic Assessments,” Statistical Science 5 (1990): 2–12; Peter Tillers and Jonathan Gottfried, “United States v. Copeland: A Collateral Attack on the Legal Maxim That Proof Beyond a Reasonable Doubt Is Unquantifiable,” Law, Probability and Risk 5 (2006): 135–157; [Judge] Jon O. Newman, “Taking ‘Beyond a Reasonable Doubt’ Seriously,” Judicature 103, no. 2 (2019). And see Judgment by the Numbers: Converting Qualitative to Quantitative Judgments in Law, special issue of Journal of Empirical Legal Studies 8 (2011): 1–97.

4. Whether instructions to juries in criminal cases should include numerical percentages is a very different question, implicating the psychology of communication and the psychology of jury deliberation. I leave that question aside here.

5. See Georgi Gardner, “In Defence of Reasonable Doubt,” Journal of Applied Philosophy 34 (2017): 221–241; David Kaye, “Laws of Probability and the Law of the Land,” University of Chicago Law Review 47 (1979): 34–56; C. M. A. McCauliff, “Burdens of Proof: Degrees of Belief, Quanta of Evidence, or Constitutional Guarantees?,” Vanderbilt Law Review 35 (1982): 1293–1336; Anne W. Martin and David A. Schum, “Quantifying Burdens of Proof: A Likelihood Ratio Approach,” Jurimetrics 27 (1987): 383–401; [Judge] Jack B. Weinstein and Ian Dewsbury, “Comment on the Meaning of ‘Proof Beyond a Reasonable Doubt,’ ” Law, Probability and Risk 5 (2006): 167–173. An influential critique of some implications of the beyond-a-reasonable-doubt standard is Larry Laudan, Truth, Error, and Criminal Law: An Essay in Legal Epistemology (Cambridge: Cambridge University Press, 2006). A critique of that critique is Raphael M. Goldman and Alvin I. Goldman, “Review of Truth, Error, and Criminal Law: An Essay in Legal Epistemology, by Larry Laudan,” Legal Theory 15 (2009): 55–66.

6. Not all. See Rita James Simon and Linda Mahan, “Quantifying Burdens of Proof: A View from the Bench, the Jury, and the Classroom,” Law and Society Review 5 (1971): 319–330, finding lower estimates based on surveys.

7. As we will see in Chapter 4, things are not quite this simple. To recover against Jill, Jack must prove (1) that Jill was negligent, (2) that Jack was injured, and (3) that Jill’s negligence caused Jack’s injuries; and if Jack must prove each of these by a preponderance of the evidence, then Jack’s burden turns out to be greater than a bare preponderance of the evidence because Jack will lose if he fails to prove any one of these three elements. Conversely, if Jack need only prove that the conjunction of these three elements is true by a preponderance of the evidence, then his burden—for any one of the three can be lower than a preponderance. This is the conjunction problem, and it has been well-massaged by scholars of the law of evidence. See Dale A. Nance, The Burden of Proof: Discriminatory Power, Weight of Evidence, and the Tenacity of Belief (Cambridge: Cambridge University Press, 2016), 75–77; David S. Schwartz and Elliott R. Sober, “The Conjunction Problem and the Logic of Jury Findings,” William and Mary Law Review 59 (2017): 619–692; Mark Spottswood, “Unraveling the Conjunction Paradox,” Law, Probability and Risk 15 (2016): 259–296; Bartosz W. Wojciechowski and Emmanuel M. Pothos, “Is There a Conjunction Fallacy in Legal Probabilistic Decision Making?,” Frontiers in Psychology 9 (2018): 391–401.

8. The trial inspired an outpouring of books, most of them dreadful. Two of the more reliable accounts are Darnell M. Hunt, O.J. Simpson Facts and Fictions: News Ritual in the Construction of Reality (Cambridge: Cambridge University Press, 1999), and Jeffrey Toobin, The Run of His Life: The People v. O.J. Simpson (New York: Random House, 1996).

9. See B. Drummond Ayres Jr., “Civil Jury Finds Simpson Liable in Pair of Killings,” New York Times, Feb. 5, 1997, A1; Richard Winton, “Kim Goldman’s Crusade: Make O.J. Simpson Pay and Never Forget,” Los Angeles Times, June 12, 2019.

10. Although the civil jury and not the criminal jury was told about threatening statements O.J. Simpson had made to Nicole Brown Simpson, there is little indication that admitting such statements in the criminal trial would have changed the outcome.

11. On the Scottish option and its effect, see Rachel Ormston et al., Scottish Government, Scottish Jury Research: Findings from a Large Scale Mock Jury Study, at www.gov.scot (Oct. 9, 2019).

12. In Bostock v. Clayton County, Georgia, 140 S. Ct. 1731 (2019), the Supreme Court ruled that Title VI’s prohibition on sex discrimination in employment prohibited discrimination on the basis of sexual orientation regardless of the sex of the victim. Presumably the same applies to Title IX as well.

13. For accounts of the background law and the issues described here, see SurvJustice Inc. v. DeVos, 2019 WL5684522 (N.D. Cal., Nov. 1, 2019); Ilana Frier, “Campus Sexual Assault and Due Process,” Duke Journal of Constitutional Law and Public Policy Sidebar 15 (2020): 117–143; William C. Kidder, “(En)Forcing a Foolish Consistency? A Critique and Comparative Analysis of the Trump Administration’s Proposed Standard of Evidence Regulation for Campus Title IX Proceedings,” Journal of College and University Law 45 (2020): 1–47; Sarah Swan, “Discriminatory Dualism in Process: Title IX, Reverse Title IX, and Campus Sexual Assault,” Oklahoma Law Review 73 (2020): 69–99.

14. See United States v. Comstock, 560 U.S. 126 (2010); Addington v. Texas, 441 U.S. 418 (1979).

15. The phrase the Supreme Court used was “convincing clarity,” which says the same thing with a slightly different grammatical structure. New York Times Co. v. Sullivan, 376 U.S. 254 (1964). The same burden of proof applies when the plaintiff seeks to prove, not actual knowledge of falsity, but “reckless disregard” of the truth, the latter requiring a failure to investigate in the face of demonstrated “serious doubts” about the truth of the publication. St. Amant v. Thompson, 379 U.S. 64 (1968).

16. William Blackstone, Commentaries on the Laws of England (London: 1769), as published in facsimile by University of Chicago Press (1979), 352.

17. Many of these other ratios are collected in Daniel Pi, Francesco Parisi, and Barbara Luppi, “Quantifying Reasonable Doubt,” Rutgers University Law Review 72 (2020): 455–508; Frederick Schauer and Richard Zeckhauser, “On the Degree of Confidence for Adverse Decisions,” Journal of Legal Studies 25 (1996): 27–52, at 34n11; Alexander Volokh, “n Guilty Men,” University of Pennsylvania Law Review 146 (1997): 173–212.

18. Recent years have seen efforts to identify mistaken convictions and provide after-the-fact exoneration. See Brandon L. Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong (Cambridge, MA: Harvard University Press, 2011). But a mistaken conviction is not necessarily a product of mistaken institutional design. Changing the procedures to lessen the number of mistaken convictions will likely increase the number of mistaken acquittals, and then we are back to the Blackstonian task of determining how many mistaken acquittals we are willing to tolerate in order to achieve how much of a reduction in the number of mistaken convictions. For Blackstone, that determination did not involve any question of racially disproportionate effects, a question that now properly pervades the contemporary application of the Blackstonian calculus. Moreover, accepting the inevitability of some number of mistaken convictions does not preclude compensation or other redress when those mistakes become apparent in individual cases. See Erik Encarnacion, “Why and How to Compensate Exonerees,” Michigan Law Review First Impressions 114 (2016): 139–154.

19. Ramos v. Louisiana, 140 S. Ct. 1390 (2020).

20. See Laudan, Truth, Error, and the Criminal Law. Moreover, Laudan argues that we should include in our calculus not only the mistakes. A proper analysis, he insists, would also include the benefits of accurate convictions and accurate acquittals, and these benefits should be weighed against the costs of mistaken acquittals and mistaken convictions. In expanding our understanding of the consequences of both correct and incorrect decisions in this way, he concludes, we would wind up with a more socially beneficial assessment of the proper burden of proof. Moreover, it would be a burden of proof that properly varies with the nature (and thus severity) of the crime charged, unlike the uniform approach now in force.

21. See Sarah Brown, “6 Things to Know About the New Title IX Guidance,” Chronicle of Higher Education, July 21, 2021; Alexis Gravely, “Thoughts from the Public on Title IX,” Inside Higher Ed, June 8, 2021, www.insidehighered.com. The entire issue remains in flux, however, largely because the Senate Committee on Health, Education, Labor, and Pensions refused, on August 3, 2021, to advance the nomination of Catherine Lhamon to be Assistant Secretary of the Office of Civil Rights of the Department of Education, the office in charge of Title IX enforcement. The burden of proof for Title IX proceedings was prominent in Lhamon’s hearings, with the Committee’s tie vote along partisan lines, leaving the issue open as of this writing.

22. An extensive historical and constitutional analysis is Thomas B. Ripy, “Standard of Proof in Senate Impeachment Proceedings,” Congressional Research Service Report 98-990 (Jan. 7 1999), available at https://crs.reports.congress.gov. More recent, but dealing less with the standard of proof, is Jared P. Cole and Todd Garvey, “Impeachment and the Constitution” (Nov. 20, 2019), https://crs.reports.congress.gov..

23. Compare the book by an international player so accused, Terence Reese, Story of an Accusation (New York: Simon and Schuster, 1966), with the book about the same episode by the accuser, Alan Truscott, The Great Bridge Scandal: The Most Famous Cheating Case in the History of the Game (New York: Exposition Press, 1969). The decision-making body applied a high standard of proof, close but not identical to “beyond a reasonable doubt,” and concluded, in the style of the Scottish “not proven” verdict, that the accusation had not been “supported.”

24. In American professional football, the standard for reversing an on-the-field call by a referee is “unmistakable visual evidence,” which seems analogous to “beyond a reasonable doubt.”

25. See Andrew Ashworth and Lucia Zedner, Preventive Justice (Oxford: Oxford University Press, 2014), 200–203.

26. For the view that a presupposed moral and political right to liberty raises the burden of proof for such restrictions, see Eric Winsberg, Jason Brennan, and Chris W. Surprenant, “How Government Leaders Violated Their Epistemic Duties during the SARS-CoV-2 Crisis,” Kennedy Institute of Ethics Journal 30 (2020): 215–242. But note that Winsberg et al.’s burden-raising right to general freedom of movement is not a right recognized in existing American legal doctrine. See Carmichael v. Ige, 470 F.3d 1133 (D. Haw. 2020).

27. See Roger Przybylski, Recidivism of Adult Sexual Offenders, published by the Office of Sex Offender Sentencing, Monitoring, Apprehending, Registering, and Tracking, US Department of Justice, Office of Justice Programs (2015), available at www.smart.ojop.gov.

28. One reason for not fully exploring the issues here is to avoid repeating what is in Frederick Schauer, Profiles, Probabilities, and Stereotypes (Cambridge, MA: Harvard University Press, 2003).

29. In addition to Ashworth and Zedner, Preventive Justice, see the various perspectives reflected in Andrew Ashworth, Lucia Zedner, and Patrick Tomlin, eds., Prevention and the Limits of the Criminal Law (Oxford: Oxford University Press, 2013), and also in Sandra G. Mayson, “In Defense of Consequentialist Prevention,” Criminal Law and Philosophy 15, no. 1 (April 2021).

30. See Jennifer Carlson, “Gun Studies and the Politics of Evidence,” Annual Review of Law and Social Science 16 (2020): 183–202.

31. The evidentiary dimensions of violent interactive videogames were debated in the Supreme Court. Brown v. Entertainment Merchants Ass’n, 564 U.S. 786 (2011). Because restrictions on such games implicate freedom of speech under the First Amendment, the Supreme Court presupposed that the burden of proof necessary to justify a restriction was higher than it would have been to justify a more typical (and not constitutionally problematic) restriction of personal liberty. Differently put, but consistent with the decision theory of the burden of proof, the First Amendment commanded that the mistake of an unwarranted restriction be treated as worse than the mistake of an unwarranted nonrestriction. And the First Amendment also commanded that the mistake of an unwarranted restriction be treated as worse than an unwarranted restriction on activity not implicating the First Amendment (or other constitutional rights).

32. When we seek to attribute responsibility for a particular past act, whether that attribution takes place in the legal system or otherwise, we often use a deterministic conception of causation. Under this view, event C is a cause of consequence (effect) E if and only if E would not have happened but for C, and only if C always produces E. But when we are making policy about future acts, as with making policy about cigarettes or guns or opioids, such deterministic conceptions of causation are of little assistance, and probabilistic causation is the almost universal understanding. See Frederick Schauer and Barbara A. Spellman, “Probabilistic Causation and the Law,” Journal of Institutional and Theoretical Economics 176 (2020): 4–17. The philosophical literature on probabilistic causation includes Patrick Suppes, A Probabilistic Theory of Causality (Amsterdam: North Holland, 1970); Nancy Cartwright, “Causal Laws and Effective Strategies,” Noûs 13 (1979): 419–437; Ellery Eels, “Probabilistic Causal Interaction,” Philosophy of Science 53 (1986): 52–64; I. J. Good, “A Causal Calculus I–II,” British Journal for the Philosophy of Science 11 (1961): 305–318; David Papineau, “Probabilities and Causes,” Journal of Philosophy 82 (1985): 57–74; Wesley Salmon, “Probabilistic Causality,” Pacific Philosophical Quarterly 31 (1980): 50–74; Deborah Rosen, “In Defence of a Probabilistic Theory of Causality,” Philosophy of Science 45 (1978): 604–613. A comprehensive and accessible statistical analysis is A. Philip Dawid, “Statistical Risk,” Synthese 194 (2017): 3445–3474.

33. In the environmental context, the principle holds that “where there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures.” Principle 15 of the UN Rio Declaration on Environment and Development, available at http://www.un.org/documents/ga/conf151/aconf15126-1annex1.htm. See Jane Holder and Maria Lee, Environmental Protection, Law and Policy, 2nd ed. (Cambridge: Cambridge University Press, 2008), 15–34; Marco Martuzzi and Joel A. Tickner, The Precautionary Principle: Protecting Public Health, the Environment and the Future of Our Children (Copenhagen: World Health Organization, 2004); Elizabeth Fisher, “Precaution, Precaution Everywhere: Developing a ‘Common Understanding’ of the Precautionary Principle in the European Community,” Maastricht Journal of European and Comparative Law 9 (2002): 7–28. Skeptical analyses include Cass R. Sunstein, Laws of Fear: Beyond the Precautionary Principle (Cambridge: Cambridge University Press, 2005); Aaron Wildavsky, But Is It True? A Citizen’s Guide to Environmental Health and Safety Issues (Cambridge, MA: Harvard University Press, 1997).

34. For an extensively referenced overview of the controversies, see Daniel Steel, “The Precautionary Principle and the Dilemma Objection,” Ethics, Policy and Environment 16 (2013): 321–340.

35. See Klaus Messerschmidt, “COVID-19 Legislation in the Light of the Precautionary Principle,” Theory and Practice of Legislation 8 (2020): 267–292.

36. A sampling of the more than fifty lawsuits, all dismissed, would include Texas v. Pennsylvania, No. 155, Orig. (Sup. Ct., Dec. 11, 2020); Trump for President, Inc. v. Secretary of Commonwealth of Pennsylvania, 2020 WL 7012522 (3d Cir., Nov. 27, 2020); Trump v. Wisconsin Elections Commission, 2020 WL 7318940 (E.D. Wis., Dec. 12, 2020); Bowyer v. Ducey, 2020 WL 7238261 (D. Ariz., Dec. 9, 2020); King v. Whitmer, 2020 WL 7134198 (E.D. Mich., Dec. 7, 2020); Wood v. Raffensperger, 2020 WL 6817153 (N.D. Ga. Nov. 20, 2020).

37. An example of the common view—that “proof” is in some unspecified way stronger than evidence, perhaps by being strong enough to justify a legal judgment—is a 1956 how-to-do-it book by Marshall Houts entitled From Evidence to Proof: A Searching Analysis of Methods to Establish Fact (Springfield, IL: Charles Thomas, 1956). More recently, see Beth A. Bechky, Blood, Power, and Residue: How Crime Labs Translate Evidence into Proof (Princeton, NJ: Princeton University Press, 2021).

38. See Naomi Oreskes and Erik M. Conway, Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Climate Change (New York: Bloomsbury USA, 2010).

39. Diane Caruana, “Latest Study Finds No Conclusive Evidence that Vaping Leads to Smoking,” Apr. 21, 2020, www.vapingpost.com.

40. See M. R. Avery et al., “Mechanisms of Influence: Alcohol Industry Submissions to the Inquiry into Fetal Alcohol Disorders,” Drug and Alcohol Review 35 (2016): 665–672; International Alliance for Responsible Drinking, “No Conclusive Evidence of Link between Occasional, Light, or Moderate Drinking and Fetal Alcohol Spectrum Disorders,” www.iard.org.

41. “No Definitive Proof of Effect of Video Games on Work Time,” at www.nintendoenthusiast.com.

42. Brown v. Entertainment Merchants Ass’n, 564 U.S. 786 (2011). In most of the world, the idea that free speech concerns are even relevant to violent video games would be considered bizarre. But the United States is a protective outlier on free speech issues compared to the rest of the world, even the rest of the liberal democratic industrialized world. See Frederick Schauer, “The Exceptional First Amendment,” in American Exceptionalism and Human Rights, ed. Michael Ignatieff (Princeton, NJ: Princeton University Press, 2005), 29–56. The view that regulation of such games must satisfy a First Amendment–elevated burden of justification was common ground even among the justices who disagreed about the evidence. On the effects of violent video games generally, compare American Psychological Association, “APA Task Force on Violent Media: Technical Report on the Review of the Violent Video Game Literature,” https://www.apa.org/pi/families/review-video-ga,es.pdf (2015), with Christopher F. Ferguson, Allen Copenhaver, and Patrick Markey, “Reexamining the Findings of the American Psychological Association’s 2015 Task Force on Violent Media: A Meta-Analysis,” Perspectives on Psychological Science 15 (2020): 1423–1443.

43. To be clear, there is no claim here that measures of statistical significance map easily onto measures of the burden of proof. See Michelle M. Burtis, Jonah B. Gelbach, and Bruce H. Kobayashi, “Error Costs, Legal Standards of Proof, and Statistical Significance,” Supreme Court Economic Review 25 (2017): 1–58; Michael S. Pardo, “A Comment on Statistical Significance and Standards of Proof,” Supreme Court Economic Review 25 (2017): 59–63.

44. See Valentin Amrhein, Sander Greenland, and Blake McShane, “Scientists Rise Up against Statistical Significance,” www.nature.com (March 20, 2019), reporting on Blakey B. McShane, David Gal, Andrew Gelman, Christian Robert, and Jennifer L. Tackett, “Abandon Statistical Significance,” American Statistician 73 (2019): 235–245.

Chapter 4. How to Tell the Truth with Statistics

1. Darrell Huff, How to Lie with Statistics (New York: W. W. Norton, 1954).

2. The assumptions are added in order to avoid the worst forms of the so-called reference class problem. If the reliability assessments for Subarus for a particular model year are based on the average of various different models, some of which are especially reliable and others of which are not, then using the all-models average for an inference about the especially unreliable model, or even about an unspecified model, is unwarranted. But if all Subarus are relevantly similar, then the reference class problem of moving from this class to a representative member of this class is substantially eliminated. See Edward K. Cheng, “A Practical Solution to the Reference Class Problem,” Columbia Law Review 109 (2009): 2081–2105; Alan Hájek, “The Reference Class Problem Is Your Problem Too,” Synthese 156 (2007): 563–585; Christian Wallman and Jon Williamson, “Four Approaches to the Reference Class Problem,” in Making It Formally Explicit: Probability, Causality and Indeterminism, ed. Gábor Hofer-Szabó and Leszek Wronski (Cham, Switzerland: Springer, 2017), 61–81.

3. For the moment we are talking about individual attributes—solid or striped—and not about causation, relationships, or behavior. Things get more complicated when we are talking about causation: Is the fact that 90 percent of cases of lung cancer are caused by cigarette smoking evidence that this particular case of lung cancer, about which we know nothing more, was caused by cigarette smoking? And even more complications arise when the class that produces the aggregate data has nonidentical subclasses. This is the ecological inference problem, and statisticians and others have long debated whether individual-level causation or other individual-level relationships or patterns of behavior can be inferred from group-level data about those relationships and behavior. An important overview and proposed solution are in Gary King, A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior from Aggregate Data (Princeton, NJ: Princeton University Press, 1997). See also Gary King, Ori Rosen, and Martin A. Tanners, eds., Ecological Inference: New Methodological Strategies (Cambridge: Cambridge University Press, 2010); Nancy Cleave, Philip J. Brown, and Clive D. Payne, “Evaluation of Methods for Ecological Inference,” Journal of the Royal Statistical Society 158 (1995): 55–72; Lutz Erbring, “Individual Writ Large: An Epilogue on the ‘Ecological Fallacy,’ ” Political Analysis 1 (1989): 235–269; D. James Greiner, “Causal Inference in Civil Rights Litigation,” Harvard Law Review 122 (2008): 533–598; Jon Wakefield, “Ecological Inference in the Social Sciences,” Statistical Methodology 7 (2010): 307–322. The ecological inference problem aside, the use in the legal system of aggregate (statistical) data as evidence of specific causation is both common and defensible. See Susan Haack, “Risky Business: Statistical Proof of Specific Causation,” in Evidence Matters: Science, Proof, and Truth in the Law (Cambridge: Cambridge University Press, 2014), 264–293.

4. As Laurence Tribe, who objects to the use of quantified statistical evidence in trials, puts it, “All factual evidence is ultimately ‘statistical,’ and all legal proof ultimately ‘probabilistic,’ in the epistemological sense that no conclusion can ever be drawn from empirical data without some step of inductive inference—even if only an inference that things are usually what they are perceived to be.” Laurence H. Tribe, “Trial by Mathematics: Precision and Ritual in the Legal Process,” Harvard Law Review 84 (1971): 1329–1393, at 1330. And note, consistent with Tribe’s view, that saying that evidence is probabilistic says nothing about whether those probabilities can or should be reduced to, or expressed in terms of, numbers.

5. People v. Collins, 438 P.2d 33 (Cal. 1968).

6. Smith v. Rapid Transit, 58 N.E.2d 754 (Mass. 1945); Tribe, ” Trial by Mathematics,” at 1340–1341, 1346–1350.

7. See Emily Berman, “Individualized Suspicion in the Age of Big Data,” Iowa Law Review 105 (2020): 463–506, at 475–476; Edward K. Cheng, “Reconceptualizing the Burden of Proof,” Yale Law Journal 122 (2013): 1254–1279; David Enoch, Levi Spectre, and Talia Fisher, “Statistical Evidence, Sensitivity, and the Legal Value of Knowledge,” Philosophy and Public Affairs 40 (2012): 197–224; Michael S. Pardo, “The Paradoxes of Legal Proof: A Critical Guide,” Boston University Law Review 99 (2019): 233–290. A good overview of the literature is Enrique Guerra-Pujol, “Visualizing Probabilistic Proof,” Washington University Jurisprudence Review 7 (2014): 39–75. To the same effect as Smith v. Rapid Transit, but on more complex facts, is Day v. Boston & Me. R.R., 52 A. 771 (Me. 1902), mysteriously distinguishing probabilistic from “real” evidence.

8. L. Jonathan Cohen, The Probable and the Provable (Oxford: Clarendon Press, 1977), 73–76.

9. See, in addition to the above references, Martin Smith, “When Does Evidence Suffice for Conviction?,” Mind 127 (2018): 1193–1218; Gary L. Wells, “Naked Statistical Evidence of Liability: Is Subjective Probability Enough?,” Journal of Personality and Social Psychology 62 (1992): 739–752.

10. Charles R. Nesson, “Reasonable Doubt and Permissive Inferences: The Value of Complexity,” Harvard Law Review 92 (1979): 1187–1225, at 1192–1196.

11. In addition to references already cited, see Michael Blome-Tillman, “Statistical Evidence, Normalcy, and the Gatecrasher Paradox,” Mind 129 (2020): 563–577; Marcello Di Bello, “Trial by Statistics: Is a High Probability of Guilt Enough to Convict?,” Mind 128 (2019): 1045–1084; David Enoch and Talia Fisher, “Sense and ‘Sensitivity’: Epistemic and Instrumental Approaches to Statistical Evidence,” Stanford Law Review 67 (2015): 557–611; Georgi Gardiner, “Legal Burdens of Proof and Statistical Evidence,” in Routledge Handbook of Applied Epistemology, ed. David Coady and James Chase (London: Routledge, 2018), 179–195; David Kaye, “The Paradox of the Gatecrasher and Other Stories,” Arizona State Law Journal 1979 (1979): 101–143; Sarah Moss, “A Knowledge Account of Legal Proof,” in Oxford Studies in Epistemology, vol. 7 (Oxford: Oxford University Press, 2021); Mike Redmayne, “Exploring the Proof Paradoxes,” Legal Theory 14 (2008): 281–309.

12. This example was suggested to me by David Enoch and Levi Spectre.

13. It is worth noting, again, that the alleged paradox does not depend on the use or non-use of actual numbers. If, as in the real Smith v. Rapid Transit case, the evidence was only that the vast majority of the buses operating on that route were operated by a specific company, that would still, for some people, create a problem, as it did for the court, because the intuition is that the problem is created by the lack of individuation and not by the presence of statistics in numerical form.

14. In the case on which the Blue Bus hypothetical is based, the absence of any other evidence that this was Rapid Transit’s bus might be evidence that it was not. Most buses display their ownership quite prominently, and the failure to offer evidence about the bus’s ownership might raise suspicion—which would be evidence—that it was not actually a Rapid Transit bus. See James Brook, “The Use of Statistical Evidence of Identification in Civil Litigation: Well-Worn Hypotheticals, Real Cases, and Controversy,” St. Louis University Law Journal 29 (1985): 293–352, at 301–303.

15. One of the most rigorous defenses of a requirement for individualized evidence is Judith Jarvis Thomson, “Liability and Individualized Evidence,” in Rights, Restitution, and Risk: Essays in Moral Theory, ed. William Parent (Cambridge, MA: Harvard University Press, 1986), 225–250.

16. This generalization not only is apparently true, but also is a staple of popular culture, as exemplified by Michelle Wolf’s satirical “The Husband Did It” routine and a novel entitled It’s Always the Husband, by Michelle Campbell (New York: St. Martin’s Press, 2017).

17. See Michael J. Saks and Robert F. Kidd, “Human Information Processing and Adjudication: Trial by Heuristics,” Law and Society Review 15 (1981): 123–160.

18. The most prominent reference is Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Strauss and Giroux 2011). See also Maya Bar-Hillel, “The Base-Rate Fallacy in Probability Judgments,” Acta Psychologica 44 (1980): 211–233.

19. See Alex Broadbent, “Epidemiological Evidence in Proof of Specific Causation,” Legal Theory 17 (2011): 237–278; Melissa Moore Thompson, “Causal Inference in Epidemiology: Implications for Toxic Tort Litigation,” North Carolina Law Review 71 (1992): 247–291.

20. The case on which the Blue Bus hypothetical is based was real, but some complain that the made-up examples in the statistical evidence literature are without value because they could never arise in real life. Ronald J. Allen, “Naturalized Epistemology and the Law of Evidence,” Quaestio Facti 2 (2021): 253–284. But that worry is misguided. A controlled scientific experiment seeks knowledge by varying a single attribute in order to determine if that attribute makes a difference, and, outside the laboratory, analytic isolation does the same in a more qualitative context. Barbara A. Spellman, “In Defense of Weird Hypotheticals,” Quaestio Facti 2 (2021): 325–337. The fictional examples in the evidence literature are designed to isolate the factor of statistical probability and the permissible (or not) use of the characteristics of a class to support inferences about individual members of that class. Analytic isolation fosters close examination of how much difference, if any, this one factor makes, even if that factor—here, statistical likelihood—rarely appears alone, without other evidence, in the so-called real world. And so we use these artificial examples to learn about our non-artificial world in the way that an epidemiological experiment uses laboratory experiments to identify disease-causing or disease-curing agents that also might never appear alone outside of the laboratory.

21. This better-evidence-forcing view, applied to a wide range of evidentiary rules, is the central theme of Dale Nance, “The Best Evidence Principle,” Iowa Law Review 73 (1988): 227–298.

22. The literature on the base-rate fallacy includes Aron K. Barbey and Steven A. Sloman, “Base-Rate Respect: From Ecological Rationality to Dual Processes,” Behavioral and Brain Sciences 30 (2007): 241–254; Bar-Hillel, “The Base-Rate Fallacy”; Eugene Borgida and Richard Nisbett, “The Differential Impact of Abstract versus Concrete Information on Decisions,” Journal of Applied Social Psychology 7 (1977): 258–271; Baruch Fischhoff and Maya Bar-Hillel, “Diagnosticity and the Base-Rate Effect,” Memory and Cognition 12 (1984): 402–410; Keith J. Holyoak and Barbara A. Spellman, “Thinking,” Annual Review of Psychology 44 (1993): 265–315; Daniel Kahneman and Amos Tversky, “Subjective Probability: A Judgment of Representativeness,” Cognitive Psychology 3 (1972): 430–454.

23. The statement in the text is a slight overstatement. If in fact Henry committed these acts against these four accusers, then the four accusation are not truly statistically independent, because the existence of a common cause defeats independence. At the point at which we do not know of this common cause, however, the fact that the accusers do not know each other and do not know of the others’ accusations is sufficient independence for our purposes.

24. For a qualified defense of a qualified aggregation principle, see Alon Harel and Ariel Porat, “Aggregating Probabilities across Cases: Criminal Responsibility for Unspecified Offenses,” Minnesota Law Review 94 (2009): 261–308; Ariel Porat and Eric A. Posner, “Aggregation and Law,” Yale Law Journal 122 (2012): 2–68. An even more qualified defense is Frederick Schauer and Richard Zeckhauser, “On the Degree of Confidence for Adverse Decisions,” Journal of Legal Studies 25 (1996): 27–52. On the related puzzles where one act can violate multiple prohibitions, see Larry Alexander, “The Aggregation / Culpability Puzzle,” San Diego Legal Studies Paper 2-474, https://ssrn.com/abstract=3720171 (Dec. 18, 2020).

25. This issue differs from that in the infamous “Brides in the Bath” English case from 1915. There, George Joseph Smith had been prosecuted for murdering his wife by drowning her in a bathtub, and there was evidence that two of Smith’s previous wives had died under very similar circumstances. R. v. Smith, 11 Cr. App. 229, 84 L.J.K.B. 2153 (1915). The evidence was admitted on the theory that it was extraordinarily unlikely that three wives of the same man would die in such similar fashion, and thus that Smith likely committed all three of these murders, and not just one unspecified. In this respect the issues in Smith are similar to those in the Pennsylvania conviction of Bill Cosby, subsequently reversed on other grounds. Commonwealth v. Cosby, 252 A.3d 1092 (Pa. 2021). At least one of the claims in the Cosby case, a claim resembling the prosecution claims in Smith, was that the method that Cosby used to drug and then sexually assault multiple women were so unusual that one person—Cosby—must have committed all of them. Because the Pennsylvania Supreme Court reversed Cosby’s conviction on the grounds that he testified in a civil deposition in reliance on the prosecutor’s commitment not to prosecute criminally, a commitment the state did not honor, the court never reached any of the issues regarding the evidence of other acts.

26. Kevin Clermont finds such a possibility “startling” and “astounding.” Kevin M. Clermont, “Aggregation of Probabilities and Illogic,” Georgia Law Review 47 (2012): 165–180.

27. This section is a highly abbreviated version of the main themes in Frederick Schauer, Profiles, Probabilities, and Stereotypes (Cambridge, MA: Harvard University Press, 2003). Further reflection over almost twenty years would lead me to a few different conclusions, a few different arguments, and more than a few more felicitous ways of expressing those conclusions and arguments. But the principal claims still strike me as correct, and are summarized here.

28. Department of Commerce v. New York, 139 S. Ct. 2551 (2019).

29. Utah v. Evans, 536 U.S. 452 (2002); Department of Commerce v. United States House of Representatives, 525 U.S. 316 (1999).

Chapter 5. Testimony, and Not Only in Court

1. R. F. Atkinson, Knowledge and Explanation in History: An Introduction to the Philosophy of History (Ithaca, NY: Cornell University Press, 1978), 42.

2. On the language-based nature of testimony, and how language distinguishes testimonial from observational, perceptual, experiential knowledge, see Elizabeth Fricker, “The Epistemology of Testimony,” Proceedings of the Aristotelian Society, suppl. 61 (1987): 57–83.

3. Only recently in the history of philosophy have philosophers treated testimony as being of distinct philosophical interest, as opposed to being the philosophically uninteresting way in which philosophically interesting knowledge is transmitted. Fortunately, that neglect has been remedied. See, among other important treatments, C. A. J. Coady, Testimony: A Philosophical Study (Oxford: Oxford University Press, 1992); Jennifer Lackey, Learning from Words: Testimony as a Source of Knowledge (Oxford: Oxford University Press, 2008); Jennifer Lackey and Ernest Sosa, eds., The Epistemology of Testimony (Oxford: Oxford University Press, 2006); Richard Moran, The Exchange of Words (Oxford: Oxford University Press, 2018); Steven Shapin, A Social History of Truth (Chicago: University of Chicago Press, 1994); Jonathan Adler, “Epistemological Problems of Testimony,” Stanford Encyclopedia of Philosophy, https://plato.stanford.edu/entries/testimony-episprob/; Robert Audi, “The Place of Testimony in the Fabric of Knowledge and Justification,” American Philosophical Quarterly 34 (1997): 405–422; Elizabeth Fricker, “ ‘Believing the Speaker’ versus Believing on Evidence: A Critique of Moran,” European Journal of Philosophy 27 (2019): 267–276; Fricker, “The Epistemology of Testimony,” Proceedings of the Aristotelian Society, suppl. 61 (1987): 57–83; Peter Lipton, “The Epistemology of Testimony,” Studies in the History and Philosophy of Science 29 (1998): 1–31; Angus Ross, “Why Do We Believe What We Are Told?,” Ratio 28 (1986): 69–88.

4. See Jay A. Soled, “Exploring and (Re)Defining the Boundaries of the Cohan Rule,” Temple Law Review 79 (2006): 939–970.

5. Cohan v. Commissioner of Internal Revenue, 39 F.2d 540 (2d Cir. 1930).

6. Prior to becoming an appellate judge, Hand had spent fifteen years as a federal trial judge. We can speculate that these fifteen years of listening to oral testimony as the principal form of courtroom evidence made Hand particularly attuned to the way in which oral testimony can count as evidence.

7. 26 U.S.C. §274 (2018).

8. See, more recently than Cohan, and more recently than the 1962 changes, La Forge v. Commissioner of Internal Revenue, 434 F.2d 370 (2d Cir. 1970); Baker v. Commissioner of Internal Revenue, T.C. Memo. 2014-122 (2014); Mertens Law of Federal Income Taxation §25F:12 (Dec. 2020 update).

9. Sometimes, trusting the testimony of another is based on personal commitments that have little to do with truth and little to do with reliance on the veracity of the testifier. See Philip J. Nickel, “Trust and Testimony,” Pacific Philosophical Quarterly 93 (2012): 301–316. I trust my often-dissembling friend, and thus take his word for something, only because he is my friend and not because his saying something provides very good evidence for its truth. But here we are dealing with epistemic trust—actual belief in the truth of what someone has said just because of the knowledge that we assume lies behind their testimony. Epistemic trust is to be distinguished from what we can think of as social or moral trust.

10. The qualification in the text is an oblique reference to the theory of speech acts, which recognizes that much speech does not assert anything, and therefore does not make the kind of statement that there can be evidence for, or that is evidence for something else. To take an example from J. L. Austin, the founder of modern speech act theory, saying “I do” at a wedding ceremony is to engage in an act for which the very idea of evidence is inapt. Once two people each say “I do” under appropriately specified circumstances, they are married by virtue of the saying, and nothing about that saying can be true or false. Just saying it makes it so. J. L. Austin, How to Do Things with Words, ed. J. O. Urmson and Marina Sbisà, 2nd ed. (Cambridge, MA: Harvard University Press, 1975).

11. See Nicholas Fandos, “Herrera Beutler Says McCarthy Told Her Trump Sided with Capitol Mob,” New York Times, Feb. 13, 2021.

12. Maggie Haberman, “First Lady Stayed in N.Y.C. to Get New Deal, Book Says,” New York Times, June 14, 2020, A29.

13. The hearsay rule retains considerable vitality in criminal cases as a result of the requirement in the Sixth Amendment that defendants have the opportunity to “confront” the witnesses against them. As a result, some out-of-court statements that might be admissible under some exception to the hearsay rule are excluded by application of the Sixth Amendment. The modern reinvigoration of this aspect of the confrontation clause began with Crawford v. Washington, 541 U.S. 36 (2004), and has generated considerable litigation and numerous Supreme Court decisions since.

14. The figure of thirty-one is derived by adding the twenty-three unqualified exceptions in Rule 803 to the six exceptions in Rule 804 that require that the maker of the original statement (declarant) be unavailable to the two in Rule 801 that are treated as not being hearsay at all even though they were so treated traditionally and historically.

15. See Frederick Schauer, “On the Supposed Jury-Dependence of Evidence Law,” University of Pennsylvania Law Review 155 (2006): 165–202.

16. Were this an actual question in an actual court, or even if anything turned on it outside of a court hearing, there might well be reasons to doubt the article’s message that Melania Trump does not speak French. For one thing, there is little reason to believe that either the book author or the Times reporter are sympathetic to the Trump family. As a result, their lack of sympathy might color their interpretations of what others have said. Moreover, the photographers might be French, and thus might embody the typical French attitude about the Francophone abilities of non-native speakers who believe they speak French. Furthermore, there might be evidence inclining against the tenor of the Times article and the Jordan book. Even if the hearsay evidence of what the photographers believed is evidence against the proposition that she speaks French, other evidence might support that proposition. For example, someone from Slovenia, a country whose language is not widely spoken outside of its borders, and who operates professionally in international circles, as Ms. Trump did in her earlier years, is perhaps especially likely to have some competence in the major European languages. And although this statistical likelihood is hardly conclusive on the question of the linguistic abilities of a particular Slovene, it is still a piece of evidence for that conclusion.

17. Traditionally the example is of someone saying “I am Napoleon.” Commonwealth v. Whitman, 901 N.E.2d 1206 (Mass. 2009); Edward W. Hinton, “States of Mind and the Hearsay Rule,” University of Chicago Law Review 1 (1934): 394–423, at 397.

18. Austin, How to Do Things with Words, 5.

19. James Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economics, Societies and Nations (New York: Anchor Books, 2004).

20. The research inclining against the wisdom of crowds hypothesis includes Irving L. Janis, Victims of Groupthink: A Psychological Study of Foreign-Policy Decisions and Fiascoes, 2nd ed. (Boston: Wasdsworth, 1982); Norbert Kerr, Robert MacCoun, and Geoffrey P. Kramer, “Bias in Judgment: Comparing Individuals and Groups,” Psychological Review 103 (1996): 687–719; Jan Lorenz, Heiko Rauhut, Frank Schweitzer, and Dirk Helbing, “How Social Influence Can Undermine the Wisdom of Crowds Effect,” Proceedings of the National Academy of Sciences 108 (2011): 9020–9025; Garold Stasser and William Titus, “Pooling of Unshared Information in Group Decision Making: Biased Information Sampling during Discussion,” Journal of Personality and Social Psychology 48 (1985): 1467–1478.

21. For overviews of the research, see Thomas W. Malone and Michael S. Bernstein, Handbook of Collective Intelligence (Cambridge, MA: Harvard University Press, 2015); Anita Williams Woolley, Ishani Aggarwal, and Thomas W. Malone, “Collective Intelligence and Group Performance,” Current Directions in Psychological Science 24 (2015): 420–424.

22. Among many accounts including the quoted words, see Anne Gearan and Josh Dawsey, “Trump Issued a Call to Arms. Then He Urged His Followers to ‘Remember This Day Forever,’ ” Washington Post, Jan. 6, 2021.

23. John Milton, Areopagitica: A Speech for the Liberty of Unlicensed Printing, to the Parliament of England (1644), in John Milton, Prose Writings, Everyman’s Library (New York: Dutton, 1958).

24. John Stuart Mill, On Liberty, ed. David Spitz (New York: W. W. Norton, 1975) (orig. pub. 1859), chap. 2; Abrams v. United States, 250 U.S. 616, 630 (1919) (Holmes, J., dissenting). For illuminating background, see Thomas Healy, The Great Dissent: How Oliver Wendell Holmes Changed His Mind—and Changed the History of Free Speech in America (New York: Henry Holt, 2013).

25. Anthony Lewis, Freedom for the Thought We Hate: A Biography of the First Amendment (New York: Basic Book, 2007), as quoted in Timothy Garton Ash, Free Speech: Ten Principles for a Connected World (New Haven, CT: Yale University Press, 2016), 76.

26. See Christine Smallwood, “Astrology in the Age of Uncertainty,” New Yorker, Oct. 28, 2019.

27. Of course there are genuine conspiracies. But the phrase “conspiracy theory,” as commonly used, describes a false belief in the kind of conspiracy that explains a large amount of a set of behaviors, the unifying explanation satisfying some psychological need in the believer for much of what the believer dislikes all to be part of some master plan concocted by some evil force. See Karen M. Douglas, Robbie M. Sutton, and Aleksandra Cichocka, “The Psychology of Conspiracy Theories,” Current Directions in Psychological Science 26 (2017): 538–542. Why people have such beliefs is beyond the scope of this book, but what is relevant here is that a conspiracy theory is one way in which various psychological and sociological mechanisms may produce the acceptance of falsity despite evidence of its falsity.

28. For my own thoughts on this question—the explanatory force of the truth of a proposition in determining whether the proposition will be accepted, see Frederick Schauer, “Free Speech, the Search for Truth, and the Problem of Collective Knowledge,” SMU Law Review 70 (2017): 231–252.

29. It would provide nice support for the skeptical point made here if it were true that lemmings follow each other to their death in an act of collective suicide. Alas, it is not, providing still another example of the fact that what “everyone knows” is often incorrect. See N. C. Stenseth and R. A. Ims, eds., The Biology of Lemmings (London: Academic Press, 1993).

30. See Federal Rule of Evidence 608(a); Michelson v. United States, 335 U.S. 469 (1948); Julia Simon-Kerr, “Credibility by Proxy,” George Washington Law Review 85 (2017): 152–225, at 177–183.

Chapter 6. Testing Testimony

1. For a comprehensive comparative and historical study of the oath, see Helen Silving, “The Oath: I,” Yale Law Journal 68 (1959): 1329–1390; and Silving, “The Oath: II,” Yale Law Journal 68 (1959): 1552–1577. See also Nadine Farid, “Oath and Affirmation in the Court: Thoughts on the Power of a Sworn Promise,” New England Law Review 40 (2006): 555–561; Ian Gallacher, “ ‘Swear Not at All’: Time to Abandon the Testimonial Oath,” New England Law Review 52 (2018): 247–299.

2. On the difficulty and consequent rarity of criminal prosecutions for perjury, see, among many sources, Barbara A. Babcock, “Taking the Stand,” William and Mary Law Review 35 (1993): 1–19, at 9; I. Bennett Capers, “Crime, Legitimacy, and Testilying,” Indiana Law Journal 83 (2008): 835–880; Roberto Suro and Bill Miller, “Perjury: A Tough Case to Make,” Washington Post, Sept. 24, 1998.

3. See Eugene Kiely, “Roger Stone’s Crimes,” at www.factcheck.org (Feb. 20, 2020).

4. Daniels Fund Ethics Initiative, “Martha Stewart’s Insider Trading Scandal,” at www.danielsethics.mgt.unm.edu (2011); Gerald Walpin, “Clinton’s Future: Can He Polish His Image and Keep His License to Practice Law?,” Hofstra Law Review 28 (1999): 473–491.

5. On exaggerating the probability of low-probability negative events, see Amos Tversky and Daniel Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” Science 185 (1974): 1124–1131; Tversky and Kahneman, “Advances in Prospect Theory: Cumulative Representation of Uncertainty,” Journal of Risk and Uncertainty 5 (1992): 297–323. See also Daniel Kahneman, Paul Slovic, and Amos Tversky, eds., Judgment under Uncertainty: Heuristics and Biases (Cambridge: Cambridge University Press, 1982); Petko Kusev, Paul van Schaik, Pater Ayton, and John Dent, “Exaggerated Risk: Prospect Theory and Probability Weighting in Risky Choice,” Journal of Experimental Psychology: Learning, Memory, and Cognition 35 (2009): 1487–1505; W. J. Wouter Botzen, Howard Kunreuther, and Erwann Michel-Kerjan, “Divergence between Individual Perceptions and Objective Indicators of Tail Risks: Evidence from Floodplain Residents in New York City,” Judgment and Decision Making 10 (2015): 365–385.

6. See Michael J. Saks and Barbara A. Spellman, The Psychological Foundations of Evidence Law (New York: NYU Press, 2016), 118–119.

7. For a similar categorization, see Stephen Mark Rosenbaum, Stephan Billinger, and Nils Stieglitz, “Let’s Be Honest: A Review of Experimental Evidence of Honesty and Truth-Telling,” Journal of Economic Psychology 45 (2014): 181–196.

8. See Nina Mazar, On Amir, and Dan Ariely, “The Dishonesty of Honest People: A Theory of Self-Concept Maintenance,” Journal of Marketing Research 45 (2008): 633–644. But compare Bruno Verschuere et al., “Registered Replication Report on Mazar, Amir, and Ariely (2008),” Advances in Methods and Practices in Psychological Science 1 (2018): 299–317. More generally, see Dan Ariely, The Honest Truth about Dishonesty: How We Lie to Everyone—Especially Ourselves (New York: HarperCollins, 2012).

9. United States v. Turner, 558 F.2d 46, 50 (2d Cir. 1977); Mazar et al., “Dishonesty of Honest People.”

10. In the wee hours of January 7, 2020, Democratic representative Conor Lamb of Pennsylvania provoked extraordinary and almost physical anger from some Republican colleagues for asserting that those colleagues had “lied” in alleging (or agreeing with others’ allegations) that there was substantial evidence of fraud in the 2020 presidential election. The anger of these Republican colleagues was apparently based on Lamb’s violation of a congressional norm against accusing one’s colleagues of lying, a norm that might be understood as based on a further underlying norm, akin to an oath, against intentional falsity on the floor of the House. So Lamb stood accused of accusing others of violating the equivalent of an oath, and things went downhill from there. See Timothy Bella and Lasteshia Beachum (Washington Post), “ ‘Sit Down!’ ‘No, You Sit Down!’ Democrat’s Speech Nearly Triggers Fistfight on House Floor,” Seattle Times, Jan. 7, 2021.

11. The pledge quoted in the text is the one in force at the US Military Academy at West Point. The first college honor code was adopted by the College of William and Mary in 1736, but it, like most others these days apart from those of the military academies, does not contain an obligation, implicit or explicit, to report violators.

12. This observation is adapted from Antony Flew, who in commenting on the “Socratic method” of instruction widespread in the Socratic dialogues and in American law schools, observed that instructors “who have attempted to follow Socratic method will have been made aware of the importance of the fact that Plato was able to script the answers as well as the questions.” Antony Flew, A Dictionary of Philosophy (New York: St. Martin’s Press, 1979), 306.

13. John Henry Wigmore, Evidence in Trials at Common Law, rev. J. H. Chadbourn (Boston: Little, Brown, 1974), §1367, at 32.

14. And even Wigmore, just a few sentences later, added that a “skillful” cross-examiner may “make the truth appear like a falsehood.”

15. The problem is (at least) twofold. First, a multipart question, typically coupled with a set of multipart assumptions and assertions, allows the responder to choose which questions to answer or which assumptions to accept and which to challenge, thus making evasion easier. And, second, the questioners at press conferences, unwilling to share their minute in the limelight, rarely ask follow-up questions to someone else’s question. Good cross-examination is the art of the follow-up, making evasion more difficult. But if the questioners at press conferences refuse to build on previous questions by other questioners, and refuse to pressure the respondent at the podium to answer the question the responder is insistent on avoiding when the question happens to have come from someone else, the virtues of cross-examination, such as they are, are substantially diluted.

16. A classic work, still valuable, is Francis Lewis Welllman, The Art of Cross-Examination, 4th ed. (New York: Touchstone / Simon and Schuster, 1997) (orig. pub. 1903).

17. See Tom Lininger, “Bearing the Cross,” Fordham Law Review 74 (2005): 1353–1423.

18. See Barbara A. Spellman and Elizabeth R. Tenney, “Credible Testimony in and out of Court,” Psychonomic Bulletin and Review 17 (2010): 168–173.

19. It is often believed that British university hiring committees do this with letters of recommendation coming from Americans.

20. On calibration in law generally, see Frederick Schauer and Barbara A. Spellman, “Calibrating Legal Judgments,” Journal of Legal Analysis 9 (2017): 125–151. Because courts reviewing the decisions of administrative agencies, and appellate courts reviewing the decisions of lower courts, might benefit from calibrating the decisions of the decision makers below in just this way, it would seem useful for the practice of calibration to have more of a presence in law than it actually does. Officially, such calibration is rarely available, and we do not see appellate judges explicitly scrutinizing more carefully the decisions of trial judges they know to make mistakes or who are known to have biases of one sort or another. But unofficially, such calibration is more common than most reviewing courts or other institutions are willing to admit.

21. See Alfred Avins, “The Right to Be a Witness and the Fourteenth Amendment,” Missouri Law Review 31 (1966): 471–504; Bennett Capers, “Evidence without Rules,” Notre Dame Law Review 94 (2019): 867–908; Gabriel J. Chin, “ ‘Chinaman’s Chance’ in Court: Asian Pacific Americans and Racial Rules of Evidence,” U.C. Irvine Law Review 3 (2013): 965–990; Sheri Lynn Johnson, “The Color of Truth: Race and the Assessment of Credibility,” Michigan Journal of Race and Law 1 (1996): 261–346.

22. Jasmine B. Gonzalez Rose, “Toward a Critical Race Theory of Evidence,” Minnesota Law Review 101 (2017): 2243–2308.

23. Miranda Fricker, Epistemic Injustice: Power and the Ethics of Knowledge (Oxford: Oxford University Press, 2007). For elaboration of Fricker’s ideas, see Rae Langton, Review [of Fricker], Hypatia 25 (2010): 459–464; Federico Luzzi, “Testimonial Injustice without Credibility Deficit (or Excess),” Thought 5 (2016): 203–211; Wade Munroe, “Testimonial Injustice and Prescriptive Credibility Deficits,” Canadian Journal of Philosophy 46 (2016): 924–947.

24. So too for credibility enhancements, or boosts. In the United States, it rarely hurts one’s credibility to have a British accent, and in the United Kingdom, as Professor Henry Higgins reminded us in My Fair Lady, regional or class accents can greatly influence a person’s credibility and much else.

25. See E. Paige Lloyd, Kurt Hugenberg, Allen R. McConnell, Jonathan W. Kuntsman, and Jason C. Deska, “Black and White Lies: Race-Based Biases in Deception Judgments,” Psychological Science 28 (2017): 1125–1136.

26. See the reports described in Catharine A. MacKinnon, “Reflections on Sex Equality under Law,” Yale Law Journal 100 (1991): 1281–1328, at 1306n115.

27. John Henry Wigmore, The Principles of Judicial Proof, 2nd ed. (Boston: Little, Brown,1931), chap. 19, at 334, “updating” Wigmore’s The Principles of Judicial Proof, as Given by Logic, Psychology and General Experience and Illustrated in Judicial Trials (Boston: Little, Brown, 1913), and as quoted and discussed in William Twining, Theories of Evidence: Bentham and Wigmore (London: Weidenfeld and Nicolson, 1985), at 138–139.

28. California Jury Instructions, Crim. No. 10.22 (3rd ed., 1970), as quoted and finally rejected in People v. Rincon-Pineda, 538 P.2d 247, 252 (Ca. 1975).

29. Casenote, Fordham Urban Law Journal 4 (1976): 419–430. Wigmore had views about the credibility of rape victims as well, favorably quoting Dr. Karl Menninger to the effect that the “psychic complexes” and “probably universal” “fantasies of being raped” of women and young girls led them to “contrive[e] false charges of sexual offenses by men.” Wigmore, Principles of Judicial Proof, as quoted in Richard O. Lempert et al., A Modern Approach to Evidence: Text, Problems, Transcripts and Cases, 5th ed. (St. Paul, MN: West Academic, 2014), at 480.

30. Fricker, Epistemic Injustice; Ian James Kidd, José Medina, and Gaile Pohlhaus Jr., eds., Routledge Handbook of Epistemic Injustice (London: Routledge, 2017); Morten Fibieger Byskov, “What Makes Epistemic Injustice an ‘Injustice’?,” Journal of Social Philosophy 50 (2020): 1–18; David Coady, “Two Concept of Epistemic Injustice,” Episteme 7 (2020): 101–113.

31. And thus some statistically nonspurious generalizations may still properly be rejected for use on moral or political grounds. Frederick Schauer, Profiles, Probabilities, and Stereotypes (Cambridge, MA: Harvard University Press, 2003).

32. See Selene Cansino et al., “The Rate of Source Memory Decline across the Adult Life Span,” Developmental Psychology 49 (2013): 973–985; Trey Hedden and John D. E. Gabrieli, “Insights into the Aging Mind: A View from Cognitive Neuroscience,” Nature Reviews Neuroscience 5 (2004): 87–96; Scott A. Small, “Age-Related Memory Decline: Current Concepts and Future Directions,” Archives of Neurology 58 (2001): 360–364.

33. Federal Rule of Evidence 608 allows witnesses to testify as to their opinion of another witness’s honesty, and as to another witness’s reputation for honesty. And Federal Rule of Evidence 609 allows evidence of the prior criminal convictions of a witness to be used to attack the credibility—to “impeach”—that witness. The empirical basis for this form of calibration is, to put it mildly, not well established, but it is nevertheless a long-standing feature of most common-law legal systems. See Andrea R. Spirn, “The Place for Prior Conviction Evidence in Civil Actions,” Columbia Law Review 86 (1986): 1267–1282; Roger C. Park, “Impeachment with Evidence of Prior Convictions,” Southwestern University Law Review 36 (2008): 793–817.

Chapter 7. Of Lies and Liars

1. A current version of the rule is Federal Rule of Evidence 803(2). For history and critique of the rule (as well as the related exception for “present sense impressions”) in both its current and traditional versions, see Steven Baicker-McKee, “The Excited Utterance Paradox,” Seattle University Law Review 41 (2017): 111–178; Aviva Orenstein, “ ‘My God!’: A Feminist Critique of the Excited Utterance Exception to the Hearsay Rule,” California Law Review 85 (1997): 159–223; Jon Waltz, “The Present Sense Impression Exception to the Rule against Hearsay: Origins and Attributes,” Iowa Law Review 66 (1981): 869–898.

2. See Michael J. Saks and Barbara A. Spellman, The Psychological Foundations of Evidence Law (New York: NYU Press, 2016), 192–196.

3. A now-canonical work is Sissela Bok, Lying: Moral Choice in Public and Private Life (New York: Vintage Books, 1978). More recently, see Brooke Harrington, ed., Deception: From Ancient Empires to Internet Dating (Stanford, CA: Stanford University Press, 2009); Seana Valentine Shiffrin, Speech Matters: On Lying, Morality, and the Law (Princeton, NJ: Princeton University Press, 2014); Cass R. Sunstein, Liars: Falsehoods and Free Speech in an Age of Deception (New York: Oxford University Press, 2021); Christine Korsgaard, “The Right to Lie: Kant on Dealing with Evil,” Philosophy and Public Affairs 15 (1986): 325–349.

4. George Fisher, “The Jury’s Rise as a Lie Detector,” Yale Law Journal 107 (1997): 575–711, at 624–625.

5. Kenneth S. Abraham, “The Common Law Prohibition on Party Testimony and the Development of Tort Liability,” Virginia Law Review 95 (2009): 489–515; John H. Langbein, “Historical Foundations of the Law of Evidence: A View from the Ryder Sources,” Columbia Law Review 96 (1996): 1168–1202, at 1184–1186.

6. For Shiffrin (Speech Matters, 12), a lie is “an intentional assertion of A to B of a proposition P such that

A does not believe P, and

A is aware that A does not believe P, and

A intentionally presents P in a manner or context that objectively manifests an intention that B is to take and treat P as an accurate representation of A’s belief.”

7. Donald Trump has taken, and even out of office is still taking, presidential lying to new heights, but it is worth remembering that it was Democratic senator Bob Kerrey who observed that then-president Bill Clinton was an “unusually good liar.” And that was even before most of the details about Jennifer Flowers, Paula Jones, Monica Lewinsky, and others became public. See Stephen Chapman, “Bill Clinton and the Boy Who Cried Wolf,” Chicago Tribune, Jan. 25, 1998.

8. See Caroline Kitchener and Matt Thompson, “A Journalist’s Guide to Presidential Lying,” Columbia Journalism Review, June 6, 2018; James Fallows, “How to Deal with the Lies of Donald Trump: Guidelines for the Media,” The Atlantic, Nov. 28, 2016; Matthew Ingram, “When Should Journalists Use the ‘L’ Word?,” Fortune, Jan. 26, 2017; Pete Vernon, “Lie? Falsehood? What to Call the President’s Words,” Columbia Journalism Review, May 29, 2018; Gary Weiss, “Calling Trump a Liar Sets a Thorny Precedent,” Columbia Journalism Review, Nov. 1, 2016.

9. Regarding the false claim to have been awarded the Medal of Honor: United States v. Alvarez, 567 U.S. 709 (2012), holds exactly such a statement to be protected by the First Amendment.

10. Yes, there is such a product. See Terry Dunkle, “Slim Slippers: A Precautionary Tale,” at www.quackwatch.com (2002). And no, they do not work. Nor do diet earrings and diet soap, the latter allegedly washing off the fat. See Federal Trade Commission, Deception in Weight Loss Advertising Workshop, at www.ftc.gov/bcp/workshops/weightloss/transcripts/transscript-full.pdf (2002).

11. See Cameron McWhirter and Lindsay Wise, “Trump Pressured Georgia Secretary of State to ‘Find’ Votes,” Wall Street Journal, Jan. 4, 2021.

12. 52 U.S.C. § 20511 (2018). See Eric Lipton, “Trump Call to Georgia Official Might Violate State and Federal Law,” New York Times, Jan. 3, 2021.

13. See Frederick Schauer and Richard Zeckhauser, “Paltering,” in Harrington, Deception, 38–54.

14. For the full and fascinating history, see Jill Lepore, The Secret History of Wonder Woman (New York: Knopf, 2014).

15. 293 F. 1013 (D.C. Cir. 1923).

16. See David Faigman et al., Modern Scientific Evidence: The Law and Science of Expert Testimony (St. Paul, MN: Thomson West, 2020), §§ 1:2–1:6.

17. See Ken Alder, The Lie Detectors: The History of an American Obsession (New York: Free Press, 2007); Kerry Segrave, Lie Detectors: A Social History (Jefferson, NC: McFarland, 2003); Allison Marsh, “A Brief History of the Lie Detector,” IEEE Spectrum, August 2019; Philip Reichert and Louis F. Bishop Jr., “Sir James MacKenzie and His Polygraph: The Contributions of Louis Faugeres Bishop, Sr.,” American Journal of Cardiology 24 (1969): 401–403; Paul V. Trovillo, “A History of Lie Detection,” Journal of Criminal Law and Criminology 29 (1939): 848–881, and 30 (1939): 104–119.

18. See Don Grubin and Lars Mardin, “Lie Detection and the Polygraph: A Historical Review,” Journal of Forensic Psychiatry and Psychology 16 (2005): 357–369; Martina Vicianova, “Historical Techniques of Lie-Detection,” European Journal of Psychology 11 (2015): 522–534.

19. See Samatha Mann, Aldert Vrij, and Ray Bull, “Detecting True Lies: Police Officers’ Ability to Detect Suspects’ Lies,” Journal of Applied Psychology 89 (2004): 137–149; Miron Zuckerman, Richard Koestner, and Robert Drover, “Beliefs about Cues Associated with Deception,” Journal of Nonverbal Behavior 6 (1981): 105–114.

20. See Bella M. DePaulo et al., “Cues to Deception,” Psychological Bulletin 129 (2003): 74–118. See also DePaulo, The Psychology of Lying and Detecting Lies (Summerland, CA: Bella DePaulo, 2018); DePaulo, The How and Whys of Lies (Summerland, CA: Bella DePaulo, 2010); Charles F. Bond Jr. and Bella DePaulo, Is Anyone Really Good at Detecting Lies? Professional Papers (Summerland, CA: Bella DePaulo, 2011).

21. See Gary D. Bond, “Deception Detection Expertise,” Law and Human Behavior 32 (2008): 339–351; Peter J. DePaulo and Bella M. DePaulo, “Can Deception by Salespersons and Customers Be Detected through Nonverbal Behavioral Cues?,” Journal of Applied Social Psychology 19 (1989): 1552–1577. The use of multiple cues appears to help slightly, but even then the accuracy of lie detection by human observers is far less than commonly believed. Maria Hartwig and Charles F. Bond Jr., “Lie Detection from Multiple Cues: A Meta-Analysis,” Applied Cognitive Psychology 28 (2014): 661–676.

22. See Karl Ask, Sofia Calderon, and Erik Mac Giolla, “Human Lie-Detection Performance: Does Random Assignment versus Self-Selection of Liars and Truth-Tellers Matter?,” Journal of Applied Research in Memory and Cognition 9 (2020): 128–136; Tim Brennen and Svein Magnusssen, “Research on Non-Verbal Signs of Lies and Deceit: A Blind Alley,” Frontiers in Psychology 11 (2020); Maria Hartwig and Charles F. Bond Jr., “Why Do Lie-Catchers Fail? A Lens Model Meta-Analysis of Human Lie Judgments,” Psychological Bulletin 137 (2011): 643–659.

23. For examples, see United States v. Sanchez, 118 F.3d 192, 197 (4th Cir. 1997); United States v. Nelson, 217 Fed. App’x 291 (4th Cir. 2006); State v. Jones, 753 N.W.2d 677, 690 (Minn. 2008); State v. Domicz, 907 A.2d 395 (N.J. 2006). Things are changing, albeit slowly. New Mexico now generally permits the use of lie-detection results in court (New Mexico Rule of Evidence 11-707; State v. Dorsey, 539 P.2d 204 (N.M. 1975), and some state and federal courts have eliminated the traditional blanket exclusion of polygraph evidence in favor of a more flexible approach that takes into account, among other factors, the proficiency of the examiner, the reliability of the particular device or method used, and the use to which the evidence is being put. See United States v. Benavidez-Benavidez, 217 F.3d 720 (9th Cir. 2000); State v. Porter, 698 A.2d 739, 769 (Conn. 1997); Commonwealth v. Duguay, 720 N.E.2d 458 (Mass. 1999). And see also United States v. Semrau, 693 F.3d 510 (6th Cir. 2012), rejecting defendant’s attempt to use fMRI (brain scan) evidence to establish lack of criminal intent.

24. United States v. Scheffer, 523 U.S. 303, 313 (1998), quoting United States v. Barnard, 490 F.2d 907, 912 (9th Cir. 1974); Fisher, “The Jury’s Rise as a Lie Detector.” The Scheffer decision held that a defendant had no constitutional right to present polygraph evidence in his own defense but said nothing about whether using such evidence was permissible as a matter of nonconstitutional evidence law.

25. See National Research Council, The Polygraph and Lie-Detection (Washington, DC: US Government Printing Office, 2003); Madeleine Baran and Jennifer Vogel, “Inconclusive: The Truth about Lie Detector Tests,” APM Reports, Sep. 20, 2016, at www.apmreports.org; M. Raheel Bhutta, Melissa J. Hong, Yun-Hee Kim, and Keum-Shik Hing, “Single-Trial Lie Detection Using a Combined fNIRS-Polygraph System,” Frontiers in Psychology (2015); John C. Kircher, Steven W. Horowitz, and David C. Raskin, “Meta-Analysis of Mock Crime Studies of the Control Question Polygraph Technique,” Law and Human Behavior 12 (1988): 79–90; Ionnis Pavlidis, Norman Eberhardt, and James A. Levine, “Seeing Through the Face of Deception,” Nature 415 (2002): 35; Runxin Yu et al., “Using Polygraph to Detect Passengers Carrying Illegal Items,” Frontiers in Psychology (2019).

26. For descriptions of these various techniques, with references, see Frederick Schauer, “Lie-Detection, Neuroscience, and the Law of Evidence,” in Philosophical Foundations of Law and Neuroscience, ed. Dennis Patterson and Michael S. Pardo (Oxford: Oxford University Press, 2016), 85–104.

27. Sebastian Bedoya-Echeverry et al., “Detection of Lies by Facial Thermal Imagery Analysis,” Revista Facultad de Inginiera 26 (2017), http:/dx.doi.org/10.19053/01211129.v.26.v44.2017.5771; Maria Serena Panasiti et al., “Thermal Signatures of Voluntary Deception in Ecological Conditions,” Nature Science Reports 6 (2016), art. 35174 (Oct. 13, 2016).

28. Xiao Pan Ding, Xiaoqing Gao, Genyue Fu, and Kang Lee, “Neural Correlates of Spontaneous Deception: A Functional Near Infrared Spectroscopy (fNIRD) Study,” Neuropsychologia 51 (2013): 704–712; Febghua Tian, Vikrant Sharma, Frank Andrew Kozel, and Hanli Liu, “Functional Near Infrared Spectroscopy to Investigate Hemodynamic Responses in the Prefrontal Cortex,” Brain Research 1303 (2009): 120–130.

29. Marzieh Daneshi Kohan, Ali Motie Nasrabadi, and Mohammad Bagher Shamsollahi, “Interview Based Connectivity Analysis of EEG in Order to Detect Deception,” Medical Hypotheses 136 (2020), art. 109517; J. Peter Rosenfeld, “P300 in Detecting Concealed Information and Deception: A Review,” Psychophysiology 57 (March 11, 2019), art. E13362.

30. See, among many sources, Qian Cui et al., “Detection of Deception Based on fMRI Activation Patterns Underlying the Production of a Deceptive Response and Receiving Feedback about the Success of the Deception after a Mock Murder Crime,” Social Cognitive and Affective Neuroscience 9 (2014): 1472–1480; Daniel D. Langleben et al., “Polygraphy and Functional Magnetic Resonance Imaging in Lie Detection,” Journal of Clinical Psychiatry 77 (2016): 1372–1380. A balanced critique and overview is Martha J. Farah,. J. Benjamin Hutchinson, Elizabeth A. Phelps, and Anthony D. Wagner, “Function MRI-Based Lie Detection: Scientific and Social Challenges,” Nature Reviews Neuroscience 15 (2014): 312–319.

31. For an overview of the skepticism, and for my own skepticism about the skepticism, see Schauer, “Lie-Detection, Neuroscience”; Frederick Schauer, “Neuroscience, Lie-Detection, and the Law: A Contrarian View,” Trends in Cognitive Science 14 (2010): 101–103; Schauer, “Can Bad Science Be Good Evidence? Neuroscience, Lie-Detection, and Beyond,” Cornell Law Review 95 (2010): 1191–1220.

32. Indeed, use by a defendant in an attempt to raise a reasonable doubt was the original proposed (and rejected) use in the Frye case, and was the proposed (and rejected) use in the Supreme Court’s Scheffer decision.

33. Deena Skolnick Weisberg et al., “The Seductive Allure of Neuroscience Explanations,” Journal of Cognitive Neuroscience 20 (2008): 470–477; Martha J. Farah and Cayce J. Hook, “The Seductive Allure of ‘Seductive Allure,’ ” Perspectives on Psychological Science 8 (2013): 88–90; Cayce Hook and Martha J. Farah, “Look Again: Effects of Brain Images and Mind-Brain Dualism on Lay Evaluations of Research,” Journal of Cognitive Neuroscience 25 (2013): 1397–1405. Subsequent research has clarified that neuroscience information may be irrationally influential, but that nothing about the visual and seemingly photographic presentation of that information contributes to the effect. Diego Fernandez-Duque, Jessica Evans, Colton Christian, and Sara D. Hughes, “Superfluous Neuroscience Information Makes Explanations of Psychological Phenomena More Appealing,” Journal of Cognitive Neuroscience 27 (2015): 926–944.

34. 29 U.S.C. §§ 2001 et seq. (2018).

35. Katie Galioto, “Justin Fairfax Releases Polygraph Test as He Fights Back against Accusers,” Politico, Apr. 3, 2019.

36. John Walsh, “Here Is the Polygraph Test Christine Blasey Ford Took Following Her Sexual Assault Accusation against Brett Kavanaugh,” Business Insider, Sept. 26, 2018.

37. www.converus.com.

38. www.discernscience.com. For description and critique of these and other recent developments, see Jake Bittle, “Lie Detectors Have Always Been Suspect. AI Has Made the Problem Worse,” APM Reports, Sept. 20, 2016.

Chapter 8. Can We Believe Our Eyes and Ears?

1. The issue here is not the reliability, for any of the senses, of the sensation itself. It is of the reliability of sensation as evidence for what is sensed. If someone reports that they smelled rotten eggs, the question is not whether their smelling sensation is what they have reported, but whether that sensation was caused by actual rotten eggs, or, as is likely, by leaking natural gas.

2. See Thomas D. Albright, “Why Eyewitnesses Fail,” Proceedings of the National Academy of Sciences 114 (2017): 7758–7764; Neil Brewer and Gary L. Wells, “Eyewitness Identification,” Current Directions in Psychological Science 20 (2011): 24–27; Steven E. Clark, “Costs and Benefits of Eyewitness Identification Reform,” Perspectives on Psychological Science 7 (2012): 238–259; Steven E. Clark and Ryan D. Godfrey, “Eyewitness Identification and Innocence Risk,” Psychonomic Bulletin and Review 16 (2009): 22–42; Gary L. Wells, Amina Memon, and Steven D. Penrod, “Eyewitness Evidence: Improving Its Probative Value,” Psychological Science in the Public Interest 7 (2006): 45–75; Gary L. Wells et al., “Policy and Procedure Recommendations for the Collection and Preservation of Eyewitness Identification Evidence,” Law and Human Behavior 44 (2020): 3–36; John Wixted and Gary L. Wells, “The Relationship between Eyewitness Confidence and Eyewitness Accuracy: A New Synthesis,” Psychological Science in the Public Interest 18 (2017): 10–65.

3. The law often calls this “factual innocence.” Many wrongful convictions are wrongful because of the denial of a defendant’s procedural rights—for example, the right to have an effective lawyer, to have illegally obtained evidence suppressed, to confront opposing witnesses, and to have a properly instructed jury. Convictions obtained when such rights have been violated are legally wrong and properly overturned, too often too late, but overturning convictions on those grounds says little about whether the defendant actually committed the deed that produced the conviction. By contrast, other overturnings—exonerations in a narrower sense—are based on witness recantation, subsequently uncovered evidence, someone else confessing to the crime that the defendant is alleged to have committed, and various other grounds for a post-conviction belief that the defendant simply did not do it. This latter basis for exoneration—factual innocence—is the principal concern here.

4. See especially C. J. Brainerd and V. F. Reyna, The Science of False Memory (New York: Oxford University Press, 2005); Elizabeth F. Loftus, Eyewitness Testimony (Cambridge, MA: Harvard University Press, 1979); Loftus, “Planting Misinformation in the Human Mind: A 30-Year Investigation of the Malleability of Memory,” Learning and Memory 12 (2005): 361–366; Daniel L. Schachter, The Seven Sins of Memory: How the Mind Forgets and Remembers (Boston: Houghton Mifflin, 2001).

5. For a valuable philosophical analysis of such manipulation, see Jennifer Lackey, “Eyewitness Testimony and Epistemic Agency,” Noûs (June 11, 2021).

6. State v. Henderson, 27 A.3d 872 (N.J. 2011); Elizabeth F. Loftus, “Eyewitness Science and the Legal System,” Annual Review of Law and Social Science 14 (2018): 1–10.

7. The expression “with their own eyes” is common, but how someone could actually see something with someone else’s eyes is not entirely clear. Presumably the expression is designed to distinguish the first-person observation from hearsay—what someone has heard someone else report.

8. A sampling of the extensive research includes Brainerd and Reyna, Science of False Memory; Daniel L. Schachter, Seven Sins of Memory; Michael C. Anderson and Barbara A. Spellman, “On the Status of Inhibitory Mechanisms in Cognition: Memory Retrieval as a Model Case,” Psychological Review 102 (1995): 68–100; David A. Gallo, “False Memories and Fantastic Beliefs: 15 Years of the DRM Illusion,” Memory and Cognition 38 (2010): 833–848; Simona Ghetti, “Memory for Nonoccurrences: The Role of Metacognition,” Journal of Memory and Language 48 (2003): 722–739; Henry L. Roediger III, “Memory Illusions,” Memory and Language 35 (1996): 76–100; Daniel L. Schachter and Scott D. Slotnick, “The Cognitive Neuroscience of Memory Distortion,” Neuron 44 (2004): 149–160.

9. Clark, “Costs and Benefits,” unconventionally, controversially, but wisely emphasizes the costs (in terms of an increase in the number of mistaken acquittals) as well as benefits of eyewitness identification reform. And see also the work of Larry Laudan, who has repeatedly focused attention on how the devices of the criminal law—especially, in his view, the requirement of proof beyond a reasonable doubt—produce errors of false acquittal, possibly in greater numbers than is generally assumed. Larry Laudan, Truth, Error, and the Criminal Law: An Essay in Legal Epistemology (Cambridge: Cambridge University Press, 2006); Laudan, “The Rules of Trial, Political Morality, and the Costs of Error,” Oxford Studies in the Philosophy of Law 1 (2011): 195–227; Laudan, “Is Reasonable Doubt Reasonable?,” Legal Theory 9 (2003): 295–331. For a challenge to Laudan’s empirical claims, see Diego del Vecchi, “Laudan’s Error: Reasonable Doubt and Acquittals of Guilty People,” International Journal of Evidence and Proof 24 (2020): 211–232.

10. A rich philosophical literature, largely from a feminist perspective, goes by the name of “standpoint epistemology.” A valuable overview is Elizabeth Anderson, “Feminist Epistemology and the Philosophy of Science,” Stanford Encyclopedia of Philosophy, https://plato.stanford.edu/archives/spr2020/entries/feminist-epistemology/ (2020). But we must distinguish three claims, one I embrace and two I resist. The claim I embrace, one falling under the heading of epistemic injustice, is that we have long ignored or discounted the accounts of marginalized groups, including women, racial and ethnic and religious minorities, and those with less traditional sexual orientations and identities. By ignoring those accounts, we make epistemic and evidentiary mistakes as well as perpetuate injustice generally. But two other claims are more controversial. One is that the perspective of the victim is better evidence because of that perspective. But here it depends on the question. Obviously a person denied a job because of her race or gender knows what that feels like in ways that no one else can. But if the question is about what happened, and not how it felt, what the victim knows is not necessarily better evidence than what the perpetrator knows. And most controversial of all is the claim that it is perspective all the way down, with the existence of multiple perspectives proving that there is no fact of the matter. Throughout this book I obviously resist this claim, but explaining why would take a book in itself.

11. Susan Sontag, On Photography (New York: Dell, 1973).

12. I am grateful to Bob Allison, Kevin Cullen, Larry DiCara, and Shaun O’Connell for sharing their recollections with me and correcting mine. The election itself is the subject of a still-important book by Murray B Levin, The Alienated Voter: Politics in Boston (New York: Holt, Rinehart and Winston, 1960).

13. Kendall Walton, “Transparent Pictures: On the Nature of Photographic Realism,” Critical Inquiry 11 (1984): 246–277. And see also the discussion in Michael Morris, Real Likenesses: Representation in Paintings, Photographs, and Novels (Oxford: Oxford University Press, 2020).

14. Bhadra Sharma and Sameer Yasir, “Nepal Seeks to Ban 2 Climbers It Says Faked Everest Summit,” New York Times, Feb. 1, 2021, A8.

15. The examples in the text are selected because Goya’s Disasters of War and Manet’s Execution of Emperor Maximilian are two of the most revered nonphotographic visual works about war of the nineteenth century, and of all time.

16. Simon Brann Thorpe, Toy Soldiers (London: Dewi Lewis, 2015). To be sure, Thorpe was making a point about war as well as about photography.

17. Among thousands of examples, see Kleveland v. United States, 345 F.2d 134 (2d. Cir. 1965); Moore v. Leaseway Transport Corp., 402 N.E.2d 1160 (N.Y. 1980).

18. United States v. Lubich, 2013 WL186675 (Ct. Mil. App. Armed Forces, May 3, 2013); Fisher v. State, 643 S.W.2d 571 (Ark. 1982).

19. On so-called Day in the Life videos, typically used to depict an injured person’s life before the litigation-prompting accident, see Dempster v. Lamorak Insurance Co., 2020 WL5665172 (E.D. La., Sep. 23, 2020); Families Advocate. LLC v. Sanford Clinic North, 2019 WL10943310 (D.N. Dak., June 12, 2019).

20. In reality, Daguerre was preceded by the his countryman Joseph Nicéphore Niépce and the Englishman Thomas Wedgwood, but Daguerre’s is the name that most endures. See Beaumont Newhall, The History of Photography (New York: Museum of Modern Art; Boston: Little, Brown, 1982), 13–26.

Chapter 9. Of Experts and Expertise

1. Conventionally, I date the transformation to all of the nominations subsequent to those of Judge Robert Bork in 1987.

2. See John Schwartz and Hiroko Tabuchi, “By Calling Climate Change ‘Controversial,’ Barrett Created Controversy,” New York Times, Oct. 15, 2020 (updated Oct. 22, 2020), at https://www.nytimes.com/2020/10/15/climate/amy-coney-barrett-climate-change.html.

3. See Susan Haack, “Federal Philosophy of Science: A Deconstruction—and a Reconstruction,” NYU Journal of Law and Liberty 5 (2010): 394–435; Haack, Evidence Matters: Science, Proof, and Truth in the Law (Cambridge: Cambridge University Press, 2014), 122–155.

4. A thorough examination of the problem of nonexpert evaluation of expertise is Scott Brewer, “Expert Testimony and Intellectual Due Process,” Yale Law Journal 107 (1998): 1535–1681, esp. 1616–1647. In the philosophical literature, see Alvin I. Goldman, “Experts: Which Ones Should You Trust?,” Philosophy and Phenomenological Research 63 (2001): 85–110; John Hardwig, “Epistemic Dependence,” Journal of Philosophy 82 (1985): 335–349. Hardwig’s influential article argues that to make a decision based on the knowledge or expertise of others is to make a decision without evidence, a conclusion with which I (and Goldman) disagree. Only by starting with the assumption that little other than first-person perception can qualify as evidence can we reject the testimony of others (including the testimony of experts) as evidence, an assumption that much of this book is committed to resisting.

5. On this problem in the context of the use or non-use of expertise by appellate judges, see Frederick Schauer, “The Dilemma of Ignorance: PGA Tour, Inc. v. Casey Martin,” Supreme Court Review 2001 (2002): 267–297.

6. The example is real, being an abbreviated reference to the case of Daryl Bem, a well-credentialed Cornell University social psychologist whose research on extrasensory perception (ESP) is commonly thought to have triggered the methodological concerns and debates in the field that are often described as the “replication crisis.” For an overview, see Daniel Engber, “Daryl Bem Proves ESP Is Real,” Slate, June 7, 2017. For a more detailed account and explanation, see Barbara A. Spellman, “A Short (Personal) Future History of Revolution 2.0,” Perspectives on Psychological Science 10 (2015): 886–899.

7. See Roger Cooter, The Cultural Meaning of Popular Science: Phrenology and the Organization of Consent in Nineteenth-Century Britain (Cambridge: Cambridge University Press, 1984); John van Wyhe, Phrenology and the Origins of Victorian Scientific Naturalism (Burlington, VT: Ashgate, 2004); Minna Scherlinder Morse, “Facing a Bumpy History,” Smithsonian, Oct. 1997; Pierre Schlag, “Law and Phrenology,” Harvard Law Review 110 (1997): 877–921.

8. See the methodologically rigorous debunking of phrenology’s main claims in Parker Jones, F. Alfaro-Almagro, and S. Jbabdi, “An Empirical 21st Century Evaluation of Phrenology,” Cortex 106 (2018): 26–35.

9. Frighteningly, roughly 30 percent of the American people. See Sabrina Stierwalt, “Is Astrology Real? Here’s What Science Says,” Scientific American, June 25, 2020.

10. On the lack of scientific basis for astrology, see Alexander Boxer, A Scheme of Heaven: The History of Astrology and the Search for Our Destiny (New York: W. W. Norton, 2020); Richard A. Crowe, “Astrology and the Scientific Method,” Psychological Reports 67 (1990): 163–191; I. W. Kelly, “Astrology and Science: A Critical Examination,” Psychological Reports 44 (1979): 1231–1240; Jayant V. Narlikar, Sudhakar Kunte, Navendra Dabholkar, and Prakash Ghatpande, “A Statistical Test of Astrology,” Current Science 96 (2009): 641–643; Bernie I. Silverman, “Studies of Astrology,” Journal of Psychology 77 (1971): 141–149; G. A. Tyson, “An Empirical Test of the Astrological Theory of Personality,” Personality and Individual Differences 5 (1984): 247–250. A quirky, seeming counterexample is provided by the so-called relative age effect. In youth sports, the age at which you begin, or the league in which you are placed, varies with birth month. So children born in, say, December, are placed with those who are generally younger, and those born in January are placed with those generally older. And because it turns out that when you start, and the level at which you start, is predictive of future athletic success, birth month is predictive of athletic success. And, of course, astrological sign is, for most days of the month, coincident with birth month. The conclusion, therefore, is that astrological sign is predictive of athletic success. Vittoria Addona and Philip A. Yates, “A Closer Look at the Relative Age Effect in the National Hockey League,” Journal of Quantitative Analysis of Sports 6 (2010): 1017; William Hurley, Dan Lior, and Steven Tracze, “A Proposal to Reduce the Age Discrimination in Canadian Minor Hockey,” Canadian Public Policy 27 (2001): 65–75. And on the relative age effect in educational achievement, see Joshua Angrist and Alan Krueger, “Does Compulsory School Attendance Affect Schooling and Earning?,” Quarterly Journal of Economics 106 (1991): 979–1014.

11. See Federal Rules of Evidence 703, 704, 705; Christopher B. Mueller and Laird C. Kirkpatrick, Evidence, 4th ed. (New York: Aspen, 2009), §§ 7.8–7.13.

12. For a skeptical take on the view that expert evidence must meet a higher standard of reliability in order to be admitted as evidence in civil or criminal trials, see Frederick Schauer and Barbara A. Spellman, “Is Expert Evidence Really Different?,” Notre Dame Law Review 88 (2013): 1–26. The bases for the skepticism, as argued there, are two. One is that the traditional view is premised on the worry that jurors, and perhaps even judges, will be so impressed by experts that their testimony will be given more weight than it intrinsically deserves. But research on jury evaluation of expert evidence indicates that this long-held assumption might not be true. Second, a corollary of the first and consistent with a running theme of this book, is that relatively weak expert evidence may still have some evidentiary value. If the worry about expert evidence being given too much evidentiary value is not supported by the relevant research, then there seems little reason to discard expert evidence when its evidentiary—probative—value is no less than the evidentiary value of the various other forms of inconclusive evidence that are routinely admitted.

13. Frye v. United States, 293 F. 1013 (D.C. Cir. 1923).

14. See Editors of Lingua Franca, The Sokal Hoax: The Sham That Shook the Academy (Lincoln: University of Nebraska Press, 2000).

15. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). Daubert, along with Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999), and General Electric Co. v. Joiner, 522 U.S. 136 (1997), still represent the last word (from the Supreme Court) on the question.

16. See Shannon Hall, “Exxon Knew about Climate Change Almost 40 Years Ago,” Scientific American, Oct. 26, 2015; Maxine Joselow, “GM, Ford Knew about Climate Change 50 Years Ago,” E and E News, Oct. 26, 2020, at www.eenews.net. On the politics and incentives of climate change denial more generally, see, among hundreds of sources, Riley E. Dunlap, “Climate Change Skepticism and Denial: An Introduction,” American Behavioral Scientist 57 (2013): 691–698; Ferenc Jankó et al., “Sources of Doubt: Actors, Forums, and Language of Climate Change Skepticism,” Scientometrics 124 (2020): 2251–2277; Raul P. Lejano, “Ideology and the Narrative of Climate Skepticism,” Bulletin of the American Meteorological Society 100 (2019): ES415–ES421; Charles W. Schmidt, “A Closer Look at Climate Change Skepticism,” Environmental Health Perspectives 118 (2010): A536–A540.

17. Useful starting places include Hervé Le Treut et al., “Historical Origins of Climate Change Science,” in Climate Change 2007: The Physical Science Basis: Contribution of Working Group 1 to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, ed. S. Solomon et al. (Cambridge: Cambridge University Press, 2007); IPCC 2013, Climate Change 2013: The Physical Science Basis: Contribution of Working Group 1 to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (Cambridge: Cambridge University Press, 2013). The IPCC process and reports have hardly been without challenge and controversy. See Eli Kintisch, “IPCC / Climategate Criticism Roundup,” Science, Feb. 15, 2010.

18. On the nature of GMO opposition, see Edward B. Royzman, Corey Cusimano, Stephen Metas, and Robert F. Leeman, “Is Opposition to Genetically Modified Food ‘Morally Absolutist’? A Consequence-Based Perspective,” Perspectives on Psychological Science 15 (2020): 250–272.

19. See Nassim Nicholas Taleb, “The Precautionary Principle (with Application to the Genetic Modification of Organisms),” ArXive (2014).

20. In the United States, a 2016 report of the National Academies of Science is one of the principal sources of assurance on GMO safety. National Academies of Science, Genetically Engineered Crops: Experience and Prospects (Washington, DC: National Academies Press, 2016). To the same effect, and reporting on the similar conclusions of the American Academy for the Advancement of Science and other high-prestige scientific groups, see Jane E. Brody, “Are G.M.O. Foods Safe?,” New York Times, Apr. 2, 2018, D5; Niras Chokshi, “Stop Bashing G.M.O. Foods, More than 100 Nobel Laureates Say,” New York Times, June 30, 2016; Mark Lynas, “GMO Safety Debate Is Over,” May 23, 2016, at www.allianceforscience.cornell.edu; William Saletan, “Unhealthy Fixation,” Slate, July 15, 2015. But the “in the United States” modifier in the text is important, as the views outside the United States, and especially in western Europe, are very different and substantially more skeptical of GMO safety. See Angelika Hilbeck et al., “No Scientific Consensus on GMO Safety,” Environmental Sciences Europe 27 (2015), art. 4.

21. On the difficult issue of how nonexperts are to evaluate expert opinion as evidence when there is genuine expert disagreement, see Ben Almassi, “Conflicting Expert Testimony and the Search for Gravitational Waves,” Philosophy of Science 76 (2009): 570–584; David Christensen, “The Epistemology of Disagreement: The Good News,” Philosophical Review 116 (2007): 187–217; Gideon Rosen. “Nominalism, Naturalism, Epistemic Relativism,” Philosophical Perspectives 15 (2001): 69–91.

22. And see, with an admirable change of mind on this very issue, Gary Comstock, “Ethics and Genetically Modified Foods,” in The Philosophy of Food, ed. David M. Kaplan (Berkeley: University of California Press, 2012), 122–139.

23. See Jelani Cobb, “African-American Resistance to the Covid-19 Vaccine Reflects a Broader Problem,” New Yorker, Dec. 19, 2020; Amelia M. Jamison, Sandra Crouse Quinn, and Vicki S. Fellmuth, “ ‘You Don’t Trust a Government Vaccine’: Narratives of Institutional Trust and Influenza Vaccination among African-American and White Adults,” Social Science and Medicine 222 (2019): 87–94; Peng-jun Lu et al., “Racial and Ethnic Disparities in Vaccination Average among Adult Populations in the U.S.,” Vaccine 33 (supp.) (2015): D83–D91.

24. See Jonathan M. Berman, Anti-Vaxxers: How to Challenge a Misinformed Movement (Cambridge, MA: MIT Press, 2020); Anna Kirkland, Vaccine Court: The Law and Politics of Public Inquiry (New York: NYU Press, 2016); Gregory A. Poland and Robert M. Jacobson, “The Age-Old Struggle against the Antivaccinationists,” New England Journal of Medicine 364 (2011): 97–99; Robert M. Wolfe and Lisa K. Sharp, “Anti-Vaccinationists Past and Present,” British Medical Journal 325 (2002): 430–432.

25. In particular, see the federal Institute of Medicine’s extensive study of all of the then-existing studies. Institute of Medicine, Immunization Safety Review: Vaccines and Autism (Washington, DC: National Academies Press, 2004).

26. Nathan Ballantyne, “Epistemic Trespassing,” Mind 128 (2019): 367–395, revised as chapter 8 in Ballantyne, Knowing Our Limits (New York: Oxford University Press, 2019), 195–219.

27. See Atomic Heritage Foundation, “Leo Szilard’s Fight to Stop the Bomb,” at www.atomicheritage.org.

28. Wilson D. Miscamble, The Most Controversial Decision: Truman, the Atomic Bomb, and the Defeat of Japan (Cambridge: Cambridge University Press, 2005); Richard Rhodes, The Making of the Atomic Bomb (New York: Simon and Schuster, 1986).

Chapter 10. The Science of Crime

1. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), followed, importantly, by General Electric Co. v. Joiner, 522 U.S. 136 (1997), and Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999). The Supreme Court’s decisions on expert evidence are interpretations of the Federal Rules of Evidence and are not based on the US Constitution. Accordingly, they are not binding on state courts, although most (but not all) states have followed the Supreme Court’s lead. See Paul C. Giannelli, “Daubert in the States,” Criminal Law Bulletin 34 (1998): 154–166. Although the Supreme Court’s insistence that scientific and other expert evidence must be determined by the trial judge to be reliable before it can be admitted into evidence applies to all cases, the Court’s decisions arose in civil cases involving tort liability and not criminal cases involving forensic identification.

2. See Brandon L. Garrett and Peter J. Neufeld, “Invalid Forensic Science Testimony and Wrongful Convictions,” Virginia Law Review 95 (2009): 1–97; Jon B. Gould, Julia Carrano, Richard A. Leo, and Katie Hail-Jares, “Predicting Erroneous Convictions,” Iowa Law Review 99 (2014): 471–517.

3. An early skeptic was US District Judge Louis Pollak, although he was subsequently skeptical of his own skepticism. United States v. Llera Plaza, 179 F. Supp. 2d 1192 (E.D. Pa. 2001), vacated at 188 F. Supp. 2d 549 (2002). More comprehensively, see American Association for the Advancement of Science, Forensic Science Assessment: A Quality and Gap Analysis of Latent Fingerprint Examination (Washington, DC: AAAS, 2017); Erica Beecher-Monas, Evaluating Scientific Evidence: An Interdisciplinary Framework for Intellectual Due Process (New York: Cambridge University Press, 2007), 104–109; Brandon L. Garrett and Gregory Mitchell, “How Jurors Evaluate Fingerprint Evidence: The Relative Importance of Match Language, Method Information and Error Acknowledgment,” Journal of Empirical Legal Studies 10 (2013): 484–511; Garrett and Mitchell, “The Proficiency of Experts,” University of Pennsylvania Law Review 166 (2018): 901–960; Jonathan J. Koehler, “Fingerprint Error Rates and Proficiency Tests: What They Are and Why They Matter,” Hastings Law Journal 59 (2008): 1077–1100; Gregory Mitchell and Brandon L. Garrett, “The Impact of Proficiency Testing Information and Error Aversions on the Weight Given to Fingerprint Evidence,” Behavioral Science and Law 37 (2019):195–210; Jennifer L. Mnookin, “Fingerprints: Not a Gold Standard,” Issues in Science and Technology 20 (2003): 47–54; Jason M. Tangen, Matthew B. Thompson, and Duncan J. McCarthy, “Identifying Fingerprint Expertise,” Psychological Science 22 (2011): 995–997.

4. National Research Council, Strengthening Forensic Science in the United States: A Path Forward (Washington, DC: National Academies Press, 2009). A more recent report, somewhat narrower in focus, largely but not completely confirms much of the 2009 report’s skepticism. President’s Council of Advisors on Science and Technology (PCAST), Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, available at https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensic_science_report_final_pdf (2016). And for an extensive and revealing account of how the forensics community reacted and adapted to the 2009 National Research Council report, see Beth A. Bechky, Blood, Powder, and Residue: How Crime Labs Translate Evidence into Proof (Princeton, NJ: Princeton University Press, 2021).

5. Itiel E. Dror and Nicholas Scurich, “(Mis)use of Scientific Measurements in Forensic Science,” Forensic Science International: Synergy 2 (2020): 330–338; see, in the context of testimony by a ballistics expert, United States v. Glynn, 578 F. Supp. 2d 567 (S.D.N.Y. 2008); United States v. Montiero, 407 F. Supp. 2d 351 (D. Mass. 2006). For the views of a less skeptical judge, see United States v. Casey, 928 F. Supp. 2d 397 (D. Puerto Rico, 2013). And most recently, also less skeptical, see United States v. Johnson, 2019 WL1130258 (S.D.N.Y., Mar. 11, 2019).

6. United States v. Cantoni, 2019 WL1259630 (E.D.N.Y., Mar. 19, 2019).

7. See Nancy Ritter, “The Science behind Firearm and Tool Mark Examination,” National Institute of Justice Journal 274 (Dec. 2014); John Song et al., “Estimating Error Rates for Firearm Identification in Forensic Science,” Forensic Science International 284 (2018): 15–32.

8. United States v. Glynn. Other judges have been more willing to allow qualified experts to use less qualified language. United States v. Johnson.

9. See, famously, United States v. Spock, 416 F.2d 165 (1st Cir. 1969). Benjamin Spock, famous for his books on child rearing, was an active member of the Vietnam-era antiwar movement and had urged draft-eligible men to take various actions (including destroying their draft cards) in violation of Selective Service System regulations. Although Dr. Spock would have preferred to have had his conviction for counseling and abetting violation of the Selective Service Act reversed on the grounds of the unconstitutionality of the Act, the unconstitutionality of the Vietnam War, or his rights to freedom of speech under the First Amendment, in fact the appeals court overturned the jury verdict because the judge had required the jury to answer a series of questions—special questions, in the standard terminology—keyed to the particular elements of the offense, rather than permitting them simply to find the defendant guilty or not guilty.

10. See United States v. Harris, 2020 WL 6488714 (D.D.C., Nov. 4, 2020). In United States v. Tibbs, 2019 WL 4359486 (D.C. Super. Ct., Sept. 5, 2019), the judge also discussed the differences between genuine peer review and the kinds of review that preceded publication in a professional ballistics journal. For the latter, the reviewers knew who had written the article, and the authors knew who the reviewers were, both facts casting doubt on the soundness of peer review in legitimating a method by virtue of its publication. Moreover, the publication was accessible only to members of the typically prosecution-employed and prosecution-aligned community of professional ballistics examiners, thus depriving the results of scrutiny by those most likely to identify defects in the methods.

11. Although the distinction between admissibility and weight is a technical legal one, it is applicable to all uses of evidence in all contexts. Outside of the courtroom, the counterpart of admissibility is the decision to treat some fact as evidence in the first place. And this is a decision that, in theory, is independent of the question of how much weight that single piece of evidence will have when considered alongside all of the other evidence we have.

12. Roy N. King and Derek J. Koehler, “Illusory Correlations in Graphological Inference,” Journal of Experimental Psychology: Applied 6 (2000): 336–348; Carla Dazzi and Luigi Pedrabissi, “An Empirical Study of Handwriting Analysis,” Psychological Reports 105 (2009): 1255–1268; Anne Trubek, “Sorry, Graphology Isn’t Real Science,” JSTOR Daily, May 17, 2017, at www.daily.jstor.org. Note the confusing use of the term “graphology” to refer to the use of handwriting to describe personality or even predict the future, which has no empirical basis, but also to describe the use of handwriting to identify the writer, a method that has at least some empirical basis, even though the questions of how much and how we know how much are exactly the matters at issue.

13. For a Federal Bureau of Investigation description of the methods, see Diana Harrison, Ted M. Burkes, and Danielle Seiger, “Handwriting Examination: Meeting the Challenges of Science and the Law,” Forensic Science Communications 11, no. 4 (2009), available at www.archives.fbi.gov. And for a recent comprehensive description of the methods, but cited with the caution that the author makes his living by using those methods, see Ron Morris, Forensic Handwriting Identification, 2nd ed. (Cambridge, MA: Academic Press, 2021).

14. Almeciga v. Center for Investigative Reporting, Inc., 185 F. Supp. 3d 401 (S.D.N.Y. 2016) (Rakoff, J.); Samuel R. Gross, “Detection of Deception: The Case of Handwriting Expertise,” Virginia Law Review 87 (2001): 1847–1855; Michael Risinger, Mark P. Denbeaux, and Michael J. Saks, “Exorcism of Ignorance as a Proxy for Rational Knowledge: The Lessons of Handwriting Identification Expertise,” University of Pennsylvania Law Review 137 (1989): 731–792.

15. See Michael D. Risinger, “Cases Involving the Reliability of Handwriting Identification Evidence since the Decision in Daubert,” Tulsa Law Review 43 (2013): 477–596.

16. See Kristy Martire, Bethany Gowns, and Danielle Navarro, “What Do the Experts Know? Calibration, Precision, and the Wisdom of Crowds among Forensic Handwriting Experts,” Psychonomic Bulletin and Review 25 (2018): 2346–2355. See also National Institute of Standards and Technology, Working Group on Human Factors in Handwriting Examination, Forensic Handwriting Examination and Human Factors (Washington, DC: US Department of Commerce, 2020).

17. Compare Almeciga with United States v. Pitts, 2018 WL1116550 (E.D.N.Y.. Feb. 26, 2018).

18. 880 F. Supp. 1027 (S.D.N.Y. 1995).

19. For the claim, consistent with Starzecpyel but going further, that the question should not be science or not science, but, instead, whether some approach, science or not, was empirically sound, see Susan Haack, “An Epistemologist in the Bramble Bush: At the Supreme Court with Mr. Joiner,” Journal of Health Politics, Policy and Law 20 (2001): 217–248.

20. Kumho Tire Co. v. Carmichael. Kumho Tire involved a plaintiff’s witness in a products liability tort case, the witness purporting to be an expert in tire failure analysis. The witness testified that the blowout on an old and practically bald tire that had been consistently poorly maintained and incorrectly inflated was nevertheless caused by defective manufacture. And because the witness’s methods of visual and tactile examination of the tire had never been tested in any way, the Supreme Court appeared to treat this as just the kind of junk science or junk expertise that had inspired the Court’s entry into the area in the first place.

21. If the correct degree of acknowledgment of the importance of avoiding false convictions is already built into the “proof beyond a reasonable doubt” standard, then is heightening the requirements even further for prosecution use of otherwise relevant and admissible evidence a type of “double counting”? Yes, suggests Larry Laudan in Truth, Error and the Criminal Law: An Essay in Legal Epistemology (Cambridge: Cambridge University Press, 2006).

22. See John T. Rago, “A Fine Line between Chaos and Creation: Lessons on Innocence Reform from the Pennsylvania Eight,” Widener Law Review 12 (2006): 359–441.

23. A sensationalized and plainly ballistics-friendly account of the case is Colin Evans, Slaughter on a Snowy Morn: A Tale of Murder, Corruption and the Death Penalty Case that Revolutionised the American Courtroom (London: Icon Books, 2010).

24. For an extended and well-argued suggestion to just this effect, see Christopher Slobogin, Proving the Unprovable: The Role of Law, Science, and Speculation in Adjudicating Culpability and Dangerousness (New York: Oxford University Press, 2007), 131–144.

25. See, for example, United States v. Bell, 833 F.2d 272 (11th Cir. 1987); United States v. Lieberman, 637 F.2d 95 (2d Cir. 1980); United States v. Rich, 580 F.2d 929 (9th Cir. 1978).

26. See, for example, Brandon L. Garrett, Autopsy of a Crime Lab: Exposing the Flaws in Forensics (Berkeley: University of California Press, 2021).

27. Useful guides to DNA evidence include National Research Council, The Evaluation of Forensic DNA Evidence (Washington, DC: National Academies Press, 1996); David H. Kaye and George Sensabaugh, “Reference Guide on DNA Identification Evidence,” in National Research Council / Federal Judicial Center, Reference Manual of Scientific Evidence (Washington, DC: National Academies Press, 2011), 129–210; William C. Thompson, “Forensic DNA Evidence: The Myth of Infallibility,” in Genetic Explanations: Sense and Nonsense, ed. Sheldon Krimsky and Jeremy Gruber (Cambridge, MA: Harvard University Press, 2013), 227–255.

28. See David Faigman et al., Modern Scientific Evidence, 7th ed. (St. Paul, MN: Thomson / West, 2020), vol. 4, § 30:19. On the interrelationship between the ideal statistical probability and the statistical probability of laboratory error, compare Margaret A. Berger, “Laboratory Error Seen through the Lens of Science and Policy,” U.C. Davis Law Review 30 (1997): 1081–1111, with Jonathan J. Koehler, “Why DNA Likelihood Ratios Should Account for Error (Even When a National Research Council Report Says They Should Not),” Jurimetrics 37 (1997): 425–437.

29. Ex parte Perry v. State; 586 So.2d 242 (Ala. 1991); United States v. Beasley, 102 F.3d 1440 (8th Cir. 1996); People v. Venegas, 954 P.2d 525 (Cal. 1998).

30. Presenting DNA statistics in this way, and omitting the crucial “no other evidence” qualification, is an example of the “defense attorney’s fallacy,” no more and no less a fallacy than the prosecutor’s fallacy of arguing that the existence of a very small probability of a single random match coming from anyone other than the defendant proves by itself that the defendant is guilty beyond a reasonable doubt. William C. Thompson and Edward L. Schumann, “Interpretation of Statistical Evidence in Criminal Trials: The Prosecutor’s Fallacy and the Defense Attorney’s Fallacy,” Law and Hunan Behavior 11 (1987): 167–187. The DNA revolution has inspired a large literature on whether the statistics involved can be understood by jurors (and, for that matter, judges). See D. H. Kaye and Jonathan J. Koehler, “Can Jurors Understand Probabilistic Evidence?,” Journal of the Royal Statistical Society, Series A 154 (1991): 75–81; Jonathan J. Koehler, “The Psychology of Numbers in the Courtroom: How to Make DNA Statistics Seem Impressive or Insufficient,” Southern California Law Review 74 (2001): 1275–1306; Jason Schklar and Shari Seidman Diamond, “Juror Reactions to DNA Evidence: Errors and Expectancies,” Law and Human Behavior 23 (1999): 159–184.

31. For a much more detailed (and sophisticated) analysis of the evidentiary force of DNA tests from a Bayesian perspective, see Ian Ayres and Barry Nalebuff, “The Rule of Probabilities: A Practical Approach for Applying Bayes’ Rule to the Analysis of DNA Evidence,” Stanford Law Review 67 (2015): 1447–1503. See also David J. Balding, Weight-of-Evidence for Forensic DNA Profiles (Chichester, UK: John Wiley, 2005); Bess Stiffelman, “No Longer the Gold Standard: Probabilistic Genotyping Is Changing the Nature of DNA Evidence in Criminal Trials,” Berkeley Journal of Criminal Law 24 (2019): 110–146.

32. Greg Handikian, Emily West, and Olga Akselrod, “The Genetics of Innocence: Analysis of 194 U.S. DNA Exonerations,” Annual Review of Genomics and Human Genetics 12 (2011): 97–120; Gerald LaPorte, “Wrongful Convictions and DNA Exonerations: Understanding the Role of Forensic Science,” National Institute of Justice Journal 279 (Apr. 2018): 1–16.

Chapter 11. The Expanding Domain of Expertise

1. See https://www.iapcollege.com/program/lifestyle-expert-certificate-course-online/.

2. The philosopher Karl Popper famously refused to think of Freudian (or any form of) psychoanalysis as scientific. Karl R. Popper, Conjectures and Refutations: The Growth of Scientific Knowledge, 4th ed. (London: Routledge and Kegan Paul, 1972), 33–65; Popper, Realism and the Aim of Science, ed. W. W. Bartley III (Totowa, NJ: Rowman and Littlefield, 1983), 159–193. Even after Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), however, courts have recognized Freudian psychology as legitimate expertise. United States v. Rouse, 100 F.3d 560 (8th Cir. 1996); Clark v. Edison, 881 F. Supp. 2d 192 (D. Mass. 2012).

3. The sentence in the text was written prior to the appearance of the 2021 movie Minari, in which chicken sexing plays a surprisingly large role.

4. See R. D. Martin, The Specialist Chick Sexer (Box Hill, Victoria, Australia: Bernal, 1994); Irving Biederman and Margaret M. Shiffrar, “Sexing Day-Old Chicks: A Case Study and Expert Systems Analysis of a Difficult Perceptual-Learning Task,” Journal of Experimental Psychology: Learning, Memory and Cognition 13 (1987): 640–645; James McWilliams, “The Lucrative Art of Chicken Sexing,” Pacific Standard, Sept. 8, 2018, at https://psmag.com/magazine/the-lucrative-art-of-chicken-sexing.

5. Taylor Dafoe, “A Deep-Pocketed Art Collector Just Dropped More than $92 Million at Sotheby’s on This Pristine Botticelli Portrait,” Artnet, Jan. 28, 2021.

6. These, in descending order of authenticity, historical importance, and, commonly, value, are the descriptions typically given by the major auction houses to the works they are offering at auction. The “best” description would contain none of these qualifiers, and thus a description of a painting as simply “Botticelli” would indicate the auction house’s own opinion (presumably expert-certified) that Botticelli actually painted the work being offered.

7. For a good laugh, and references to other good laughs, see John Mariani, “The Uselessness of Winespeak,” Forbes, July 2, 2019.

8. Denis Dutton calls this “nominal” authenticity, as distinguished from the more ephemeral “expressive” authenticity, the latter being the judgment about whether the work genuinely expresses what is important about the artist or about the genre in which the work is created. Denis Dutton, “Authenticity in Art,” in The Oxford Handbook of Aesthetics, ed. Jerrold Levinson (New York: Oxford University Press, 2003), 258–274.

9. In addition to the references in Chapter 2, see John Godley, Van Meegeren: Master Forger (New York: Scribner’s, 1967); Frederik H. Kreuger, Han van Meegeren Revisited: His Art and the List of His Works (Delft: Quantes, 2013). On July 7, 2004, Sotheby’s sold Vermeer’s A Young Woman Seated at the Virginal, which had taken ten years to authenticate to Sotheby’s (and, presumably, the buyer’s) satisfaction, for slightly over $30 million dollars. On November 2, 2020, an oil painting by van Meegeren sold at auction in The Hague, Netherlands, for $1,046, and on December 9, 2020, a van Meegeren chalk drawing sold in the same location for $641. Q.E.D. (www.artprice.com).

10. See Ronald Spencer, ed., The Expert versus the Object: Judging Fakes and False Attributions in the Visual Arts (New York: Oxford University Press, 2004). Also useful is Michael Findlay, The Value of Art: Money, Power, Beauty (New York: Prestel, 2014), at 42–43.

11. Daubert.

12. “The art world places little faith in signatures. An expert on Jackson Pollock once explained why. ‘How long would it take you to learn to sign Pollock’s signature’ he asked, ‘and how long would it take you to learn to paint like Pollock?’ ” Edward Dolnick, The Forger’s Spell: A True Story of Vermeer, Nazis, and the Greatest Arty Hoax of the Twentieth Century (New York: HarperCollins, 2008), 151. Amusingly, one of the forgeries that brought about the demise of the art gallery Knoedler was an alleged Pollock with the signature misspelled as “Pollok.” Patricia Cohen, “Note to Forgers: Don’t Forget the Spell Check,” New York Times, July 11, 2014.

13. An accessible discussion of such methods is Jehane Ragai, The Scientist and the Forger: Insights into the Scientific Detection of Forgery in Paintings (London: Imperial College Press, 2015). And for a practitioner’s insight into how forgers might attempt to defeat those methods, see Eric Hebborn, The Art Forger’s Handbook (Woodstock, NY: Overlook Press, 2004).

14. See Lluís Peñuelas I Reixach, “The Authentication of Artworks,” in The Authorship, Authentication and Falsification of Artworks (Barcelona: Poligrafa, 2011), 124–153. This sense of subjective is to be distinguished from the sense in which taste is subjective. Monet’s being a better painter than Guillaumin is a largely subjective judgment because there is no actual fact of the matter, but this painting being by Monet or not may be subjective in the different sense because the evidence is unclear and interpretation and judgment become essential.

15. Indeed, a common characteristic—or pathology—of many art authentication experts is that they take the evidence produced by their own “eye” as being better than the scientific evidence, even when the two forms of evidence are directly in conflict. See Dolnick, The Forger’s Spell, and the various contributions in Spencer, The Expert versus the Object.

16. See Cedric Neumann and Hal Stern, “Forensic Examination of Fingerprints: Past, Present, and Future,” Chance 29 (2016): 9–16. Modern methods have moved beyond counting points of similarity.

17. Moreover, the personal incentives of authenticators might lead them to have higher or lower personal burdens of proof. A past failure might lead one authenticator to use an elevated burden of proof in order to avoid another mistake, but a past failure might lead a different authenticator to have a lower burden of proof in an attempt at redemption, as with Abraham Bredius, the mistaken authenticator of the most famous Vermeer forgery. Dolnick, The Forger’s Spell.

18. Sociological in the sense of the collective judgment of the community of experts themselves, and the collective judgment of the community of consumers of the expert opinions.

19. Susan Mulcahy, “Why Were So Many Stettheimer Art Works Up for Sale? Not All Were Real,” New York Times, Feb. 7, 2021, C1.

20. This assumes that, at least for museums, one or two cases of display of inauthentic artworks would be less fatal to its reputation than would be the same number or percentage of instances of inauthenticity for an auction house or dealer. No one will stop going to the Louvre if they take down two or three paintings revealed to be forgeries. At some point, of course, museums displaying a large number of works known to be inauthentic would lose their clientele as well.

21. Van Meegeren. Vermeer painted no such painting, although van Meegeren was able successfully to persuade many experts and museums otherwise. Dolnick, The Forger’s Spell.

22. No, said the 1964 Commission on the Assassination of President Kennedy (the Warren Commission). Yes, say more than a half century of conspiracy theories, some of which are described and debunked in Sean Munger, “Oswald Acted Alone: Faith vs. Fact in the Assassination of John F. Kennedy,” at www.SeanMunger.com (Nov. 22, 2020).

23. Yes, said a Norfolk County jury, which found them guilty, and yes, said Governor Alvan Fuller, who refused to commute their death sentences. No, said a large number of distinguished contemporary commentators, including then-professor and later-Supreme Court justice Felix Frankfurter. See Paul Avrich, Sacco and Vanzetti: The Anarchist Background (Princeton, NJ: Princeton University Press, 1980); Moshik Temkin, The Sacco-Vanzetti Affair: America on Trial (New Haven, CT: Yale University Press, 2009).

24. This question is the subject of active and occasionally heated debate. For an overview of the positions and of the evidence marshaled for various answers, see “When Was New Zealand First Settled?,” in Te Ara: The Encyclopedia of New Zealand, at www.teara.govt.nz.

25. An account of the dispute can be found in Adam Serwer, “The Fight over the 1619 Project Is Not about the Facts,” The Atlantic, Dec. 23, 2019. And although the article’s title says that the dispute is not about the facts, the critics, as the article makes clear, claim the dispute is about “matters of verifiable fact.” So there are two disputes. One dispute is about the facts. And the other dispute is about whether the dispute is about the facts.

26. Among literally scores—maybe hundreds—of books on the Titanic, the classic is Walter Lord, A Night to Remember (New York: Henry Holt, 2005) (orig. pub. 1955). Of the more recent comprehensive contributions, see Charles Pellegrino, Farewell, Titanic: Her Final Legacy (Hoboken, NJ: John Wiley, 2012). Regarding whether Garfield could have been saved if he had been treated by physicians who were not excessively self-confident, the answer is “probably yes.” See Candice Millard, Destiny of the Republic: A Tale of Madness, Medicine and the Murder of a President (New York: Anchor Books, 2012).

27. They almost certainly were not. See “The Trouble with False Teeth,” at www.mountvernon.org, providing strong support for the hippopotamus ivory hypothesis.

28. Plausibly yes. See G. W. Bernard, Anne Boleyn: Fatal Attractions (New Haven, CT: Yale University Press, 2010).

29. See Carolyn E. Holmes, The Black and White Rainbow: Reconciliation, Opposition, and Nation-Building in Democratic South Africa (Ann Arbor: University of Michigan Press, 2020).

30. See P. H. Nowell-Smith, What Actually Happened (Lawrence: University of Kansas Press, 1971); and, for example, R. F. Atkinson, Knowledge and Explanation in History: An Introduction to the Philosophy of History (Ithaca, NY: Cornell University Press, 1978), 39–68.

31. See, for overviews, Loren Haskins and Kirk Jeffrey, Understanding Quantitative History (Cambridge, MA: MIT Press, 1990); Konrad H. Jarausch and Kenneth A. Hardy, Quantitative Methods for Historians: A Guide to Research, Data, and Statistics (Chapel Hill: University of North Carolina Press, 1991). Several journals, especially Cliodynamics: The Journal of Quantitative History and Cultural Evolution and Historical Methods: A Journal of Quantitative and Interdisciplinary History, have the focus described in the text.

32. An interesting contrast is between the adjudication of guilt or liability in a legal proceeding and the adjudication of guilt by a referee in football, basketball, or hockey. In these and many other sports, and certainly before the confounding introduction of video replays, the referee would see something and immediately adjudicate it by calling a foul or some other sort of infraction. And although such determinations themselves involve evidence, typically they are the evidence of first-person and simultaneous observation, a form of evidence not available in legal proceedings and rarely available to the historian.

33. See Gilbert J. Garraghan, A Guide to Historical Method (New York: Fordham University Press, 1946); Martha Howell and Walter Prevenier, From Reliable Sources: An Introduction to Historical Methods (Ithaca, NY: Cornell University Press, 2001); R. J. Shafer, A Guide to Historical Method, 3rd ed. (Homewood, IL: Dorsey Press, 1977); Torstén Thuren, Källkritik (Stockholm: Almqvist & Wiksellm 1997).

34. See Margaret Sullivan, “How Do You Use an Anonymous Source? The Mysteries of Journalism Everyone Should Know,” Washington Post, Dec. 10, 2017. As the title of Sullivan’s article indicates, the use of anonymous sources as evidence has traditionally been discouraged, but the best description of that principle, even for the elite print press, is probably “soft and getting softer.”

35. See Charles L. Barzun, “Rules of Weight,” Notre Dame Law Review 83 (2008): 1957–2018; John H. Wigmore, “Required Numbers of Witnesses: A Brief History of the Numerical System in England,” Harvard Law Review 15 (1901): 83–108.

Chapter 12. The Relevance of the Past to the Present

1. Arlette Saenz, “Neera Tanden to Keep Reaching Out to Senator Next Week as Confirmation Is in Jeopardy,” www.cnn.com (Feb. 20, 2021). The nomination was subsequently withdrawn.

2. “White House Calls Cohen Liar Ahead of Testimony,” U.S. New and World Report, Feb. 26, 2019.

3. Alan Blinder, “Was That Ralph Northam in Blackface? An Inquiry Ends without Answers,” New York Times, May 22, 2019. Although there remains uncertainty about whether a picture showing one student in blackface and another in Ku Klux Klan garb included Northam, he did acknowledge that he had appeared in blackface while a medical student.

4. E. J. Dionne Jr., “Biden Admits Plagiarism in School but Says It Was Not ‘Malevolent,’ ” New York Times, September 18, 1987.

5. Constance L. Hays, “Martha Stewart’s Sentence: The Overview; 5 Months in Jail, and Stewart Vows, ‘I’ll Be Back,’ ” New York Times, July 17, 2004.

6. Pritha Sarkar, “Tennis: Injury Cheats Should Be Shamed on Court, Says Veteran Physio,” www.reuters.com, July 8, 2017.

7. Karen Crouse, “Patrick Reed’s Club Hit the Sand: Now There’s a Dust-Up,” New York Times, December 12, 2019.

8. Indeed, a question arose at the Farmers Insurance Open in San Diego on January 30, 2021, about whether the Patrick Reed mentioned in Crouse, “Patrick Reed’s Club,” had improperly claimed relief from an “embedded” golf ball on a wet day. Although an official called to the scene determined that Reed had done nothing wrong, Reed felt aggrieved because the same issue on the same day involving another competitor, Rory McElroy, had received less scrutiny. But because Reed had more of a history with rules issues than McElroy, the closer scrutiny of the former than the latter was hardly surprising. See Bob Harig, “Even in a Win, Patrick Reed Can’t Escape His Own History,” at www.espn.com (Jan. 31, 2021).

9. Rick Reilly, Commander in Cheat: How Golf Explains Trump (New York: Hachette, 2019). Cheating at golf appears to be a bipartisan activity. See Don van Natta Jr., “Presidential Mulligans: Taking Second Chances, Par for Clinton’s Course,” New York Times, Aug. 29, 1999; Brian Viner, “Slick Willy and Tricky Dicky Prove That Golf’s Ethics Can Land Us All in the Rough,” The Independent, May 20, 2006.

10. N. R. Kleinfeld, “Motorman’s Colleagues Say He Drank at Work,” New York Times, August 30, 1991.

11. Federal Rule of Evidence 404(b). For commentary, see Christopher Mueller, Laird Kirkpatrick, and Liesa Richter, Evidence, 6th ed. (New York: Wolters Kluwer, 2018), § 4.15 at pp. 200–207; Glen Weissenberger, “Making Sense of Extrinsic Act Evidence: Federal Rule of Evidence 404(b),” Iowa Law Review 70 (1985): 579–614. The basic principle long precedes the Federal Rules of Evidence. See Fleming James Jr. and John J. Dickinson, “Accident Proneness and Accident Law,” Harvard Law Review 63 (1950): 769–795.

12. Two recent offerings featuring the “paid his debt to society” theme are Boy A (2007) and Debt to Society (2015).

13. Good lawyers know that there are many ways around the rule as just stated. But although the rule has many exceptions and many methods of avoiding it, the basic principle remains that past acts cannot be used to show a propensity to commit acts of that type, and thus a greater likelihood of having committed such an act on the occasion in question.

14. See Michelson v. United States, 335 U.S. 469 (1948); United States v. Rubio-Estrada, 857 F.2d 845 (1st Cir. 1988); John H. Wigmore, Evidence at Trials at Common Law, rev. Peter Tillers (New York: Wolters Kluwer, 1983), § 58.2; Paul S. Milich, “The Degrading Character Rule in American Criminal Trials,” Georgia Law Review 47 (2013): 775–800.

15. Spencer v. Texas, 385 U.S. 554, 575 (1967) (opinion of Warren, C.J.).

16. See United States v. Green, 617 F.3d 233 (3d Cir. 2011). A good overview of the American law is in Richard O. Lempert et al., A Modern Approach to Evidence, 5th ed. (St. Paul, MN: West Academic, 2014), 347–357. See also Thomas J. Reed, “The Development of the Propensity Rule in Federal Criminal Cases, 1840–1975,” University of Cincinnati Law Review 51 (1982): 299–325. An important exception can be found in the controversial Federal Rules of Evidence 413–415, which permit exactly the otherwise prohibited propensity evidence in cases charging sexual misconduct, based largely on the even more controversial assumption that sexual offenses are more the product of uncontrollable tendencies than are other criminal acts. See Katherine Baker, “Once a Rapist? Motivational Evidence and Relevancy in Rape Law,” Harvard Law Review 110 (1997): 563–624; Christina E. Wells and Erin Elliott Motley, “Reinforcing the Myth of the Crazed Rapist: A Feminist Critique of Recent Rape Legislation,” Boston University Law Review 81 (2001): 127–198.

17. For an overview of the debates and positions, often characterized as the “person-situation” debate, see David C. Funder, “Personality,” Annual Review of Psychology 52 (2001): 197–221.

18. See especially Gordon W. Allport, The Person in Psychology: Selected Essays (Boston: Beacon Press, 1968); Allport, Personality and Psychological Interpretation (Oxford: Henry Holt, 1937). More recently, and responding to many challenges, see Willia Fleeson and Eranda Jayawickreme, “Whole Trait Theory,” Journal of Research on Personality 56 (2015): 82–92.

19. Stanley Milgram, Obedience to Authority: An Experimental View (New York: Harper and Row, 1974); Milgram, “Some Conditions of Obedience and Disobedience to Authority,” Human Relations 18 (1965): 57–76; Milgram, “Behavioral Study of Obedience,” Journal of Abnormal and Social Psychology 67 (1963): 371–378. On the controversies arising out of the experiments, see Thomas Blass, The Man Who Shocked the World: The Life and Legacy of Stanley Milgram (New York: Basic Books, 2004); Arthur G. Miller, The Obedience Experiments: A Case Study of Controversy in Social Science (New York: Praeger, 1986).

20. On the “situationist” reaction to the dispositionists, see Walter Mischel, “Toward an Integrative Science of the Person,” Annual Review of Psychology 55 (2004): 1–22; Mischel, Personality and Assessment (Hoboken, NJ: Wiley, 1968). See also, with varying degrees of commitment to situationism, Lee Ross and Richard E. Nisbett, The Personality and the Situation: Perspectives of Social Psychology (New York: McGraw-Hill, 1991); John M. Darley and C. Daniel Batson, “From Jerusalem to Jericho: A Story of Dispositional and Situational Variables,” Journal of Personality and Social Psychology 27 (1973): 100–108; Douglas T. Kenrick and David C. Funder, “Profiting from Controversy: Lessons from the Person-Situation Debate,” American Psychologist 43 (1988): 23–24.

21. See Rodolfo Mendoza-Denton et al., “Person x Situation Interactionism in Self-Coding (I am … when …): Implications and Affect Regulation and Social Information Processing,” Journal of Personality and Social Psychology 80 (2001): 533–544.

22. A nice overview of the various positions as applied to questions of evidence is Susan M. Davies, “Evidence of Character to Prove Conduct: A Reassessment of Relevancy,” Criminal Law Bulletin 27 (1991): 504–537.

23. Commonwealth v. Adjutant, 800 N.E.2d 346 (Mass. 2003).

24. Commonwealth v. Pring-Wilson, 863 N.E.2d 936 (Mass. 2007). Rule 404(a)(2)(B) explicitly allows a defendant in a criminal case to “offer evidence of an alleged victim’s pertinent trait.”

25. Under Federal Rule of Evidence 401, a piece of evidence is relevant “if it has any tendency to make a [material] fact more or less probable than it would be without the evidence.”

26. Maggie Severns, “Burr’s Alleged Conflicts Extend beyond His Coronavirus-Related Stock Trades,” Politico, May 15, 2020, www.politico.com.

27. I say only a “good chance” because, depending on the exact nature of the charges and other evidence, the earlier acts might be admissible to show knowledge even if not to show propensity.

28. The example is controversial, and supporters of the kinds of pit bull restrictions found in, for example, Belgium, Canada, New Zealand, Norway, most German states, and some American cities have been accused of “breedism,” “canine racism,” and worse. But the statistics—the evidence—support the conclusion that a randomly selected pit bull (America Staffordshire Terrier, technically) is more likely to be dangerously aggressive than a randomly selected dog without regard to breed. See Frederick Schauer, Profiles, Probabilities, and Stereotypes (Cambridge, MA: Harvard University Press, 2003).

Chapter 13. Seeing What We Want to See

1. The question is not about whether Salvator Mundi is a forged Leonardo, a topic—forgery—touched on in Chapter 2 and discussed in more depth in Chapter 11. Rather, the question is whether the painting was painted not by Leonardo but instead by one of his followers, students, disciples, or contemporaries. For a taste of the controversy, see Margaret Dalivalle, Martin Kemp, and Robert B. Simon, Leonardo’s Salvator Mundi and the Collecting of Leonardo in the Stuart Courts (Oxford: Oxford University Press, 2019); Ben Lewis, The Last Leonardo: The Secret Lives of the World’s Most Expensive Painting (New York: Random House / Ballantine Books, 2019); Dalya Alberge, “Leonardo Scholar Challenges Attribution of $450m Painting,” The Guardian, Aug. 6, 2018; Brook Mason, “What It Takes for a Leonardo da Vinci Painting to Be Deemed Universally Authentic,” Architectural Digest, May 22, 2019; Matthew Shaer, “The Invention of the ‘Salvator Mundi’ or, How to Turn a $1,000 Art-Auction Pickup into a $450 Million Masterpiece,” at www.vulture.com (Apr. 14, 2019); Kevin Shau, “On Leonardo da Vinci’s Salvator Mundi—Is it Authentic?,” at Art-Direct (Apr. 15, 2019), www.medium.com.

2. Brad Raffensperger, “I Have Fought to Uphold the Integrity of Elections in Georgia. It Doesn’t Matter if the Attacks Come from the Guy I Voted for or Not,” USA Today, Nov. 25, 2020.

3. Roberta Herzberg, “McCloskey versus McIntyre: Implications of Contested Elections in a Federal Democracy,” Publius 16 (1986): 93–109.

4. Jeffery A. Jenkins, “Partisanship and Contested Election Cases in the Senate, 1789–2002,” Studies in American Political Development 19 (2005): 53–74.

5. For discussion, with references, see Frederick Schauer, “Facts and the First Amendment,” UCLA Law Review 57 (2010): 897–919.

6. Ziva Kunda, “The Case for Motivated Reasoning,” Psychological Bulletin 108 (1990): 440–498; Kunda, “Motivation and Inference: Self-Serving Generation and Evaluation of Evidence,” Journal of Personality and Social Psychology 53 (1987): 636–647. See also Peter H. Ditto, David A. Pizarro, and David Tannenbaum, “Motivated Moral Reasoning,” Psychology of Learning and Motivation 50 (2009): 307–338; Ziva Kunda, Social Cognition: Making Sense of People (Cambridge, MA: MIT Press, 1999); William M. P. Klein and Ziva Kunda, “Motivated Person Perception: Constructing Justifications for Desired Beliefs,” Journal of Experimental Social Psychology 28 (1992): 145–168; Matthew J. Hornsey, “Why Facts Are Not Enough: Understanding and Managing the Motivated Rejection of Science,” Current Directions in Psychological Science 23 (2020): 583–591; Stephan Lewondowsky and Klaus Oberauer, “Motivated Rejection of Science,” Current Directions in Psychological Science 25 (2016): 217–222; Daniel C. Molden and E. Tory Higgins, “Motivated Thinking,” in The Oxford Handbook of Thinking and Reasoning, ed. Keith Holyoak and Robert Morrison (Oxford: Oxford University Press, 2012), 390–409.

7. On myside bias, see Keith E. Stanovich and Richard F. West, “Natural Myside Bias Is Independent of Cognitive Ability,” Thinking and Reasoning 13 (2007): 225–247; Keith E. Stanovich and Richard F. West, “On the Failure to Predict Myside Bias and One-Side Bias,” Thinking and Reasoning 14 (2008): 129–167; Keith E. Stanovich, Richard F. West, and E. Toplak, “Myside Bias, Rational Thinking, and Intelligence,” Current Directions in Psychological Science 22 (2013): 259–264. See also Vladimira Cavojová, Jakub Srol, and Magdalena Adamus, “My Point Is Valid, Yours Is Not: Myside Bias in Reasoning about Abortion,” Journal of Cognitive Psychology 30 (2018): 656–669.

8. See Martin Baekgaard et al., “The Role of Evidence in Politics: Motivated Reasoning and Persuasion among Politicians,” British Journal of Political Science 49 (2017): 1117–1140; Oliver James and Gregg G. Van Ryzin, “Motivated Reasoning about Public Performance: An Experimental Study of How Citizens Judge the Affordable Care Act,” Journal of Public Administration Research and Theory 27 (2017): 197–209; Dan M. Kahan, “Ideology, Motivated Reasoning, and Cognitive Reflection,” Judgment and Decision Making 8 (2013): 407–424; Kahan, “Foreword: Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law,” Harvard Law Review 126 (2011): 1–77.

9. Naomi Oreskes and Erik M. Conway, Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming (New York: Bloomsbury Press, 2010).

10. Sheila Jasanoff, Science at the Bar: Law, Science, and Technology in America (Cambridge, MA: Harvard University Press, 1995).

11. See Stephen J. Ceci and Wendy M. Williams, “The Psychology of Fact-Checking,” Scientific American, Oct. 25, 2020; Chares G. Lord, Lee Ross, and Mark R. Lepper, “Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence,” Journal of Personality ad Social Psychology 37 (1979): 2098–2109.

12. See Norwood Russell Hanson, Patterns of Discovery: An Inquiry into the Conceptual Foundations of Science (Cambridge: Cambridge University Press, 1958). Especially when it comes to science and scientists, the idea is not without controversy. See Jerry Fodor, “Observation Reconsidered,” Philosophy of Science 51 (1984): 23–43.

13. See Zina B. Ward, “On Value-Laden Science,” Studies in the History and Philosophy of Science, https://doi.org/10.1016/j.shpsa.2020.09.06 (Oct. 21, 2020).

14. Julian Reiss and Jan Sprenger, “Scientific Objectivity,” in Stanford Encyclopedia of Philosophy, https://plato.stanford.edu/archives/win2020/entries/scientific-objectivity/ (2020); Isaac Levi, “Must the Scientist Make Value Judgments?,” Journal of Philosophy 57 (1960): 345–357.

15. A good overview is Raymond S. Nickerson, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises,” Review of General Psychology 2 (1998): 175–220. See also Jane Beattie and Jonathan Baron, “Confirmation and Matching Biases in Hypothesis Testing,” Quarterly Journal of Experimental Psychology, Section A 40 (1988): 269–297; P. C. Wason, “On the Failure to Eliminate Hypotheses in a Conceptual Task,” Quarterly Journal of Experimental Psychology 12 (1960): 129–140.

16. Cass R. Sunstein, “The Law of Group Polarization,” Journal of Political Philosophy 10 (2002): 175–195. See also Caitlin Drummind and Baruch Fischhoff, “Individuals with Greater Science Literacy and Education Have More Polarized Beliefs on Controversial Science Topics,” Proceedings of the National Academy of Science, USA, 114 (2017): 9587–9592; David Schkade, Cass R. Sunstein, and Reid Hastie, “When Deliberation Produces Extremism,” Critical Review 22 (2010): 227–252.

17. Judges are fond of quoting the sportscaster Vin Scully, who observed that “statistics are used much like a drunk uses a lamppost: for support, not illumination.” In re Wachovia Corp. “Pick a Payment” Mortgage Marketing and Sales Practices Litigation, 2013 WL 5424963 (N.D. Cal. Sept. 25, 2013). Sadly, this could be said about much of the use of evidence too.

18. See Man-pui Sally Chan, Christopher R. Jones, Kathleen Hall Jamieson, and Dolores Albarracin, “Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation,” Psychological Science 28 (2017): 1531–1546, finding that a detailed debunking message correlated positively with the persistence of the misinformation that the debunking message was intended to debunk. And see also Thomas T. Hills, “The Dark Side of Information Proliferation,” Perspectives on Psychological Science 14 (2019): 323–330; David N. Rapp, “The Consequences of Reading Inaccurate Information,” Current Directions in Psychological Science 25 (2016): 281–295.

19. Lauren Giella, “Fact Check: Did Marjorie Taylor Greene Perpetuate Parkland Shooting Conspiracy Theory?,” Newsweek, Jan. 27, 2021; Andrew Solender, “Trump-Backed Candidate Marjorie Taylor Greene Promotes 9 / 11 Conspiracy Theory,” Forbes, Aug. 13, 2020.

20. On just this usage of “Panglossian,” here in the context of a resistance to recognizing that that which is morally bad might be legally correct, and that that which is morally good might nevertheless be illegal, see Jeffrey Brand-Ballard, Limits of Legality: The Ethics of Lawless Judging (New York: Oxford University Press, 2010), 86–88, 311–312. For my own use of Brand-Ballard’s sense of Panglossianism, see Frederick Schauer, “Rights, Constitutionalism, and the Perils of Panglossianism,” Oxford Journal of Legal Studies 38 (2018): 635–652.

21. See Troy H. Campbell and Aaron C. Kay, “Solution Aversion: On the Relation between Ideology and Motivated Disbelief,” Journal of Personality and Social Psychology 107 (2014): 809–814; Dan H. Kahan, Hank Jenkins-Smith, and Donald Braman, “Cultural Cognition of Scientific Consensus,” Journal of Risk Research 14 (2011): 147–174.

22. Overviews include Gerrit Antonides, Psychology in Economics and Business (Dordrecht: Springer, 1991), 193–214; Bertram Gawronski and Fritz Strack, eds., Cognitive Consistency: A Foundational Principle in Social Cognition (New York: Guilford Press, 2012); Dan Simon, Chadwick J. Snow, and Stephen J. Read, “The Redux of Cognitive Constraint Theories: Evidence by Constraint Satisfaction,” Journal of Personality and Social Psychology 86 (2004): 814–837.

23. Leon Festinger, Conflict, Decision, and Dissonance (Stanford, CA: Stanford University Press, 1964); Eddie Harmon-Jones, ed., Cognitive Dissonance: Reexamining a Pivotal Theory in Psychology, 2nd ed. (Washington, DC: American Psychological Association, 2019); Eddie Harmon-Jones and Cindy Harmon-Jones, “Cognitive Dissonance Theory after 50 Years of Development,” Zeitschrift für Sozialpsychologie 38 (2007): 7–16.

24. Jordan R. Axt, Mark J. Lamdaum, and Aaron C. Kay, “The Psychological Appeal of Fake-News Attributions,” Psychological Science 31 (2020): 848–857.

25. Marianne Levine, “Democrats’ Big Shift in Trump’s Second Impeachment,” Politico, Feb. 7, 2021, at www.politico.com.

26. Charles S. Taber and Milton Lodge, “Motivated Skepticism and the Evaluation of Political Beliefs,” American Journal of Political Science 50 (2006): 755–769. Even more depressing is the way in which evidence countering a false belief may increase rather than decrease the persistence of the false belief. See Chan et al., “Debunking.”