Before proceeding to criticize specific methodologies and propose replacements, it's essential to establish the fundamentals. There is an array of underlying methodological assumptions that all historians should agree on, and which should guide any inquiry into the mechanics of historical method. Misunderstanding can result if these are not laid bare from the start. This chapter surveys these assumptions and why we should all share them.
WHY HISTORY REQUIRES EXPERTISE
To laypeople who ask me what history to trust, I always offer three basic rules: (1) don't believe everything you read; (2) always ask for the primary sources of a claim you find incredible; and (3) beware of scholars who make amazing claims about history but who are not experts in the period, or aren't even experienced historians at all. That three-step guideline provides a basic inoculation against most bad history. But professional historians already know this. Indeed, historians (especially historians of antiquity) know why that third rule is so important. There are four stages of analysis we must complete to credibly examine a historical claim:
First is textual analysis. We have to use the methods of textual criticism and paleography to ascertain whether a document we presently have is authentic and accurately reflects its original—since usually only copies of copies exist today. Critical editions of ancient texts are the result of this process, but reading them requires knowledge and skill, which amateurs rarely have (or have enough of). The corollary for physical artifacts and sites (and other nontextual evidence from the past) is the establishment of authenticity and provenance, which is the purview of archaeologists, antiquarians, and forensic specialists, whose function and importance is the same. Doing all this, or assessing whether it has been done well and what its results mean, requires skill, training, and experience.
Second is literary analysis. We have to ascertain what the author of a text meant. And that requires a good understanding of the language as it was spoken and written in that time and place, and a strong grasp of the historical, cultural, political, social, economic, and religious context in which it was written, since all of that would be on the mind of both author and reader, and this would illuminate, motivate, or affect what was written. Conclusions regarding a text's genre and function fall into this category. Yet knowing all that requires considerable experience studying all those aspects of the culture in question, which is a major factor that separates professionals from amateurs. This is especially true because cultural contexts and assumptions have changed considerably, so intuition will often fail anyone who is not well familiar with those changes. What we assume about genre and the semantic ranges of words in an ancient text, for example, if based solely on experience with modern literature and vocabulary, will be wrong; likewise for assumptions about what was normal, known, or believed in that time and place. Ancient literary genres differed from ours. The range of meanings their words conveyed was different. People knew and believed different things and had very different ideas about what was normal or unusual, and indeed different things were normal or unusual than are now. It's impossible to know what all these differences were without broad and extensive study.
Third is source analysis. We must try to identify and assess an author's sources of information. This, in turn, requires knowing a lot about what sources existed then and survive now, what sort of sources an author will have used in that time and place (which again requires vast contextual knowledge only professionals tend to have), and, methodologically, how to ascertain when a particular claim or passage uses a source at all, or some known or hypothesized source in particular (skills amateurs might not be so adept at). More often than not, especially in ancient history, we simply will never know what sources an author used or if they even used any, and even when they tell us, we know they often lied or failed to accurately tell us just what it is they got from that source as opposed to assumed or added themselves. But still key is knowing what was available and what was typical, in terms of what kinds of sources existed and how they were used. Thus, in both respects, expertise is indispensable: knowing the overall milieu of ancient methods and behaviors, and knowing the specific authors and “source books” that existed and what they said. Amateurs typically don't have this information.
Fourth, and only last, is historical analysis proper. This cannot proceed until the first three stages have been completed. Those three stages are often completed by other experts and specialists upon whose work subsequent experts rely. We build on each other's work in that respect; for example, trusting that textual critics have produced the most reliable reconstructed text possible of an ancient book (like the Bible). But knowing when to rely on that groundwork, and how to understand it correctly and critically, requires its own set of skills and experience. Moreover, the skills involved in all four levels of analysis, in particular the fourth, require honing under the guidance of watchful experts, since experience only teaches when you know when you've done something wrong, so you can correct and avoid such mistakes in the future. Without such feedback you can never course-correct, and you will simply persist in any errors you became enamored with at the start. You must have experts at hand to guide you for many sustained years, such as professors and professional advisors, who can quickly identify common mistakes in your arguments, or, by informing you of facts overlooked, reveal how easy it is to overlook important facts and show you how to avoid doing that. Then, they must ensure that you draw conclusions from those facts correctly (which means logically). All so you will learn what constitutes an error and how to correct or avoid such errors in future. Professional historians receive years of training like this. Accordingly, their subsequent work product is much more reliable.
For all these reasons, laypeople must depend on qualified experts to report to them what most likely happened in the past and why. Laypeople need only have the skill to discern experts from nonexperts and to discern when to trust what an expert says. This book is not about that, but about what methods experts themselves should employ. But laypeople can improve their discernment of expert testimony if they, too, are familiar with expert methods, at least well enough to know when an expert is actually using them. This chapter will lay the initial groundwork for that by defining the basic principles of expert history that are not controversial. The rest of this book will then propose that something new be added to the arsenal of professional historical method.
THE AXIOMS OF HISTORICAL METHOD
Professional historical inquiry should be based on a set of core epistemological assumptions that I call the axioms of historical method. I take each as axiomatic, not in the sense that I can't defend them (I certainly can), but in the sense that insofar as they are to be defended, that defense comes from the field of philosophy, not history. If anyone rejects my axioms, then no further dialogue is possible on this issue until there is agreement on the broader logical and philosophical issues they represent. Producing such agreement is not the point of this book, which is only written for those who already accept these axioms (or who at least agree they should). These twelve axioms represent the epistemological foundation of rational-empirical history.1
Axiom 1: The basic principle of rational-empirical history is that all conclusions must logically follow from the evidence available to all observers.
By “basic principle” I mean sine qua non (“without which, not”), a principle without which you cannot have a rational-empirical inquiry. This means (a) private intuition, personal emotions and feelings, inspiration, revelation, or spirit communications cannot be a primary source of evidence and (b) all conclusions argued from the agreed evidence must be logically valid and free of all fallacies.2
Axiom 2: The correct procedure in historical argument is to seek a consensus among all qualified experts who agree with the basic principle of rational-empirical history (and who practice what they preach).
By “correct procedure” I mean this is the only truth-finding procedure that performs well enough to trust—which means this is the only procedure that gets us to the truth at a rate significantly better than chance. By “qualified expert” I recognize this standard comes in degrees (e.g., there is a difference between being qualified merely as a historian and being qualified as a specialist in a specific field like ancient Roman history) and that merely being qualified is not a sufficient condition for knowledge (even the most qualified experts remain unaware of findings and developments in their own field, as there is often far too much written even in highly specialized fields for any mortal to have read it all). Hence, generating consensus is a slow process that radiates outward in circles of authority, from specialists to generalists, as an argument is continually advanced and publicized through proper channels (meaning established channels, those which experts themselves trust).
Proper historical argument consists of seeking this growth of consensus and entails everything that that requires (diplomatically, rhetorically, and procedurally—hence the purpose of peer review and my recommended twelve rules, soon to follow). This process cannot be bypassed, as specialists in a field are the most qualified to assess an argument in that field, so if they cannot be persuaded, no one should be (unless their resistance can be proven—not merely assumed—to have other motives than truth-seeking). Conversely, if they are persuaded, everyone else has a very compelling reason to agree (unless, again, their acceptance can be proven—not merely assumed—to have other motives than truth-seeking). This is the social function and purpose of having such experts and specialists in the first place.3
This consensus-seeking results in a dialectic of criticism and revision, which allows errors and gaps to be identified and corrected and arguments and evidence to be trimmed or fortified, until, by the time an argument reaches a wide consensus, you will have an even stronger conclusion than you started with (since the probability of overlooked facts or errors declines with every peer who examines the case), or else you will have discovered your conclusion is incorrect or unwarranted (as you come to realize the criticisms amount to an adequate refutation). Hence, the process of argument must be to make your best case, then ask the community of qualified experts to rebut it, and then respond to their critique, repeating this cycle of exchange until either of two things happens: you must reasonably accept that your arguments, finally revised in light of wide criticism, do not suffice to justify your conclusion after all; or your critics will accept that their rebuttals are insufficient to warrant rejecting your conclusion. The latter results in a revised consensus on which future scholarship can then build. The former results in putting away an interesting but ultimately untenable theory.
This process works precisely to the extent that every expert adheres to the first axiom. Because conclusions that logically follow from public facts are exactly those in which there should be a consensus, and anyone who accepts such conclusions will join that consensus—whereas conclusions that do not logically follow from public facts should be rejected, and will be by anyone devoted to the first axiom. Although they might still accept such conclusions for reasons other than history, by virtue of their commitment to the first axiom they will admit that, and they will distinguish what can be arrived at from historical evidence alone and what can only be arrived at by other means.
It's still true that some experts will give lip service to the first axiom but not follow it, or erroneously believe they are when they aren't. But in the first case they will eventually be exposed by this process as frauds (as more and more peers demonstrate their hypocrisy), and in the second case the process itself will eventually correct them (as more and more peers demonstrate their error). Consequently, this process works the better the more experts in a field adhere to the first axiom competently and consistently. The epistemic reliability of a community of experts can be gauged by exactly that measure.
Axiom 3: Overconfidence is fallacious; admitting ignorance or uncertainty is not.
Ignorance and uncertainty are common and normal. But asserting as known or certain what in fact isn't entails some fallacy in your reasoning. One thing professional historians soon learn is how much we need to accept the fact that we will never know most of what we want to know—such as about Jesus or the origins of Christianity, or anything else in history. Compared to, for example, Richard Nixon or Mark Twain, the documentation for Jesus and the origins of Christianity is extraordinarily thin and problematic. And yet even knowing all we'd like to know about Nixon or Twain is impossible, as even for them the evidence is neither complete nor unproblematic; for Jesus and the origins of Christianity, vastly more so.
Anyone who rejects this conclusion is not an objective scholar, but a dogmatist or propagandist whose voice needn't be heeded by any respectable academic community. Likewise, most of what we can say, especially about ancient history, is “maybe” or “probably”—not “definitely.” There is obviously more than one degree of certainty. Some things we are more sure of than others, and some things we are only barely sure of at all. Hence, especially in history, and even more so in ancient history, confidence must often be measured in relative degrees of certainty, and not in black-and-white terms of only “true” and “false.” Accordingly, historians must be comfortable with ambiguity, uncertainty, and ignorance, and must critically weigh and examine their own confidence in any conclusion. The difference between a real expert and a poser is often evident in that very distinction.
Axiom 4: Every claim has a nonzero probability of being true or false (unless its being true or false is logically impossible).
Not only must we be prepared for uncertainty; we must accept that it's everywhere, differing only in degree. For anything that's possible could yet be true. By “possible” here I mean a claim that is possible in any sense at all (as opposed to a claim that is logically impossible), and by “probability” here I mean “epistemic probability,” which is the probability that we are correct when affirming a claim is true. Setting aside for now what this means or how they're related, philosophers have recognized two different kinds of probabilities: physical and epistemic. A physical probability is the probability that an event x happened. An epistemic probability is the probability that our belief that x happened is true. For example, the probability that someone's uncle invented the global positioning system is certainly very small (since only a very few people out of the billions living on earth can honestly make that claim). But the probability that your belief “my uncle invented the global positioning system” is true can still be very high. All it takes is enough evidence. The former is a physical probability, the latter an epistemic one. I will establish the proper relationship between physical and epistemic probabilities in chapter 6. For now, know only that unless the context indicates otherwise, when I speak of probability, I mean epistemic probability—though you will notice that often there seems to be no practical difference.4
Epistemic probability is the probability that we are correct in any given belief. And with its converse, we measure the probability of being mistaken. For example, if (given all we know at the moment) a claim has a 25% probability of being true, then if we say that claim is true, there is a 75% chance we are mistaken, but if we instead say that claim is false, then there is only a 25% chance we are mistaken. Therefore, if we say such a claim is false, we will more likely be correct. And so we say the claim is false. But it still has some probability of being true. Accordingly, when we say something is “probable,” we usually mean it has an epistemic probability much greater than 50%, and if we say it's “improbable,” we usually mean it has an epistemic probability much less than 50%. Everything else we consider more or less uncertain.
All claims have a nonzero epistemic probability of being true, no matter how absurd they may be (unless they're logically impossible or unintelligible), because we can always be wrong about anything. And that entails there is always a nonzero probability that we are wrong, no matter how small that probability is. And therefore there is always a converse of that probability, which is the probability that we are right (or would be right) to believe that claim. This holds even for many claims that are supposedly certain, such as the conclusions of logical or mathematical proofs. For there is always a nonzero probability that there is an error in that proof that we missed. Even if a thousand experts check the proof, there is still a nonzero probability that they all missed the same error. The probability of this is vanishingly small, but still never zero.5 Likewise, there is always a nonzero probability that we ourselves are mistaken about what those thousand experts concluded. And so on. The only exception would be immediate experiences that at their most basic level are undeniable (e.g., that you see words in front of you at this very moment, or that “Caesar was immortal and Brutus killed him” is logically impossible). But no substantial claim about history can ever be that basic. History is in the past and thus never in our immediate experience. And knowing what logically could or couldn't have happened is not even close to knowing what did. Therefore, all empirical claims about history, no matter how certain, have a nonzero probability of being false, and no matter how absurd, have a nonzero probability of being true.
Therefore, because we only have finite knowledge and are not infallible, apart from obviously undeniable things, some probability always remains that we are mistaken or misinformed or misled. Our challenge, then, is to believe only claims that we are very unlikely to be mistaken, misinformed, or misled about. But many things have different levels of certainty and thus different degrees of probability. And although the probability that a given claim is true (or false) may be vanishingly small and thus practically zero, it is never actually zero. It's vital to admit this. For the truth is not always what is physically most probable, since improbable things happen all the time. If we know nothing else, often we can still at least say what's most likely to have happened, and that may then be what's most credible to believe. But that's not the same as saying the alternative can't be true. We may have to admit it could be true, even if we don't think it is. And we may have to decide just how likely or unlikely either conclusion is, quite apart from how likely or unlikely the proposed event is.
Almost everything that happens is in some sense improbable—from the specific conjunction of your own unique DNA to the specific sequence of people you might meet on any given day. And yet it happens. Though being struck by lightning is very improbable, it nevertheless happens to hundreds of people every year. And if your wallet turns up missing, regardless of which is more probable—it being stolen or your having misplaced it—either could turn out to be true. Arriving at a reasonable conclusion as to what is the more likely explanation of any conjunction of facts will require comparing the relative probabilities of all the pertinent evidence on different theories (as I'll demonstrate in subsequent chapters), which requires admitting that theories you don't believe in nevertheless have some probability of being true, and theories you're sure are true nevertheless have some probability of being false. And you have to take seriously the effort to measure those probabilities. For when you do, you may find you can't sustain the level of certainty you once had.
In short, “that's impossible” almost always means, rather, “that's very, very improbable.” Once you acknowledge that, you will be forced to ask: How improbable? Too improbable to believe? Why? And how do you know? Any sound methodology must provide the means to answer those questions. Failing to do this is to replace knowledge with thoughtless assumption. For this and all the reasons above, throughout this book whenever I refer to “knowledge” or to what you or we “know,” I will not be assuming the philosopher's definition of knowledge (“justified true belief”), but using the terms “knowledge” and “know” as shorthand for “what we think we know, and with very good reason” (in other words, well-justified belief), so that anything we claim to “know” we could be wrong about, but are very unlikely to be.6 I am therefore excluding mere beliefs (things we aren't sure we know but believe anyway, and things we think we know but without a very good reason) and everything we don't in any sense yet know. Defined this way, knowledge does not require an impossible standard of certainty. It's simply what we are very probably right about.
Axiom 5: Any argument relying on the inference “possibly, therefore probably” is fallacious.
Though we must admit anything that's possible could yet be true, that does not argue for anything actually being true. This is a form of modal fallacy I call possibiliter ergo probabiliter (“possibly, therefore probably”) and it's so common in historical argument that it deserves particular attention. Just because you can conceive of a possible alternative explanation does not entail that your alternative is actually more likely (or in any way likely at all). For example, historians will often dismiss an Argument from Silence by proposing some explanation for why a document is silent on that detail. Of course, knowing why we don't have certain evidence still does not change the fact that we don't have that evidence. All you can then say is that this lack of evidence is inconclusive, not that it supports one conclusion over another. But more importantly, just because you can think of a reason to explain away a document's silence does not mean that reason is probable, and if it isn't probable, it isn't a valid objection to an Argument from Silence—so treating it as if it were is a fallacy (I'll further discuss the logic of an Argument from Silence in chapter 4, page 117). Likewise, historians often do little more than conceive of some possible ‘just so’ story to explain the evidence, and assume that because their interpretation fits, therefore it's true. But that's the same fallacy. An infinite array of possible explanations can be deployed for any set of evidence, and quite a lot of them will even seem an uncanny fit, yet at most only one of them can be true, which means merely finding a seemingly uncanny fit between evidence and theory is a completely unreliable method to employ.
In such ways as these, historians often assume they've won their argument if they can think of any possible explanation contrary to their opponents’. But this is the same modal fallacy again—unless the alternative is shown to be not merely possible, but highly probable (or at least probable enough to carry whatever burden is required of the argument). For example, if a historian has a very good reason to expect a document's silence on a particular detail, then that silence cannot argue against anything—though neither does this fact argue for anything. Likewise, if a fit between text and interpretation can be demonstrated not merely to seem uncanny, but to actually be uncanny (for example, by proving such a fit is very improbable unless it was the author's actual intent), then that does support that interpretation. By the same token, recognizing this fallacy is what permits us to reject absurd claims even though we grant they still have some tiny probability of being true. Like a thousand mathematics professors missing the same mistake in a formal proof: yes, it can happen, but it's so unlikely there will never be a reason to believe it happened, until we can prove it did. Likewise all the canards of radical skeptics, like “you don't know your house really exists, because you could be a brain in a vat.” That's just the same fallacy. Yes, we could be a brain in a vat. But that's extraordinarily improbable. Therefore, when we conclude our house really exists, we're still very probably right.
Just as often overlooked (or downplayed) by historians is the fact that many theories we can posit may indeed be true, yet the evidence may still be insufficient to prove them or to warrant believing them. We thus must be willing to admit what we don't know at all, and what we don't know for certain, and not mistake our not knowing whether a theory is true for our knowing it is false. In other words, just because “possibly, therefore probably” is a fallacy doesn't mean a conclusion produced by this fallacy is false, it only means a conclusion is not established by merely being possible. Consider the claim that Caesar shaved on the morning of May 12, 52 BCE, or that Caesar once played dice with a hooker named Maxsuma. We have no evidence of these facts (I just made them up). Yet not believing these things is not the same as believing they are false. Either claim may well be true, even if they were improbable. Indeed, even if they were very probable they may yet be false. More to the point, the fact that we can easily explain the silence of extant documents on these claims does not constitute a valid argument for believing them. And yet, not believing those things is not the same as not believing Caesar rode a winged horse or that Caesar once camped on the moon. We can express uncertainty on what days Caesar shaved or what hookers he may have diced with, allowing a great deal to be possible on either point without claiming to know, but we wouldn'tallow the same latitude for his flying horse or imperial moon base—knowing full well the prior probability of either of the latter is far too low to credit, unlike the prior probability of his shaving on any given day or ever having played dice with a hooker of a particular name.
Hence, though “possibly, therefore probably” is fallacious, there is no inherent evil in speculation, or in the exploration of possibilities. There are a great many things that are true about the ancient world and its people and events that we today have no evidence of at all, and we need to take seriously the range of what could yet have happened though it remains unknown to us.7 But such speculation must be used sparingly and appropriately. Theories and generalizations that are weakly supported need not be dismissed, if their plausibility can be adequately defended, but they must never be treated as anything more than they are—plausible possibilities, not confirmed facts—unless, of course, the facts at hand make them not merely plausible, but probable.
Axiom 6: An effective consensus of qualified experts constitutes meeting an initial burden of evidence.
An effective consensus of qualified experts (by which I mean at least 95 percent agreement) is probably true unless a strong and valid proof arises that it is not. This is a straightforward fact of frequency: the methods that generate such a consensus far more frequently discover the truth than err; therefore, any given result of that consensus is far more probably true than false. And this is a consequence of cumulative probability: it is far more unlikely that an incorrect argument would persuade a hundred experts than that it would persuade only one; and it's far more unlikely that it would persuade any expert than that it would persuade even a hundred amateurs. The fact that the one is harder to do than the other acts as a kind of truth filter: only very well argued conclusions survive it. And that's the very point of requiring it to be survived.
Even if we demonstrated (not merely asserted, but actually proved) that the consensus has been improperly generated (e.g., if we actually proved it is based on dogma or tradition or a repeated error rather than a genuine application of sound methods), that would only be sufficient to establish that the consensus position has not been properly established, not that it is false or that any alternative is instead correct. So even under those conditions one must still present sufficient evidence for any alternative conclusion in order to reverse the consensus. Of course, if properly pursued, such an effort would then aim at becoming the consensus. That's how fields advance and make progress; by catching and purging their own errors. But the process must still be followed.
Hence, the burden of proof clearly falls on anyone who would challenge an existing consensus, despite repeated attempts to deny this. For example, in the matter of whether Jesus actually existed as a historical person, historicists have already met the burden of evidence to produce a consensus of qualified experts. So the deniers of historicity must overcome that burden with their own. Attempts to argue that the current consensus has been improperly generated have some merit (e.g., many historicists do make assertions out of proportion to the evidence, or simply cite “the consensus” without checking how that consensus was actually generated). But such arguments are still inadequate, since there is certainly prima facie evidence for a historical Jesus (hence, historicity need not be asserted dogmatically, even when it is). Moreover, such a claim (that there is an improperly generated consensus) must still meet its own burden of evidence. In fact, any claim that the consensus is actually wrong (and not merely unfounded) requires meeting an even greater burden of evidence (in defense of some alternative theory).
This usually means that a strong consensus of experts entails a high prior probability the consensus theory is true. Yet occasionally this is not so. I believe there is ample reason to conclude that the consensus is not reliable in the study of the historical Jesus and therefore cannot be appealed to as evidence for a conclusion. Yet I still bear the burden of proving that (which I shall partly undertake here in chapter 5, although the facts just surveyed in chapter 1 already go a long way toward making the case). And the prima facie evidence for a historical Jesus, which constitutes all the valid evidence the consensus could ever appeal to, still cannot be ignored. But it should be examined anew (a task I'll undertake in the next volume).
Axiom 7: Facts must be distinguished from theories.
Another common error is the conflation of facts with theories. Proper facts are actual tangible artifacts (such as extant manuscripts and archaeological finds) and straightforward generalizations therefrom. Everything else is a theory as to how those facts came about. Some such theories can be so well and widely confirmed that we consider them facts, but one should never assume a theory is that well confirmed before checking to ensure that it is (and merely having an effective consensus of qualified experts is not always enough to confirm this).8 For example, “the first Christians found Jesus’ tomb empty” is not a fact. It is a theory, which is proposed to explain what actually is a fact: that certain later stories arose describing such a find, which now survive in various manuscripts. But there are alternative theories of how those facts came about (e.g., that those stories arose originally to convey a symbolic meaning and were embellished later as propaganda).9 That there are alternative theories of the evidence does not mean those theories are true. It may well be that an actual empty tomb is far more likely. But that has to be argued. It requires sufficient demonstration to warrant belief. It cannot simply be assumed as a fact.
Similarly, if a myth proponent wants to propose that a certain name in the Gospels symbolizes a particular astrological sign, he cannot simply claim that as a fact. It's a theory, and it can only be credible to the degree that it's the best explanation of the facts alleged to prove it (which entails considering what some other explanations of that name might be). By conflating facts with theories, very often an entire burden of evidence that is actually required is ignored or never met. This distinction is all the more important for lay readers, who will not know the difference if it's not made clear to them. Consequently, historians should be much clearer than they have been in distinguishing confirmed facts from proposed theories. As a rule, whenever there is a nonnegligible chance you are wrong, you are talking about a theory, not an established fact. And it's often the case that if you have to argue a point, there is a nonnegligible chance you are wrong.
Axiom 8: A conclusion is only as certain as its weakest premise.
It's essential to watch for the weakest link in any argument, because very often a single weak link will render all resulting conclusions just as weak—or, by their accumulation, even weaker. This frequently happens in historical reasoning when qualifiers are snuck in without accounting for their logical consequences. For example, in any argument, analyzed formally, if there is a premise of the form “maybe x,” then any conclusion depending on that premise cannot be any more certain than “maybe.” This follows for any qualifying language (like “probably,” “possibly,” or “perhaps”). For example:
|
MINOR PREMISE: |
Alexander might have wanted to assassinate his father Philip. |
|
MAJOR PREMISE: |
If Alexander wanted to assassinate his father, then he probably arranged his assassination. |
|
CONCLUSION: |
Therefore Alexander probably arranged Philip's assassination. |
This is a fallacious conclusion, since the minor premise does not say “probably” but “might,” which makes it a weaker premise than the major premise requires. But the conclusion cannot be more certain than the weakest premise. Therefore, the only valid conclusion one could produce here would be “Alexander might have arranged the assassination of Philip,” which is such a trivial conclusion as to be useless (since it's true of practically anyone of the time, and predictive of nothing).
This may seem an obvious point to make, but ignoring it is a frequent error among historians, who often make bold assertions from more hesitant premises, forgetting that somewhere along the line of their argument one of their key supporting points was more speculative than their conclusion would suggest. And it only takes one weak premise to render all resulting conclusions equally weak, no matter how strong every other premise may be or how many other strong premises there are. Yet I've seen both sides of an argument take ambiguous evidence like this and derive unambiguous conclusions from it, making this a remarkably common error.
Axiom 9: The strength of any claim is always proportional to the strength of the evidence supporting it.
In line with the axioms above, certainty must be proportional to evidence. The strength of a claim means the likelihood of it being true, which (as already noted) is the likelihood of our being mistaken if we denied it was true. But the strength of the available evidence is measured not just by quantity. In fact, quantity can mean little if the evidence is not independent. For example, having a thousand copies of a letter does not make the claims in that letter any more true, but having ten different witnesses reporting a fact independently of each other does (provided nothing else calls their testimony into question). But apart from quantity, strength is also measured by the certainty, credibility, and uniqueness of any potential causal connection between the evidence and what it is claimed to support, or by how securely and abundantly the evidence establishes a broader generalization that makes a particular claim more likely or unlikely. I'll discuss the issue of evidentiary strength in coming chapters. But in every case, our conclusions must be proportionate. We must never assert a claim to be more certain than the evidence warrants.
Axiom 10: Weak claims that contradict strong claims are probably false (and not the other way around).
Any weak claim will by definition have a lower probability of being true than a strong claim. Therefore, if a weak claim contradicts a strong claim, all else being equal, more probably than not the weak claim is false. This does not entail that the strong claim is true. But it does entail that only the strong claim could then be asserted as the most likely of the two, which means a strong claim can never be refuted with a weaker one. All too often, historians will attempt to rebut a well-supported claim by appealing to a claim whose support is much less secure. Such an approach is fallacious and thus, per the first axiom, should be rejected by all expert historians.
Axiom 11: Generalizations must be supported by evidence, and that evidence must consist of more than one example (or of an example that strongly implies a general trend), and once supported, cannot be ignored.
All too often generalizations are declared (such as about what Romans or Jews typically did or thought or said) without any supporting evidence at all—or with only one instance, which is insufficient to demonstrate what was typical, unless the character of that instance strongly supports a conclusion that it is in fact an instance of what was typical. One should also not confuse what was typical with what was possible, not only because of the fifth axiom, but also because there will often be exceptions to any generalization. If you assert a generalization as absolute (i.e., without exception), that carries a far greater burden of evidence than any more ordinary generalization. For every generalization entails its converse, which must be just as defensible on the same evidence: for example, the converse of the generalization “everyone always read aloud” is the generalization “no one ever read silently.” If the evidence is insufficient to support the latter, it's insufficient to support the former. By contrast, evidence supporting “everyone usually read aloud” is not sufficient to support “no one ever read silently.” And ‘usually’ is already a hard thing to prove from just a few examples.10
Importantly for this axiom, generalizations are not limited to historical trends but also include all rules of inference. If you apply a general rule of inference in some cases and not others, then when you don't apply the rule you must be able to apply another general rule of inference that justifies not applying that rule in that special case. And that other general rule must itself be demonstrably exceptionless (i.e., it always applies in every case) or ultimately supported by one that is. Hence, any system of general rules of inference that you construct and employ must be internally consistent, both in theory and in practice. This means you cannot arbitrarily apply or fail to apply some rule simply when it suits you. To the contrary, any such negation of a rule must be justified by another valid rule. Hence, this eleventh axiom invalidates ‘cherry-picking’ and ‘special pleading’ and other abuses of logic and evidence.
Axiom 12: When one of us cites a scholar, it should only be assumed we agree with what they say that is essential to the point we cite them for.
This is an axiom we should apply to all scholars and authors, but I have a particular interest in asserting it here. Though I may agree with many of the scholars I cite on much else besides what I cite them for, it's still a fallacy to assume I do without evidence to that effect. Of course, proving I agree with some additional point by adducing evidence that I do is not assuming I do but arguing I do, which is valid. But beyond that, it will be a fallacy to argue against me by arguing against something said by someone I cite which is not necessary to anything I myself have said.
This kind of ‘baggage’ fallacy (often deployed as a variety of the textbook fallacy of “poisoning the well”) is common enough to warrant particular condemnation. In fact, I see this fallacy committed so regularly, so widely, by accomplished scholars who ought to know better, that I feel the need to call particular attention to it now, in the hopes it will forestall a repeat performance. If you cite a scholar as proving point A, and that same scholar also argues B, but B is not necessary to A, then it is a fallacy for anyone to assume you agree with B, and a fallacy to employ this assumption to argue that if B is not credible then A is not credible. I call this the ‘baggage’ fallacy because it amounts to saddling an author with all the ‘baggage’ attached to the scholars he cites or the views he defends, when such attachment is neither entailed nor warranted. Just because I take certain positions or arrive at certain conclusions is no excuse to impute to me all the baggage that is usually supposed to come along with those positions or conclusions.
For example, when I argue a point (such as that distinct elements of Osiris cult can be seen in early Jesus cult), it might be assumed I agree with something else that supposedly goes along with this (such as that all elements of Osiris cult were present in early Jesus cult, or that Jesus is merely Osiris under another name, or that Christians just “borrowed” and “revamped” an Egyptian religion). That would be mistaken. It would likewise be mistaken to assume I agree with every other alleged parallel that has in the past been made between Jesus and other pagan gods, or that I agree with every theory as to why or how such parallels came to exist simply because I agree there are some meaningful parallels with pre-Christian gods. The same fallacy also results when I agree with something a particular book said, or cite it as a reference of importance on a specific subject, and then it's assumed I agree with everything that book said or its author elsewhere defends.
It might be argued that a scholar who makes a bad argument can't be trusted to ever make a good one, therefore citing the flaws in argument B as counting against argument A “should” be appropriate, but that's still fallacious. You can't know if any particular argument is good or bad until you actually look at it. Even the most stupid, ignorant, and incompetent author can make a good argument from time to time, documenting all the evidence and making logically valid inferences from it. That may be improbable. Which is why it is perfectly right to ignore an untrustworthy source as being a waste of your time. But it is not valid to ignore that source when a real expert finds a point they make credible and cites their work establishing it. And as an expert, I only cite sources that argue for a point in a way that is valid and sound (and of use to you to consult), regardless of what that source does otherwise. That's why you must examine my source's arguments before dismissing them, rather than attacking the author who made them.
In short, do not assume what's not in evidence. I might have many disagreements with the scholars I cite on any given point, but on matters not relevant to that point. Hence I often omit these disagreements. But my omitting them should not be mistaken for my having none. Likewise I emphasize what I believe is most defensible and build a case therefrom. But such a procedure leaves out countless details about which countless questions could be asked. Again, my omitting such things should not be mistaken for my not being aware of them. Still, if any of these omitted details undermine my main argument, it is still valid to call attention to that fact. Because my omission of such questions and details may correctly be taken as indicating that I don't believe they undermine or seriously challenge that argument. In other words, as my research in this matter has been extensive, I believe anything I have omitted can be resolved or worked out in ways perfectly consistent with my core argument. But I could be wrong about that. Hence, I always welcome a sound critique from my peers.
Nevertheless, this axiom cannot justify relying on gratuitously bad scholarship. Those who should get little or no mention are scholars who do not employ an adequate method of citation and referencing and consequently make many dubious or false claims or claims incapable of confirmation. Their work is of no use to laypeople (who can't assess which claims are credible or dubious) and of little use to experts (who have to redo all their research anyway before trusting what they say, which negates the point of reading them). Some scholars straddle the line, having insightful things to say, yet I have to fact-check anything they said before relying on it myself—the punishment for which is that I don't cite them if I had to do the work. I just cite the evidence instead.
THE TWELVE RULES OF HISTORICAL METHOD
In addition to the twelve axioms just explained, there are also twelve rules I would like to see all historians consistently follow, in order to make their work more credible and worthwhile, and to make progress possible. Though consensus-challengers are more frequently guilty of not following these rules, no one is without sin on this score, and we all fail at them from time to time (myself included). They are the standards by which we seek to correct ourselves and be corrected by our peers. Again, none should be controversial—except, of course, the second part of rule one.
Rule 1: Obey the Twelve Axioms (given above) and Bayes's Theorem (articulated in the remainder of this book). This does not mean you must use Bayes's Theorem in any mathematical sense, only that any historical argument you employ must not violate Bayes's Theorem.
Rule 2: Develop wide expertise in the period, topics, languages, and materials that you intend to blaze any trails in, or else base all your assumptions in these areas on the established (and properly cited) findings of those who have.
Rule 3: Check all claims against the evidence and scholarship, especially generalizations and assumptions (i.e., don't assume that because you heard or read it somewhere or it just seems plausible that therefore it's likely or true).
Rule 4: Confirm that an argument follows from the original language of a text with as much assurance as from your preferred translation. And confirm that your preferred translation fits the original context (both textual and sociocultural).
Rule 5: Phrase all your claims for optimal truth value. Use all necessary qualifications; avoid hyperbole; do not state as fact what is not fact or as certain what is not certain; always express degrees of certainty or uncertainty when appropriate; acknowledge the difference between a speculation and an assertion; and concede when more research is needed.
Rule 6: Don't conflate weakly supported claims with strongly supported claims, or confuse theories with facts or speculations with theories. Always be explicit in all your writings as to which is which.
Rule 7: Address all relevant and significant evidence against what you claim (including any relevant arguments from silence against what you claim).
Rule 8: Take into account problems of chronological development. Everything changed over time, and documents written much later may or may not reflect earlier views or practices, regardless of what they claim. Hence, for example, any argument for influence requires evidence not just of parallels and similarities but of the causal direction of that influence (although this works both ways: just because one source comes later than another does not entail the causal direction runs the same way, as the later source could still be attesting a tradition that predates the earlier source).
Rule 9: Always cite your primary evidence, or cite sources that either cite the relevant primary evidence themselves or cite further sources that collectively do (primary evidence being the earliest surviving evidence in the chain of causation, e.g., a modern or medieval historian citing an ancient historian is not primary evidence if the original text of that ancient historian survives, because then that is the primary evidence). In other words, never make controversial assertions without leaving a trail of sources and evidence sufficient to confirm those assertions are true.
Rule 10: Avoid reliance on scholarship published prior to 1950 and rely as much as possible on scholarship published after 1970. Work published prior to 1950 need not be ignored, but should not be relied upon if at all possible. Except perhaps for archaeology and philology (e.g., observational reporting and textual criticism), old work should be avoided altogether or employed only when supported by later work (or your own independent verification).11
Rule 11: Always report what the most recent general scholarship says on a subject, or what the current leading consensus is, if either is different from your own view. Do not give the impression that a view contrary to the leading consensus is the consensus or that a maverick view is a normal view.
Rule 12: Admit when you are wrong and publish a correction or revision. Constantly seek expert criticism to refine your work in this very respect.
Adherence to all twelve axioms and all twelve rules should consistently produce reliable history, which will continually improve with constructive debate. The only controversial element I have not defended or explained here is the first rule's insistence on employing a fundamentally Bayesian method in history. So to that I now turn.