The attack on Pearl Harbor on 7 December has given rise to one of the largest conspiracy theories in history. There are many versions, most alleging American foreknowledge, and scarcely one agrees with another. Allegations of culpable incompetence apart, which deserve attention, the two most important theories allege, first, that the British had foreknowledge of Japanese intentions but chose to conceal what they knew from the United States in order to bring America into the war; second, that President Roosevelt knew independently what the Japanese intended but took no preventative action, since he currently sought a pretext to bring his country into the war on Britain’s side. The two theories, in some of their versions, overlap.
The subject is so large that it has stimulated the production of a library of books. Almost the only matter on which they agree is that American cryptanalysts, like their British counterparts at Bletchley, were freely reading Japanese ciphers before December 1941. Exactly what was read and when, what the interpreters made of the decrypts, and how the decrypts influenced the decisions taken by President Roosevelt, his cabinet officers, the chiefs of staff and the commanders in the zone of operations form the substance of the great Pearl Harbor mystery story.
It does not connect with the story of Midway, which began to unroll only six months later. There is, however, this caveat: the cryptanalytic organisation which may have failed before Pearl Harbor was also the organisation that helped to deliver the victory of Midway. It worked in this way.
American cryptanalysis differed from its British equivalent, in organisation, recruitment and ethos. Bletchley was a joint-service civil-military body, in which little distinction of rank was observed, built up on a word-of-mouth basis in the period immediately before the outbreak of war in 1939 and recruited largely among young Oxford and Cambridge dons. Proven mathematical ability was the principal qualification. The atmosphere at Bletchley was creatively amateur, informal and high-spirited; women formed a high proportion of the staff, some in senior positions, and romance flourished. There were many Bletchley marriages.
The American cryptanalytic organisation, by contrast, was sharply divided into naval and military branches, which co-operated uneasily, was highly bureaucratic and almost completely male-dominated. Most of the cryptanalytic personnel were uniformed servicemen and the principal qualification for selection was language skill, particularly in Japanese. Unlike Bletchley, which had a high opinion of itself and cultivated a genial university common room atmosphere, the American intelligence branches were regarded by the rest of the army and navy as backwaters, staffed by officers unsuitable for operational appointments, an opinion of which their members were aware. It is remarkable, in the circumstances, how well they maintained their professional morale. A key indication of the difference between the British and American systems is that, while Bletchley has entered into British national legend and found a popular place in fiction and film, its American equivalents enjoy no such acclaim. Quite wrongly, for what the Americans achieved was equally remarkable, indeed perhaps more so, as the story of Midway indicates.
The origins of the American cryptanalytic service belong, as do those of the British, in the First World War. Major Joseph Mauborgne, head of the army’s cipher research section in 1918, was a cryptographer far ahead of his time: he perceived the concept of the random key—one not retrievable by frequency analysis or, indeed, any mathematical or linguistic logic—and devised the one-time pad, still the only intrinsically unbreakable cipher. He would eventually become a general and the U.S. Army Chief Signal Officer.5Of the same vintage but of even great importance to the American cryptanalytic effort was a civilian, William Friedman (who coined the term “cryptanalysis”). The son of a Russian Jewish immigrant, who entered the United States at the age of one, Friedman resembled in character both Dilly Knox and Alan Turing. As eccentric as either, he displayed a mathematical ability almost equivalent to Turing’s but unfortunately also Knox’s psychological fragility. He had suicidal tendencies and just before the outbreak of the Pacific War suffered a nervous collapse brought on by overwork.6
Yet Friedman was largely responsible for the most important of America’s cryptanalytic successes, the breaking of Purple. In October 1940, the army and navy agreed on a division of labour, not in any spirit of fraternal co-operation but because each lacked the numbers to do much more than concentrate on a single task; in 1938 Friedman had a staff of only eight in the Secret Intelligence Service (SIS), in December 1940 the navy in Washington only thirty-six in its equivalent OP-20-G. There were other personnel at outstations in both the continental United States and the Pacific but most were intercept operators and technicians.7 The arrangement was that the army would work on foreign diplomatic intercepts on even days of the month, the navy on odd. The navy was meanwhile, naturally, working on Japanese naval intercepts; the army was not particularly interested in foreign army intercepts, since they were too faint to yield text.
The Americans, few though they were in number, had had considerable success in breaking into both Japanese naval and diplomatic traffic in the 1930s, assisted by a succession of night-time burglaries of the Japanese consulate in New York. By 1933 the naval cryptanalysts had solved the main Japanese naval Blue Code, a book code with a cipher additive. When it was replaced in June 1939 by JN-25 (Japanese Naval Code 25, as the Americans denoted it), another book code with a more complex additive, the Americans took time to recover from the setback, but by December 1940, with the help of recently acquired IBM card-sorting machines, they had reconstructed the system of additives, the first thousand groups of the code and two of the keys used to work the system.
They anticipated cracking the system completely in 1941. By then, however, all spare cryptanalytic manpower in Washington had been diverted to a new task: decrypting the Japanese diplomatic traffic enciphered on a new ciphering machine, known to the Americans as Purple. The Purple machine (Type 97 to the Japanese) was designed to achieve the same effect as Enigma—the automatic production of an almost infinitely variable cipher—but differed from it in construction. It was less mechanical, having no rotors, but instead a set of telephonic switches, connected to two typewriters. The first was used to input the text, the second to print out the encipherment for transmission. In between, the switches moved the incoming electrical current to achieve alphabetic substitutions. Because Japanese is a syllabic, not alphabetic, language, however, all texts had first to be written in an alphabetic equivalent; and for an inexplicable reason, equivalent to the Germans’ double-encipherment of the operator’s chosen indicator at the beginning of a transmission, the Purple machine’s switches enciphered vowels and consonants separately; the number of vowel substitutions was considerably smaller than that of consonants, and once that was recognised, a way into Purple was found.8
The breaking of Purple—its product was known as Magic, the equivalent of British Ultra—would eventually yield huge intelligence advantages to the Americans, principally through the decipherment of the messages sent by the Japanese ambassador in Berlin, Baron Oshima, to Tokyo throughout the war, which revealed intimate details of Hitler’s capabilities and intentions. At the critical moment before Japan’s surprise attack on America in December 1941, however, Purple revealed little, while JN-25, the relevant Japanese naval code, was not useful for two reasons: the first was that the Imperial Navy attempted to observe, as far as was possible, radio silence in the period preparatory to the descent on Pearl Harbor; the second was that OP-20-G lacked the staff necessary to deal with the volume of intercepted traffic. The U.S. Navy’s Historical Center has now compiled a list of significant messages intercepted—but not decrypted—in the weeks before Pearl Harbor that bear on the issue of foreknowledge. Some, if read in real time, must have alerted a wary admiral to the danger threatening his fleet; in practice, the messages were bundled up and not decrypted and translated until September 1945, a month after the Japanese war had ended.9
Pearl Harbor devastated not only the U.S. Navy’s Pacific Fleet but also OP-20-G’s Pacific outstations. While its Hawaiian station (HYPO) continued to operate, its outpost in the Philippines (CAST) first withdrew into a tunnel in Corregidor, then was evacuated to Australia. American naval intelligence in the Pacific was thus reduced to HYPO, the joint station in Australia and a branch in the British Combined Bureau in Ceylon. The British had had some inter-war success in attacking Japanese naval codes but, in the climate then current, with both the American services and the public adamant about revenge against the Japanese, the prime responsibility for breaking back into Japanese naval traffic lay with OP-20-G. The Commander-in-Chief Pacific, Admiral Chester Nimitz, was on tenterhooks against a renewed Japanese naval offensive; he equally hoped to profit, perhaps from a Japanese mistake but preferably an intelligence coup, and inflict a defeat on the enemy.