[13.0] The Cryptological Fringe

v2.4.0 / chapter 13 of 13 / 01 sep 16 / greg goebel

* Cryptology has always had a "fringe" component, mostly focused around supposed codes buried in major works of literature. This chapter discusses the cryptological fringe, then concludes with some personal comments and a list of sources.



* The Friedmans, as noted, spent considerable effort on the search for hidden ciphers in the works of Shakespeare. They were not alone in such efforts; the topic of hunting for hidden ciphers in great works of literature, particularly the Bible, is a subfield of cryptology in itself. For want of a better term it might be called "pseudocryptology". Some sources call it "enigmatology" but that term is more generally applied to the study of puzzles.

The field goes back to at least 1884, when an American named Ignatius Donnelly published THE GREAT CRYPTOGRAM, making a case that Francis Bacon actually wrote the plays of Shakespeare. Donnelly was an interesting sort. He served three terms in the US Congress, and after being voted out he turned to writing popular works of pseudoscience that promoted such ideas as the historical existence of Atlantis and linking Biblical disasters to a cometary impact. His ideas resurface in different forms in fringe literature on a periodic basis.

The usual approach of pseudocryptologists is to scan through the text of interest and take out letters at intervals, for example every 13th letter, every 100th letter, or every 1,617th letter. They then arrange the letters in a block and see if patterns pop out. The patterns are typically identified as scrambled words buried in a stream of gibberish. It is not too surprising that one can actually extract seemingly sensible remarks using this scheme. It's about on a logical level with, say, playing the music of the Beatles backwards and listening for secret messages left by John Lennon. (Actually that might not be such a good comparison, since Lennon was potentially crazy enough to do such a thing.)

A modern advocate of such hidden codes, Michael Drosnin, published a series of BIBLE CODE books that listed predictions of a wide number of events hidden in the Bible. The books made the best-seller lists, despite the fact that most of the predictions identified by Drosnin in his books were, conveniently, of events that had already happened. His predictions of actual future events, such as the end of the world in the year 2000, were somewhat inaccurate.

Drosnin challenged critics to find similar predictions about, say, the assassination of a prime minister in MOBY DICK, and so an Australian mathematician named Brendan McKay went out and did precisely that, identifying "predictions" of nine such assassinations. Similar exercises were performed on WAR AND PEACE and other classics. However, this was only the latest iteration of a game that's been going on for a long time, since the Friedmans performed similar refutations against the Baconians.

Pseudocryptology also shows up in pop fiction, where it is harder to object to since nobody claims it should be taken seriously. A popular modern example was Dan Brown's 2003 novel THE DA VINCI CODE and the Tom Hanks movie made from it, which involved a search for secret documents written by the Italian Renaissance scholar Leonardo da Vinci that disastrously compromised Christianity.

The book invented a cryptological device called a "cryptex", supposedly invented by Leonardo but really dreamt up by Brown. It looked a bit like a Bazeries cylinder, consisting of a cylinder with labeled rings and a few frills, but order of the rings was fixed and the device was simply a locked container with a document inside. Dialing in the proper password opened the cryptex; dialing in the wrong password broke open a vial of acid inside the device, destroying the document inside. The cryptex was amusing but not really much more impressive than a bicycle lock. Readers pointed out that it would been easy to disable the self-destruct system: just freeze the device so the acid was frozen solid, allowing combinations to be tried out at will.

The Navaho code-talkers have been something of a fad in pop fiction as well, and one TV show even had agents trying to decode old documents written in Navaho code talk. Although there are some websites that have text of Navaho code talk, it's just as an educational device; Navaho code talk was never intended to be used as a written code, and it would have been fairly easy to crack if it had, since it would have then been little different from any other code. TV and movies also occasionally have scenes where the heroes crack government or military encryption with a laptop PC, providing a little amusement for viewers who have some familiarity with modern cryptosystems. One such tale had a sorceress pulling that trick, but she admitted she had to use magic to help.



* There are other interesting topics along the cryptological fringe. In 1912, an American rare-book dealer named Wilfrid Voynich was prowling through the library of a Jesuit college near Rome, Italy, when he chanced upon a strange centuries-old document. It was a book with large numbers of drawings of strange plants and animals, not to mention naked women bathing. This would have been something of an odd find in itself, but what made it really interesting was that it was written in an entirely unfamiliar language.

There was a 17th-century letter tucked into the document that explained it had been purchased by the Holy Roman Emperor Rudolph II in 1586, for the substantial sum of 600 gold ducats. Several scholars tried to decipher the strange script during the next century, and then -- sort of like the Ark of the Covenant in the movie INDIANA JONES & THE LOST ARK -- it disappeared into the archives for centuries. Voynich publicized the text and cryptologists tried to crack it, but nobody has ever had any success. Even cryptographers from the US Navy OP-20-G, at something of loose ends at the end of World War II, gave it a shot and got nowhere.

Several people did claim they decrypted it, but the decryptions tended to be nonsensical, and to the extent that they weren't, the would-be codebreakers claimed that different codes were used in different places in the manuscript. Their claims were not taken seriously: obviously, they were simply cooking their methods arbitrarily to get results they liked.

There is a strong suspicion that the "Voynich manuscript", as it is now called, was nothing but a scam, possibly concocted by the English conman Edward Kelly, and the text is just gibberish. However, analyses show that the text has a high degree of structure, not at all consistent with simple random text. On the other hand, it is too regular to be consistent with real languages.

Although it may be impossible to prove that the Voynich manuscript is gibberish, modern research has shown that it is indeed possible to generate random "Voynichese" text that has much the same appearance as that in the document. All it requires is a small set of ordered tables of syllables and a somewhat larger set of grilles with three holes in it them ("Cardan grilles"). Different pages can be assembled by using a different table with a different grille.

Kelley, who was known to be hanging around the court of Rudolph II at the time of the sale of the document, was also known to be familiar with Cardan grilles. It probably wouldn't have been too difficult to write real text in a code, but it would have been easier and faster to simply write gibberish, and it would have also been a safer scam: if nobody could make sense of the text, nobody could extract any clues that would reveal the fraud.



* Anyone who reads this document might conclude that I'm a codes and ciphers geek, but that's hardly the case. I'd never had much interest in the subject until 1999, when the US Public Broadcasting Service's NOVA series broadcast a documentary titled DECODING NAZI SECRETS on the British Bletchley Park operation, and the US History Channel's SWORN TO SECRECY series broadcast another show on codebreaking titled BREAKING THE JAPANESE CODE.

I started taking some notes from tapes of the shows and quickly realized that they raised many more questions than they answered. I started poking around for more information, and by early 2000 concluded that there was nothing I could do but write up a full document on the subject. After considerable effort, I released an initial version at the beginning of 2001, followed by a major update in early 2002.

Another significant update followed in late 2003. By that time, I found my eyes glazing over and thought I was more or less done with the subject, but then in the spring of 2004 I took a trip that involved dropping into the NSA museum at Fort Meade. I picked up a lot of interesting information -- the staff was friendly to me beyond any possible expectation, handing me an M-209 and letting me take close-up pictures of the thing -- and so I ended up doing another fairly substantial tweak.

So was that the last big tweak? Possibly, I haven't paid too much mind to the topic since then, but I might return to the document if I run across a substantial stash of new information. For the time being, I just check it every now and then to see if there's anything that needs fixing.

* Since I'm not a cryptographer, I originally didn't much care for the convention of using "Alice" and "Bob" in describing cryptographic examples, since I felt a little like I was pretending to be someone I wasn't. However, after cooking up a few examples I gradually became enthusiastic about the scheme.

It is logical, in that "Alice" and "Bob" are easier to keep track of than "A" and "B", and also because having different genders in the examples not only provides a bit of fairness, but makes describing two-person transactions much clearer: "SHE did this, and then HE did that." There also seems to be a semi-convention where Alice is the active member of the transaction and Bob is the passive member, but this is less a reflection on pecking order of the sexes than it is on the alphabetical order of the names.

And of course, using the name "Eve" for an eavesdropper is just perfect. I should have figured that cryptographers would come up with such a neatly logical scheme. I started using "Bob" and "Alice" in physics examples, and indeed I find in my studies that physicists are warming to the scheme as well.

By the way, if an example requires two pairs of transactions, the characters "Yuri" and "Zelda" prove convenient, once again providing a gender balance while sitting at the other end of the alphabet. I wasn't able to find a standard name for a codebreaker, so I improvised, using "Holmes". This is a distinctive name, and most readers should be able to easily visualize Sherlock Holmes as a codebreaker. In fact, it would be very easy to imagine him working in Room 40, smoking his pipe while he talked to Dilly Knox in the bathtub.

* I was particularly interested in the material on the Navaho code-talkers in the US Marines during World War II. I knew a Navaho with the English name of Irwin Johnson when I was in the US Army Signal Corps down in Texas, and I recollect from the times I heard him speak Navaho was that it was so unintelligible that it hardly sounded like a language at all. Even with a difficult foreign language such as Russian or Chinese I can make out that words are being said. In the case of Navaho, it sounded like indistinct and muddied mumbling. It obviously made a very highly effective code.

Incidentally, concerning the story of the Beale ciphers: since I released this document, I've had people writing me every rare now and then to tell me they have actually solved them. The details, they explain, can be understood if I buy the book they've published ...

* As concerns copyrights and permissions for this document, all illustrations and images credited to me are public domain. I reserve all rights to my writings. However, if anyone does want to make use of my writings, just contact me, and we can chat about it. I'm lenient in giving permissions, usually on the basis of being properly credited.

* Sources include:

Yet another useful source of information was the ENCYCLOPEDIA BRITTANNICA. Their writeup is relatively short, terse, and characteristically dry, but it is extremely concise and provides a nice survey of fundamentals.

A number of websites were consulted as well. John Savard's A CRYPTOGRAPHIC COMPENDIUM is particularly interesting, with many details on various codes, ciphers, and code machines. Another very useful document I found on the Web was a lecture written Anthony E. Sale, the director of the Bletchley Park museum, on the cracking of Lorenz, the development of Colossus, and its reconstruction. The PBS NOVA program web pages on BREAKING THE NAZI CODE TV program have a transcript and also additional details on the Enigma machine. Of course, the NSA museum site has some good information.

The PGP site has a nice little INTRODUCTION TO CRYPTOGRAPHY in PDF format. I thought that after developing my rough notes for this document reading another introductory document might not be very productive, but this file added quite a few little fine points that I incorporated into my text. RSA Laboratories also had a very useful CRYPTOGRAPHY FAQ (Frequently Asked Questions) document. I normally don't have much liking for FAQs, since they're generally just haphazard sets of notes thrown together, but this document was very concise and well-organized, if also basically oriented towards serious crypto addicts.

There are many other cryptologic websites on the net. The HANDBOOK OF CRYPTOLOGY is actually available online in PDF format for anybody who wants a professional-level reference. My attitude towards such formal texts is that they are fine for professionals, but for those not actually intending to do any professional work in a particular field, reading one is usually like digging a tunnel to get to the grocery store -- a great deal of wasted effort yielding little of any usefulness.

* Revision history:

   v1.0   / 01 jan 01 
   v1.1   / 01 jul 01 / Minor corrections and updates.
   v2.0.0 / 01 mar 02 / Major update, added CODEBREAKERS material.
   v2.1.0 / 01 sep 03 / Work begun 10 July 2002.
   v2.2.0 / 01 jun 04 / Added photos & NSA materials.
   v2.2.1 / 01 oct 05 / Review & polish.
   v2.2.2 / 01 dec 06 / Minor changes on odds calculations.
   v2.2.3 / 01 jan 07 / Correcting a bug or two.
   v2.2.4 / 01 jan 09 / Review & polish.
   v2.3.0 / 01 dec 10 / Updates on WW2 crypto, SIPRNet.
   v2.3.1 / 01 nov 12 / Smart cards & passwords.
   v2.3.2 / 01 oct 14 / Review & polish.
   v2.4.0 / 01 sep 16 / Tweaks, rewrite of POLITICS OF CIPHERS section.