Thoughts on Antiquity

It has taken me three years to get around to reading Stephen C. Carlson’s The Gospel Hoax: Morton Smith’s Invention of Secret Mark (2005). But, perhaps it was worth waiting, since the last three years have accrued several arguments and evidences that shed critical light on Carlson’s case. I already had some idea of what to expect when I began reading, but now that I finally had a bit of free time, I decided it was time to finally see what the book had to say.

A big part of the argument (25-47) concerns handwriting analysis–more specifically, methods of “questioned document examination” (QDE). There is still, to this day, confusion between QDE and graphology, with the latter certainly lying within the domain of debunked practices such as phrenology. The former aspires to the status of a science, but its ability to be able to claim use of the scientific method is fairly recent. The critical article for this part of the story is Michael Risinger, Mark P. Denbeaux, Michael J. Saks, “Exorcism of Ignorance as a Proxy for Rational Knowledge: The Lessons of Handwriting Identification “Expertise” UPenn Law Review 137.3 (1989) 731-792.

Risinger, et al. question the scientific basis of QDE and claim there is scanty empirical evidence to support that it actually works. While the article has been disputed on a number of points, a critical issue on which everyone following them has agreed on is that as of 1989 there was extremely little empirical evidence supporting that QDE works. In fact, there were only 6 studies. The first test was conducted in 1939 by Fred Inbau at Northwestern University Law School (740-743). It showed only marginally better performance among specialists versus control groups. Unfortunately, the study was so methodologically flawed it did not demonstrate anything.

Curiously, despite this, no other study was attempted again until 1975, this time by the Forensic Sciences Foundation (FSF). Unfortunately, the FSF studies assumed such a thing as expertise in handwriting analysis exists and so did not use any control groups, one of several methodological flaws. The tests were sent to numerous labs across the US and the actual results were as follows. The 1975 test offered the most promise: 89% were able to correctly match a suspected letter to its author (744). The 1984 test, somewhat more complicated, had 74% correctly catch one of the letters, but 100% failed to identify the author of another of the questioned letters (745). In the 1985 test, where check fraud was the task at hand, 41% gave correct answers and 31% said the evidence was inconclusive. The latter is a respectable response; unfortunately, the rest (28%) gave largely or completely wrong answers (745-6). In the 1986 test they had to identify the author of a note. 45% gave incorrect answers (746). The 1987 test was made easier following complaints from examiners that the previous tests had been too difficult. Here only 3% gave the incorrect answer, but 45% were unable to reach a conclusion (747). As of 1989 This was the entire published empirical basis for QDE.

Two things are evident. The positive success rate was fairly low and the high rate of inconclusive results brings into question their scientific foundation: how reproducible are the results of QDE techniques? The FSF exams were themselves methodologically flawed, but this still does not exonerate QDE of the fact that a so-called “science” developed without any official hypothesis testing. The answer is, the propositions of QDE methodology were not scientific. In his scathing rebuttal to Risinger, et al., Andre A. Moenssens, “Handwriting Identification Evidence in the Post-Daubert World” UMKC Law Review 251 (1997) 51, says of their claim “that because the reliability of handwriting identification evidence could not be mathematically or scientifically established, a confidence in the ability of the experts engaging in handwriting analysis had not been verified statistically, and courts ought not to admit it, seemed a clear non sequitur.”

Non sequitur? Hardly. But, where is this unpublished verification? Moenssens explains in footnote 223, “before the fairly recent (1995) appearance of the specialized publication International Journal of Forensic Document Examiners, much of the research that was pursued by members of the ASQDE was circulated privately only within the perhaps less than 100-person group of highly skilled questioned document examiners.” It is no surprise that they never bothered to publicly demonstrate the empirical basis of their “science”–they didn’t have to, they had the secrets all to themselves and didn’t feel they had to prove the reality of their claims to anyone else. This is what I call the occult phase of QDE.

Three more empirical studies slowly helped transform QDE from an occult art to science. Moshe Kam conducted two tests in 1991-2 and 1994-5 and then again another in 1996 with the express aim of determining whether a specialist in QDE techniques can have a success rate higher than a lay person. The 1996 results were rather positive, with only 6.5% of the specialists making match errors compared to 38.6% in the control group (Moenssens 315). Since then several QDE journals have sprouted up and the occult art phase seems to be moving into the past (slowly); although, in light of the slim empirical basis for QDE and following stricter requirements for expert witnesses resulting from Daubert v. Merrell Dow Pharmaceuticals (1993) (the so-called “Daubert” standard), handwriting evidence has come under increasing pressure and has been thrown out of court for not satisfying scientific standards at least once since then (United States v. Saelee (1997)).

How does this come to bear on Carlson’s work? First of all, as a general observation, it should be evident that QDE as a science has had rather tenuous foundations until recently, and most of the specialists in the field have been in it from before it escaped its occult art phase. Therefore, when Carslon brings up “the forensic field of questioned document examination” (25) one should not feel overly intimated, especially since at this point he quotes Albert S. Osborn (1929) who was one of the fathers of the field back when it was hardly science, completely devoid of an empirical foundation, and barely distinguishable from graphology. Furthermore, the empirical evidence for QDE, as it is, is based on examination by national experts, with non-exports scoring substantially worse. Carlson does state in note 9 on page 25 that he had the aid of a certain Julie C. Edison, so perhaps the blame should ultimately lay with her for not suggesting a more rigorous analysis or not offering any serious warnings over the photographic evidence. But, more importantly, it helps to know the script that you’re analyzing to be able to apply the techniques successfully. Carlson surely does, but how familiar is Edison with 18th century Greek script? Moreover, to attempt the type of close line movement examination possible in much larger blowups simply isn’t possible with the resolution of photos published by Smith. Therefore, the rubber stamp she offers carries fairly less weight than it would were Edison working under proper forensic lab conditions with documents whose script she were familiar with.

More importantly, QDE as it developed into a science has focused on careful and meticulous document comparison, which is what the FSF and Kam studies went out to test, and not the ability to divine forgeries simply based on the pecularities of the writing. Yet, Carlson’s identification of the oft mentioned “forger’s tremor” and pen lifts are an artifact of QDE as occult art and not modern science. He admits that innocent explanations can account for some of the features he lists, but feels it unlikely that it can for all of them together (31). But, based on what? What is the statistical foundation on which he claims this combination of observed features is unlikely to happen naturally? Where is the statistical foundation for any individually observed feature?

In fact, no tables are provided which tabulate the rates at which the relevant features appear in Morton Smith’s extant hand, the photographs of the Mar Saba manscript, or any of the unquestioned Mar Saba documents. As for the handwriting comparisons Carlson offers between Smith’s hand, MS no. 22 and Theodore, what is the statistical basis (in an examination of letter forms) on which his identification of the hands on MS no. 22 and Theodore rest? How do we know the examples he offers are not statistical oddities; in other words, what about sample bias?

Lacking any sort of statistical foundation for his observations is closely connected with certain judgements about letter forms. For example, he gets good mileage out of discussing the tau displayed in Figure 4C. He sees “two large ink blobs” as a result of “hesitation” because “he had just realized he had written the wrong form of the tau” (29). Yet, in the very same line (Figure 3A), the first tau of των, which is also short and which he explains as necessary because of “an interfering terminal abbreviation”, has “blobs” as well. So, it’s not clear that “blobs” in the other tau have anything to do with getting the form wrong. Indeed, Sabas 523 (Figure 2C) also has a short tau in του (at the end of the line). In the absense of hard numbers, he has not made the case that the form is either “wrong” or that the “blobs” are due to getting it wrong.

This brings us to an even more critical problem, the physical lack of the manuscript. Much of the work of QDE as a science involves physical examination of the documents. A particularly pertinent case is found in J. E. De la Pena, “Can an Apparently False Signature Turn Out To Be Genuine” IJFDE 3.1 (1997) 52-58, where three document examiners determined that a document with evidences of trembling in the strokes was a forgery. However, upon second examination with careful magnification they found “furrows” in the paper with faded ink where it had run out; the “tremors” were actually the result of two signatures and they were authentic. This type of examination is simply impossible for our documents.

Now, recalling that Carlson identifies the hand of MS no. 22 with Theodore, this leads to a curious digression from handwriting analysis into some creative musings about Smith’s clue-dropping inside the forged text: the infamous Madiotes as “bald swindler.” In an article published just this year, Allan J. Pantuck and Scott G. Brown, “Morton Smith as M. Madiotes: Stephen Carlson’s Attribution of Secret Mark to a Bald Swindler” JSHJ 6 (2008) 106-125, present new photographic evidence from negatives acquired from Smith’s Jewish Theological Seminary archives. Fresh examination of MS no. 22 reveals that in fact Madiotes is not the reading. More than this, Smith’s offprint corrections to MS no. 22 show him having corrected “Madiotes” to “Madeotas.” A marginal note by him reveals that the published “Madiotes” was a “retranslation to Gk. of my Eng. Translation!” (114). Apparently, Constantine Michaelides of Nea Sion had translated it back into Greek, introducing the error. Carlson’s entire Madiotes argument is based on a modern scribal error.

But, the difficulties don’t end here. A closer examination reveals that Smith’s original reading of “Madeotas” is probably incorrect and should be “Modestos,” who is a known figure and mentioned in other Mar Saba documents (nos. 22, 43 and 44) (123). Careful examination of the MS no. 22 handwriting also clearly reveals that its hand is not the same hand as that of Theodore (118-121). With lack of any statistical basis for the arguments and dependence on originally cropped photographs, it should not be surprising that Carlson’s QDE arguments were fundamentally flawed and could not have demonstrated what they claimed they could.

But, if the two hands are not the same, then what of the “forger’s tremor”? He states “In addition, the first hand also shows the blunt ends and the “forger’s tremor” indicative of the person who penned Theodore.” Let’s quickly remind ourselves of what type of tremor we’re speaking. Figure 1 in the book has a modern example; notice (if you have the book) how much larger the magnification is, and how small the line deviations are. Dan C. Purdy, “Identification of Handwriting” in Scientific Examination of Questioned Documents (2006) eds. Kelly and Lindblom, states that “broad curves and intricate writing movements should be inspected microscopically (emphasis mine)” (57) for tremors, and that “one of the most challenging problems confronting the FDE is determining the cause of tremor in a questioned document” (58). The photographs published by Smith simply do not have the required resolution; it is impossible to find that sort of tremor. Indeed, I do not see a tremor. Carlson offers some specific examples, so let us look at a couple of them more carefully.

Two omicrons are cited which appear “so shakily written as to appear square rather than circular” (28). When you examine the omicrons below you should note how pixelated they are. I scanned them at 800dpi and that was overkill for the resolution available in the book. Which ones look square and shakily written?

The first and third are ones referred to by Carlson, while the second and fourth are from Figure 2A which is Sabas 452. Given the heavy pixelation, can you tell the difference? Do any of them look “square”?

Next, let us examine omicron-upsilon combinations. Examine the three combinations below. Do you notice anything in common?

The first two are combinations cited by Carlson, while the third is from Figure 3D, also a Theodore example. However, They all share a hump to the left in the ligature. This suggests that these are not examples of “tremor,” which should be random, but of a specific handwriting feature when writing this particular ligature. So, is this merely coincidence?

The other supposed examples of “tremor” suffer similarly. It would help if very high resolution photographs were published from the negatives, but even then such identifications without microscope are notoriously subjective, and the fact that writing instruments of the age often did not produce perfectly uniform lines greatly compounds the problem. Since Carlson complains about the mu in στρωματεως (28), to illustrate this point, examine the following mus carefully. Which one of them has a “tremor”?

Perhaps you confidently said the first one or maybe the fourth one. In fact, all of these examples come from the same document, Sabas 523, unquestionably 18th century. I suspect Edison had never worked with 18th century documents or studied the irregularities of their writing implements and media. Notice how line thickness, movement and flow vary among all of them. The traditional stress of QDE on “line quality” is rendered useless in the face of such circumstances.

My post has focused largely on handwriting issues. There are, of course, several other issues concerning the content of the letter and Secret Mark; however, they at best demonstrate that it was an ancient (or perhaps medieval?) pseudepigraphon. Since I am not married to any particular position over whether Secret Mark was secondary to the canonical Mark or not, or whether the letter was actually written by Clement or not, I do not consider it essential to reply to those issues at this point. Rather, my primary concern is the claim of modern forgery by Smith, and on that point, I am not at all convinced by the handwriting analysis.