Fingerprint evidence: not exactly what CSI showed you

Every scientific type of analysis has an error rate. I've mentioned it before: science is not about certainty, it's about accurately measuring the uncertainties. Unfortunately, when it comes to forensic sciences, this causes a logical problem: scientists like to talk about being 90% sure about something, but in trials there's only two outcomes: innocent or guilty. You can't do 90% guilty and 10% innocent.

It occurred to me that this was an issue when I heard somebody talk about how fingerprint analysis has no error rate. Being a statistician, all sorts of red flags rose in my head. Of course there's an error rate! Any procedure has an error rate because the error rate is a definition: you count the number of successful identifications and divide by the total number of comparisons made. Maybe the fingerprint matching has never failed? Unfortunately, that's not true. 

As some of you know, I was in Los Angeles last month and I was lucky enough to speak to a fingerprint analyst with the LAPD. She explained the procedure to me, which is pretty cool, actually. Blood fingerprints are photographed. Everything else is dusted: black powder is used for clear surfaces, and white powder for dark surfaces. The powder sticks to the oily residue from hands and the rest is brushed away. The analyst then applies tape to the surface, which lifts the dust that has attached to the oily residues. This accurately reproduces the print onto the tape and the tape is then transferred to a card.

The typical procedure is to scan the card into the computer and send it to a number of databases, the largest of which is IAFIS, maintained by the FBI. An automated algorithm comes back with 2-3 possible matches with the prints lifted from the scene, and at that point a person compares the matches from the database with the in order to make a final identification. If the database comes back with no match, but the police has obtained the prints from a possible suspect, then again the comparison is done by hand.

Just like any other procedure that deems itself as "scientific," fingerprint analysis too needs an error rate, and I'm not alone in making this claim: in 2010 UCLA law professor Jennifer Mnookin obtained a federal grant in order to study error rates in fingerprint evidence. The difficulties are intrinsic: you have to fold in the variability from analyst to analyst and from case to case, as some fingerprints are lifted in ideal situations whereas others are only partial, blurred, etc.

To explain the importance of this type of research, in a 2010 paper [1] Mnookin asks the following question to two hypothetical law students:

"Is fingerprint identification one of the most secure forms of evidence we have, or is its scientific validity remarkably untested?"

Sadly, one of her hypothetical students

"would discover that the scientific validity of fingerprint evidence is surprisingly untested. He would ascertain, for example, that there were no generally shared and validated objective standards for declaring a ‘match’ either required by judges within the courtroom as a precondition to admissibility, or self-imposed by the fingerprint community—that in the end, deciding whether or not a match existed was left to the discretion of each examiner based on his judgement and experience. He would find, in sharp contrast to, say, DNA profiling, that there was no fully specified statistical model of fingerprinting, that would permit the likelihood that two prints came from the same person to be expressed in probabilistic terms."

Mnookin concludes:

"Assuming that errors were not randomly distributed across the array of test prints, proficiency tests designed genuinely to test both the method and its limits might provide useful information about the circumstances that tended to lead to errors, and perhaps could lead to practices that could reduce the likelihood of making such mistakes. Such proficiency tests are technically feasible, and whatever they found would teach us a good deal more about the frequency of errors than we know at present."

To get a more rounded perspective on the matter, I contacted my friend Mark Pryor, an assistant district attorney with the County of Travis, in Texas. Mark was very happy to hear from me, and he was even happier to tell me that he had just found a home for his novel The Bookseller, signing a three-book-deal with Seventh Street Books, a new mystery and thriller imprint from Prometheus Books. Way to go, Mark! He answered my questions while floating twelve inches above ground...

EEG: I understand that every time you have an expert witness take the stand they have to state what their expertise is, and their background and how many cases they've investigated, etc. Are they ever asked, over the course of the testimony, what the error rate for their kind of analysis is? Is it at all required for some kind of analyses?

MP: Yes, they always explain their qualifications, experience and training. This is for two reasons: one, to qualify them as an expert, so that they pass what's known as the Daubert test for scientific expert testimony. That's a call for the judge. But it's also so the jury can know them a little, understand who this person is giving the scientific opinion/testimony. I have never heard one asked what about error rates, actually.

EEG: Has a defense lawyer ever questioned fingerprint analyses during one of your cases, and if so, how did you reply to that? If not, how would you reply to that?

MP: I have not taken a fingerprint case to trial yet, so no. I'm sure if I did, defense counsel would thoroughly question the expert, maybe about error rates, certainly about the procedures they use and the practices in place to make sure there were no mistakes. I think what I would emphasize in my re-direct of such an expert is precisely those same things: making it clear neither I nor the fingerprint expert has any interest in making a mistake and blaming the wrong person for a crime. In other words, I want my print expert to come across precisely as he is: a disinterested expert on the science of fingerprint analysis.

EEG: More in general, do you ever "question" the scientific evidence that detectives bring to your desk before deciding whether or not to issue an arrest warrant?

MP: I personally don't have anything to do with arrest warrants, a detective will get that from the judge directly. Sometimes I'll review one if it's a big case, but I generally won't see any scientific evidence - usually because the arrest is based on non-scientific evidence because a lot of the science stuff takes a while to process (I'm thinking of DNA here). Now, for trial I always look very hard at ALL my evidence, scientific included. Again, for two reasons: I don't want to try an innocent person, and also because I don't want to find out any mistakes/weaknesses once I'm in trial, I want to know those in advance!

I confess I'm still a bit troubled by the process. TV shows like CSI make it look like scientific evidence is either black or white, while unfortunately there's a full range of grays in between. And like Mark said, the bottom line is not to try an innocent person. I'm glad the law is represented by scrupulous people like Mark, but I'm also grateful that Professor Mnookin got her grant. I couldn't find any update on her website, but I'll keep an eye on it because I'm eager to find out about their results.

As for Mark, his book The Bookseller will hit the shelves towards the end of 2012, but in the meantime you can follow his blog, DA Confidential, where he discusses being an assistant district attorney, a father, a husband, and a writer.

[1] Mnookin, J. (2007). The validity of latent fingerprint identification: confessions of a fingerprinting moderate Law, Probability and Risk, 7 (2), 127-141 DOI: 10.1093/lpr/mgm022

DNA and the criminal law

One of the things that makes DNA so fascinating is its widespread use not only in science and technology, but also, as you all know, in forensic law. I had the pleasure to discuss DNA evidence with my friend Mark Pryor, an assistant district attorney in Texas and a fellow writer. His novel, THE BOOKSELLER, is currently on submission to publishers and is represented by Ann Collette of the Helen Rees Agency.

I asked Mark to explain what happens after the DNA evidence has been gathered and a suspect has been identified.

 MP: Usually the detective writes an affidavit while consulting the prosecutor. The affidavit is used to lay out the probable cause that the defendant did the crime, then it is presented to the judge.  If the judge thinks there is probable cause, he causes the warrant to be issued.  
 
EEG: When will  you approve a detective's affidavit based on DNA evidence?


MP: Depends.
I know, poor answer.  But here's why: DNA is used to put someone in a certain place.  That's all.  In a rape case it can put the defendant inside the victim, literally, or just in her vicinity.  So usually we need more than just DNA, unless there is NO reason for the defendant to have been at the scene of the crime.  So your DNA being present when your husband is murdered in your house isn't helpful.  But my DNA there would be, although by itself it's still probably not enough for a warrant.  What would happen is that a detective would pay me a visit and find out if I had a good reason for being at your house.  Remember, DNA by itself can't tell you when someone was at a location, just that they were there.

The one time it would be enough by itself would be in a stranger-on-stranger rape case, where the victim got a good look at her attacker but didn't recognize him, didn't know him.  Even in a date-rape type scenario, where consent is usually the issue, a positive DNA hit will lead the detective to interview the accused before seeking a warrant, most likely.

EEG: What about the opposite scenario: when can DNA evidence be challenged?

MP: It's very hard to argue with DNA.  It's science, and good science, so it's not going to work if you just say, "That's wrong."  What you might be able to do is show an error in the testing.  One interesting case I've seen: semen from a rape/murder was tested against that of the suspect.  It came back negative.  Years later, a detective reopened the case and was convinced it was that suspect, so she resubmitted the sample.  Sure enough, they'd screwed up the testing and it was his.
Now that of course presents a problem when this case comes to trial because the defense lawyer will no doubt argue, "They messed up once, maybe they did twice."  Or even suggest we rigged the result.  

EEG: Interesting. As a scientist, I do know lab contamination is a reality, even when people are extremely careful. Ideally one should always repeat the same analysis multiple times, though DNA testing is still too expensive to allow for that. On a good note, in the five years I've been in this field, I've seen the technology in DNA typing improve exponentially, and the costs lower every year. I'm fairly confident that ten years from now the technology will be much more affordable and errors will be negligible.

Do you believe in the CSI effect, in other words, the fact that after the rise in popularity of shows like CSI prosecutors have a harder time convincing a jury?

MP: Oh yes.  I've blogged about this very thing, in fact. People have much higher expectations of prosecutors these days, and it's definitely something I have to address while picking juries, to make sure they understand what role DNA evidence might or might not play.  For example, I had a murder case recently--it'll be featured on 48 Hours on October 1 of this year--where that was an issue while picking the jury.  This was a cold case, from 1985, and I knew people associate cold cases with DNA, mostly because that's how they are usually solved.  But that wasn't our case. We didn't have DNA and during voir dire I wanted the jurors to understand that there are some instances when DNA testing is pointless.  In our case, the victim and defendant had been dating and so it was not disputed that he'd been at her apartment, where the murder took place.  If they'd collected samples, fingerprints etc., his would likely have been there, but if we'd presented that evidence it would have been a big "So what?" because, as I said above, DNA just tells you someone has been at a location.  Which we all knew.  

Even so, after explaining this, one potential juror insisted we would need to present her with DNA evidence before she'd convict.  I put the hypothetical to her: "What if we had the whole thing on video and you could clearly see the defendant committing the crime?"  She replied that she'd still need DNA.  Needless to say, that disqualified her from the jury panel.  And in the case itself, the defense made a big deal about how we had no DNA evidence of the defendant's guilt!  But those are the types of challenges we face.  DNA is a fantastic development, it's a wonderful tool, for law enforcement but we need to educate people as to its limitations, to the reality of what it tells us about a crime.

EEG: I couldn't agree more. CSI is a wonderful show, but unfortunately too many people watch it thinking they are learning science or forensic law. Watch it for the fun, folks!

Thank you, Mark, for taking the time to answer my questions. I look forward to the 48 Hours show in October!


To find out more about Mark and his book, visit his website, DA Confidential, where he blogs about writing and prosecuting crimes.

Photo: metal knots. Canon 40D, focal length 47mm, exposure time 1/125.