Fingerprint evidence: not exactly what CSI showed you

Every scientific type of analysis has an error rate. I've mentioned it before: science is not about certainty, it's about accurately measuring the uncertainties. Unfortunately, when it comes to forensic sciences, this causes a logical problem: scientists like to talk about being 90% sure about something, but in trials there's only two outcomes: innocent or guilty. You can't do 90% guilty and 10% innocent.

It occurred to me that this was an issue when I heard somebody talk about how fingerprint analysis has no error rate. Being a statistician, all sorts of red flags rose in my head. Of course there's an error rate! Any procedure has an error rate because the error rate is a definition: you count the number of successful identifications and divide by the total number of comparisons made. Maybe the fingerprint matching has never failed? Unfortunately, that's not true. 

As some of you know, I was in Los Angeles last month and I was lucky enough to speak to a fingerprint analyst with the LAPD. She explained the procedure to me, which is pretty cool, actually. Blood fingerprints are photographed. Everything else is dusted: black powder is used for clear surfaces, and white powder for dark surfaces. The powder sticks to the oily residue from hands and the rest is brushed away. The analyst then applies tape to the surface, which lifts the dust that has attached to the oily residues. This accurately reproduces the print onto the tape and the tape is then transferred to a card.

The typical procedure is to scan the card into the computer and send it to a number of databases, the largest of which is IAFIS, maintained by the FBI. An automated algorithm comes back with 2-3 possible matches with the prints lifted from the scene, and at that point a person compares the matches from the database with the in order to make a final identification. If the database comes back with no match, but the police has obtained the prints from a possible suspect, then again the comparison is done by hand.

Just like any other procedure that deems itself as "scientific," fingerprint analysis too needs an error rate, and I'm not alone in making this claim: in 2010 UCLA law professor Jennifer Mnookin obtained a federal grant in order to study error rates in fingerprint evidence. The difficulties are intrinsic: you have to fold in the variability from analyst to analyst and from case to case, as some fingerprints are lifted in ideal situations whereas others are only partial, blurred, etc.

To explain the importance of this type of research, in a 2010 paper [1] Mnookin asks the following question to two hypothetical law students:

"Is fingerprint identification one of the most secure forms of evidence we have, or is its scientific validity remarkably untested?"

Sadly, one of her hypothetical students

"would discover that the scientific validity of fingerprint evidence is surprisingly untested. He would ascertain, for example, that there were no generally shared and validated objective standards for declaring a ‘match’ either required by judges within the courtroom as a precondition to admissibility, or self-imposed by the fingerprint community—that in the end, deciding whether or not a match existed was left to the discretion of each examiner based on his judgement and experience. He would find, in sharp contrast to, say, DNA profiling, that there was no fully specified statistical model of fingerprinting, that would permit the likelihood that two prints came from the same person to be expressed in probabilistic terms."

Mnookin concludes:

"Assuming that errors were not randomly distributed across the array of test prints, proficiency tests designed genuinely to test both the method and its limits might provide useful information about the circumstances that tended to lead to errors, and perhaps could lead to practices that could reduce the likelihood of making such mistakes. Such proficiency tests are technically feasible, and whatever they found would teach us a good deal more about the frequency of errors than we know at present."

To get a more rounded perspective on the matter, I contacted my friend Mark Pryor, an assistant district attorney with the County of Travis, in Texas. Mark was very happy to hear from me, and he was even happier to tell me that he had just found a home for his novel The Bookseller, signing a three-book-deal with Seventh Street Books, a new mystery and thriller imprint from Prometheus Books. Way to go, Mark! He answered my questions while floating twelve inches above ground...

EEG: I understand that every time you have an expert witness take the stand they have to state what their expertise is, and their background and how many cases they've investigated, etc. Are they ever asked, over the course of the testimony, what the error rate for their kind of analysis is? Is it at all required for some kind of analyses?

MP: Yes, they always explain their qualifications, experience and training. This is for two reasons: one, to qualify them as an expert, so that they pass what's known as the Daubert test for scientific expert testimony. That's a call for the judge. But it's also so the jury can know them a little, understand who this person is giving the scientific opinion/testimony. I have never heard one asked what about error rates, actually.

EEG: Has a defense lawyer ever questioned fingerprint analyses during one of your cases, and if so, how did you reply to that? If not, how would you reply to that?

MP: I have not taken a fingerprint case to trial yet, so no. I'm sure if I did, defense counsel would thoroughly question the expert, maybe about error rates, certainly about the procedures they use and the practices in place to make sure there were no mistakes. I think what I would emphasize in my re-direct of such an expert is precisely those same things: making it clear neither I nor the fingerprint expert has any interest in making a mistake and blaming the wrong person for a crime. In other words, I want my print expert to come across precisely as he is: a disinterested expert on the science of fingerprint analysis.

EEG: More in general, do you ever "question" the scientific evidence that detectives bring to your desk before deciding whether or not to issue an arrest warrant?

MP: I personally don't have anything to do with arrest warrants, a detective will get that from the judge directly. Sometimes I'll review one if it's a big case, but I generally won't see any scientific evidence - usually because the arrest is based on non-scientific evidence because a lot of the science stuff takes a while to process (I'm thinking of DNA here). Now, for trial I always look very hard at ALL my evidence, scientific included. Again, for two reasons: I don't want to try an innocent person, and also because I don't want to find out any mistakes/weaknesses once I'm in trial, I want to know those in advance!

I confess I'm still a bit troubled by the process. TV shows like CSI make it look like scientific evidence is either black or white, while unfortunately there's a full range of grays in between. And like Mark said, the bottom line is not to try an innocent person. I'm glad the law is represented by scrupulous people like Mark, but I'm also grateful that Professor Mnookin got her grant. I couldn't find any update on her website, but I'll keep an eye on it because I'm eager to find out about their results.

As for Mark, his book The Bookseller will hit the shelves towards the end of 2012, but in the meantime you can follow his blog, DA Confidential, where he discusses being an assistant district attorney, a father, a husband, and a writer.

[1] Mnookin, J. (2007). The validity of latent fingerprint identification: confessions of a fingerprinting moderate Law, Probability and Risk, 7 (2), 127-141 DOI: 10.1093/lpr/mgm022

Deep sequencing and forensic science: how to improve DNA fingerprinting

(In case you missed it, this originally appeared last Thursday as a guest blog on the Writer's Forensics Blog.)

There are roughly three billion pairs of nucleotides in human DNA, and the vast majority is identical across individuals. When we talk about “genetic fingerprinting,” we really mean, “looking for a needle in a hay stack.” Luckily, for the most part, we all differ at the same loci. Over the years, the techniques used for DNA typing have improved greatly, diminishing both costs and the likelihood of errors. These days, most forensic laboratories use commercial kits to type specific regions of the DNA that are known to vary across the population. Here in the US, the standard for DNA fingerprinting is to type 13 loci called short tandem repeats (STRs), regions that are 4 or 5 nucleotides long. The likelihood of two individuals having all 13 loci identical is so low that we can deem it virtually impossible (with the exception of identical twins, of course).

Using PCR-based technology (which creates many clone sequences out of a small sample), the commercial kits can rapidly determine the 13 STR alleles even from old, partly degraded DNA. These alleles are then run through CODIS, the DNA database maintained by the FBI, and if the genetic profile is already in the system, a match can be determined.

However, there’s a catch, and it’s called microvariant. From time to time, an individual will have a mutation that is so uncommon it’s never been observed before. The commercial kits are made to recognize specific variants that have already been documented, so when the DNA with the rare mutation is analyzed, the kit will not be able to recognize it. This can potentially lead to mislabeling.

How can we buld a reliable library of STR alleles that faithfully represents the whole population? Until a few years ago, the two sequencing methods available — the chain-termination method (Sanger et al., 1975), and pyrosequencing (Ronaghi et al., 1996) — yielded tens of sequences at the time. The breakthrough came in 2005, when 454 Life Sciences, a biotechnology company based in Branford, CT, invented a new fiber-optic chip that allowed the typing of tens of thousands of DNA sequences [1]. The new method is called 454 sequencing or ultra-deep sequencing.

For those of us working in HIV research, this was a breakthrough. Since we had already shown that only a handful of viruses are transferred during a sexual transmission, deep sequencing allowed us to type the genome of those transmitted viruses, shedding new light on vaccine design.

But what about forensic analyses?

Researchers from Denmark used deep sequencing to analyze five STR loci and found rare base mutations and repeat variations that would have not been found using conventional methods [2]. As mentioned before, in order to reduce typing errors, it’s important to find these variants and incorporate them in the commercially available typing kits. Here in the US, a similar analysis is ongoing at the Forensic Science Program of the Western Carolina University. The goal of the study, led by Professor Mark Wilson, is to understand how deep sequencing can uncover minor variants and hence minimize the rate of inconclusive results from genetic fingerprinting analyses.

In conclusion, just like its name implies, deep sequencing can give us a new depth to DNA sequencing, unveiling new, previously unknown alleles in the population.

[1] Margulies M, Egholm M, Altman WE, Attiya S, Bader JS, et al. (2005). Genome sequencing in microfabricated high-density picolitre reactors. Nature, 437 (7057), 376-80 PMID: 16056220

[2] Fordyce SL, Ávila-Arcos MC, Rockenbauer E, Børsting C, Frank-Hansen R, Petersen FT, Willerslev E, Hansen AJ, Morling N, & Gilbert MT (2011). High-throughput sequencing of core STR loci for forensic genetic investigations using the Roche Genome Sequencer FLX platform. BioTechniques, 51 (2), 127-33 PMID: 21806557

Photo: red glass in iron cast. Focal length 85mm, shutter speed 1/50, F-stop 5.6, ISO 100. I know, it's a weird picture... It vaguely reminded me of a jumbled fiber-optic chip... very vaguely, though...

Thrilling forensics: an interview with award winning author D.P. Lyle

Cardiologist, story consultant, lecturer, award winning author. Whether you are a mystery writer, a forensics enthusiast, or a fan of medical/forensic thrillers, you can't possibly not know him: D.P. Lyle, MD, is the Macavity Award winning and Edgar® Award nominated author of FORENSICS FOR DUMMIES, FORENSICS & FICTION, HOWDUNIT: FORENSICS, the Dub Walker Thrillers STRESS FRACTURE and HOT LIGHTS, COLD STEEL, and the media tie-in novel ROYAL PAINS: FIRST, DO NO HARM based on the hit TV series. His essay on Jules Verne’s THE MYSTERIOUS ISLAND appears in THRILLERS: 100 MUST READS.

He has worked with many novelists and with the writers of popular television shows such as Law & Order, CSI: Miami, Diagnosis Murder, Monk, Judging Amy, Peacemakers, Cold Case, House, Medium, Women’s Murder Club, 1-800-Missing, The Glades, and Pretty Little Liars.

I can't tell you what an honor it is to have Dr. Lyle on my blog today.

EEG: You are an MD and an award winning writer. Did the two -- medicine and writing -- always go hand in hand in your life, or did the MD turn into writer at some point?

DPL: The medicine definitely came first. I knew I was going to medical school before age 10 and in fact knew that I would go into cardiology so that path was pretty well set at a very young age. I never considered anything else.

I grew up in the South where storytelling is a tradition so was exposed to great storytellers my entire life. I always loved to read and had a great respect for books, whether class books or novels, and I always wanted to write but wasn’t sure I could. I often said that when I retired I would write some of the stories that I had in my head and see where it went. But about 15 or so years ago I realized there was no sense in waiting to retire since that was probably going to be a long time. I enjoy what I’m doing. So I asked myself, “If not now, when?” I took a few night classes at the University of California, Irvine and at an organization called The Learning Tree and then joined a couple of writing groups and began writing.

EEG: Your thrillers are fast-paced, gripping, and rich in forensic and medical details. (Right up my alley!) How are your stories born? Do you take inspiration from your practice, from the news, or is it mostly all the research you do in forensics?

DPL: I guess one of the most common questions that writers get asked is where you get your ideas from? The answer is simply everywhere. I’ll see something on the news, or read something in a book, or come across some interesting medical or forensic fact, or something in an idle conversation with a friend will spark an idea. Of course ideas are a dime a dozen. Most don’t have the legs to become a novel, or even a short story.

The initial step is to turn that idea into a What If? What if this or that happened? From there I begin to develop a story idea and invariably several scenes will come to mind. At this point I’ll start making what I call a Plot Point outline. This is simply a list of things that could happen as the story unfolds. After working on this for a while I have a good idea whether this idea is simply another idea or a concept that can grow into 100,000 word story.

Since most of my books are medical and forensic thrillers I definitely call on my medical experience and knowledge as well as the things I’ve learned about forensics over the years. And research for me is constant. I rummage around the web daily, looking for interesting facts and stories and all the other cool things that are out there.

EEG: You've consulted for popular TV shows like Law and Order and Monk, and for many famous writers. Some of the questions have made it into your popular non-fiction books. Mark Twain used to say: “It's no wonder that truth is stranger than fiction. Fiction has to make sense.” Have you ever come across some medical case that was more absurd than anything you've seen through your consultations?

DPL: Truth is definitely stranger than fiction. I’ve seen so many things since I started medical school that it would take hours to go through even a few of them. People do strange things and strange things happen to people.

One example would be when I was an intern doing my emergency room rotation. It was a late Saturday afternoon and the emergency room was actually quiet for a change when we heard gunshots just outside the door on the receiving ramp where the ambulances drive up. It turned out that a father and son had had some differences they wanted to settle and they decided the best way to do it was with a little small arms fire. But they wanted to be near help after it was over so they decided to drive down to the emergency room of the University Hospital and have it out on the receiving ramp. The father was a better shot. The son took three in the chest and one in the abdomen while the father had two in the chest. They both survived and were both treated in the same major trauma room, their stretchers only 15 feet apart. They weren’t angry anymore as they had settled their differences. Go figure.

One afternoon a tall thin elderly black male walked into the emergency room asking for help. The odd thing was that it was the middle of July and he was wearing a long raincoat. When asked what was wrong, he opened the coat to reveal an ice pick buried to the hilt in his chest. The ice pick wavered with each heartbeat. It seems that he and his wife had had an argument, he had hit her, and she had stabbed him with an ice pick. That was about an hour and a half earlier. He had to change buses twice in order to reach the medical center and the entire time he wore the raincoat to cover the ice pick. After evaluating him with x-rays we found that the ice pick was embedded in his aorta so he was taken to the operating room where the ice pick was removed and the hole in his aorta sutured. He did fine.

All physicians have such stories and many of them revolve around happenings in the emergency room. It’s a wild place. Almost anything can happen at any minute.

EEG: Those are amazing stories. Thanks so much, Dr. Lyle, for sharing them with us, and thank you for answering my questions!

Dr. Lyle runs a forensics blog which is an essential resource for any writer, as well as forensic enthusiasts. To find out more about his lectures, consultations, and thrilling books, visit him at www.dplylemd.com.