In-Your-Face: Can Computers Catch You Telling a Lie?

A popular school of thought, dramatized in the recent TV drama Lie to Me, is that a careful study of facial expressions—especially eye movements—tells investigators if a perp is dissembling. Reality is neither as dramatic nor as decisive. Even experienced investigators average only about a 65 percent success rate, according to researchers. Could computers do a better job? Researchers at the University at Buffalo, The State University of New York (U.B.), claim their video-analysis software can analyze eye movement successfully to identify whether or not a subject is fibbing 82.5 percent of the time. The researchers, who first presented their (still unpublished) results at the 2011 IEEE International Conference on Automatic Face and Gesture Recognition a year ago, believe they have laid the foundation for a more extensive study that will include a larger sample and take into account body language in addition to eye movement to determine whether new technologies can help interrogators in their search for the truth. The 40 interviews were conducted by Mark Frank, a U.B. professor of communication and a study co-author, and included a diversity in age, gender and ethnicity. Prior to the interview, each subject was given the opportunity to "steal" a check made out to a political party or cause that the volunteer strongly opposed. Afterward, subjects sat down with a retired law-enforcement interrogator. The interviewer first posed conversational questions unrelated to the possible theft, to establish a baseline of normal eye movement, and then asked whether the interviewee had taken the check. Those subjects who successfully lied received a monetary reward for themselves as well as for a group they supported. Those cau

ght lying received nothing and were told that money would instead go to the party or group they opposed. The software compared each subject's baseline eye movement with those observed during the questions about the check. If the computer detected a large deviation, the researchers noted this change and flagged that person as a potential liar. The work of Paul Ekman, professor emeritus of psychology at the University of California, San Francisco, School of Medicine served as an inspiration for the study. Ekman's specialty is the study of emotions and their relation to facial expressions, and his work formed the basis of the techniques used by fictional psychologist Cal Lightman in Lie to Me. Frank performed postdoctoral research at U.C. San Francisco's psychiatry department under Ekman's guidance. Ekman's focus on brief, involuntary facial expression is not without critics. One former FBI special agent finds a fundamental flaw with the Buffalo study: "One problem with this research is its overreliance on the face as the only place to evince information from the body," says retired FBI counterintelligence special agent Joe Navarro, who spent 25 years with the bureau and was a founding member of its Behavioral Analysis Unit. "I can tell you as an investigator and somebody who's studied this not just superficially but in depth, you have to observe the whole body; it can't just be the face," he says, adding that a failure to take body language into consideration could lead to "an inordinate amount of false positives." In future work, the Buffalo researchers plan to take a more holistic view of behavioral cues. "We know that the eyes give signals that lead to deception, but what about general body movements?" says Ifeoma Nwogu, study co-author and a research assistant professor in U.B.'s Center for Unified Biometrics and Sensors. Faster algorithms would also enable the software to flag behavioral deviations in near real-time, she adds. They also want to expand the sample size; the 40-person study is too small to be statistically significant. An interview's setting or its context can also be important, Navarro says. "I did thousands of interviews with the FBI and most of them aren't in a laboratory, most of them aren't in a clean, sterile environment," such as the one in the Buffalo study, he says. "We did our own study years ago, and about 97 percent of interviews are actually done on the street, in the dark, after hours." These factors affect the interview subject's behavior and have an impact on whether an investigator can even clearly see a suspect's eyes. Ultimately, the best law enforcement can do today is make a judgment based on their observations and experience, taking into account that interrogations can make even honest people a little anxious.