HSNW conversation with Thirimachos Bourlai, WVUThe complexities of the human face: analyzing facial recognition technologies in unconstrained environments

Chris Archer: Explore the main challenges with regard to facial recognition technologies: How are these technologies being used by the military and in government at present?

Thirimachos Bourlai:I will first try to provide an understanding on what FR is, and then I will state what, in my opinion are the main challenges with regard to facial recognition technologies.

One of the physiological traits utilized by biometric systems to establish human identity is face. Face-based recognition systems (FRS) are gaining interest because the human face has several advantages over other biometric traits: it is non-intrusive, understandable, and can be captured in a covert manner at variable standoff distances. A typical FRS consists of the enrollment and the authentication phase. During the enrollment phase, images of the user’s face are taken and used to create face templates which are stored in a database, typically called the gallery dataset. During the authentication phase, newly recorded images of a user’s face, called probes, are used for recognition. A decision on the person’s identity (the output of the FRS) is taken on the basis of the comparison between the gallery templates and the new (probe) images.

In simple words, the similarity between two face images of the same person which is delivered by a FRS is expected to be higher than the similarity between face images of different individuals. The questions is whether this is always true when operating in real-world conditions. Unfortunately, in practice, things are not that straightforward. There are various challenges with regard to facial recognition technologies and this is why there are so many groups worldwide which work in this area.

Let me give you a real-world example where things can be really tricky. Consider a face image captured by a surveillance camera at night outside a military facility. If the person is flagged as “suspicious” or with a potential to perform a “suspicious” activity, it may be necessary to match the face image against millions of mug shots across the country. And this is just an estimate! In 2009, for example, thirty million surveillance cameras were estimated to be deployed in the U.S., which were shooting, in a week’s time, about four billion hours of footage. Thus, the problem is to be able to develop a FRS that can automatically perform such a task efficiently and in a timely manner.