Genevieve Bell tweeted a link to a NYT article today, “When No One is Just a Face in the Crowd“. It’s about commercial face recognition systems being used in stores for identifying shoplifters or big spenders.
I’m sure facial recognition software is getting better all the time, but, from what I understand, it’s still a bit delicate. Change the lighting or the angle of the camera and recognition accuracy takes a dive. Or you can mess with the set of points on your face that the recognition engine is trying to identify.
Colleagues of mine have been working on what they call Soft Biometrics. Here’s the abstract to a recent book chapter:
In a commercial environment, it is advantageous to know how long it takes customers to move between different regions, how long they spend in each region, and where they are likely to go as they move from one location to another. Presently, these measures can only be determined manually, or through the use of hardware tags (i.e. RFID). Soft biometrics are characteristics that can be used to describe, but not uniquely identify an individual. They include traits such as height, weight, gender, hair, skin and clothing colour. Unlike traditional biometrics, soft biometrics can be acquired by surveillance cameras at range without any user cooperation. While these traits cannot provide robust authentication, they can be used to provide identification at long range, and aid in object tracking and detection in disjoint camera networks. In this chapter we propose using colour, height and luggage soft biometrics to determine operational statistics relating to how people move through a space. A novel average soft biometric is used to locate people who look distinct, and these people are then detected at various locations within a disjoint camera network to gradually obtain operational statistics. (Denman, Fookes, et al. (2012) Identifying customer behaviour and dwell time using soft biometrics in Video Analytics for Business Intelligence [Studies in Computational Intelligence, Volume 409].)
Now, that’s pretty heavy on jargon, but what it means is rather than using facial recognition, it’s possible to identify a person based on their height, clothes and gait, without having to get a good picture of their face. Denman, Fookes et al are able to identify someone as that person walks from camera to camera through a space, still based only on non-facial features. Actually, the system picks each person-shaped-and-moving thing in a video stream and assigns it an identifier and then tries to get as much biometric information about each person-shaped-and-moving thing as it can. In the demo I’ve seen, it’s possible to query a video archive in near-natural language, for “a person of average height wearing dark blue pants and a red short-sleeve shirt” and have it return a series of clips matching that criteria.
This is less accurate than facial recognition for identifying a person who you know. You can’t tell a soft biometric system to “see if Ben was in the mall in the last three hours”. But you could link it to a credit-card data stream and then tell it, “if Ben made a purchase on his credit-card in the mall in the last three hours and we had a camera at that point-of-sale terminal, what time and through which door did he enter the mall?”