Facebook Face Extractor

One of the things I and my friend Enrique are interested in is applying face recognition to the social network of Facebook to automatically tag people in pictures. However, in order to do this you need data, which we obtain from our Facebook Image/Tag Downloader. This downloads pictures and location tags of people in batch. However, to do face recognition, you need to be able to extract the face from the image using face detection matched to the tags locating the people in the image. We have developed a little C++ program to do this in batch for the Facebook data.

Download: C++ Source Code

We do this using a OpenCV C++ program that uses the Viola-Jones detector to identify all the faces and eyes in the image. We match two eyes to each detected face not only to reduce the false positive rate, but to give us a metric for very basic pose normalization. detected faces and eyes are ranked based on some face geometry to identify reasonably well defined frontal faces. Once a face is found, matched with two eyes, the locations of people tagged in the image are scanned for potential identities. In order for a match to occur between a tag and a face, only that tag must lie inside the face area. This means that sloppy tags and tags on the torso instead of the face will not be matched, yet this somewhat cumbersome constraint avoids a number of troubling scenarios such as group photos where it can be very easy to mis-identify the face and tag.

Once the face is identified, it is cropped from the image and rotated such that the detected eyes are always horizontal with a constant separation. This of course is not entirely true since rotations into the plane will stretch the width of the area around the head, but it is a good approximation. The faces are then saved to a file that identifies the identity of the person and the face number: <<User ID>>-<<Instance>>.jpg. This allows for easy parsing when loading the face data into a face recognition algorithm setup.

Hack Warning

The provided source code is very much hacked together and not commented too well. My guess is most people will want to extract faces with their own favorite algorithm. Thus we provide this as mostly a starter or benchmark. That being said, if you have any questions don't hesitate to contact me and I'll do my best to get back with you.