With the increasing pervasiveness of digital cameras, the Internet, and social networking, there is a growing need to catalog and analyze large collections of photos. Because photo interest is largely determined by who appears in the picture, labeling photos with identities is particularly important. In fact, popular social networks such as Facebook allow users to place tags on photos to label people, encouraging collaboratively organized photo albums amongst friends. For instance, suppose Bob attends a graduation, takes many photos, and uploads them to a social network on which he has a thousand friends. Bob must then manually tag friends to associate them with each photo, which is tedious and time consuming. Now imagine millions of social network users tagging their photos by hand: such web-scale labeling problems present a real challenge and fascinating opportunity for automation by face recognition.

Face recognition is a well-researched field, but one key area not addressed by many conventional approaches is that realistic face identification operates under an “open-universe” assumption in which some faces should be identified, but not others (referred to as distractors). In Bob’s graduation case, only friends should be tagged while other faces should be ignored. However, from the perspective of a local newspaper publication, the presence of a public figure is more noteworthy.

Abstract

We have submitted a paper to Transactions on Pattern Analysis and Machine Intelligence describing the challenges posed by face recognition in web-scale datasets, our algorithmic solution, and a detailed comparison to state-of-the-art algorithms. Our abridged abstract is available below. Pending a decision, we will add more content to this page such as the feature descriptors to our Facebook datataset.

With millions of users and billions of photos, web-scale face recognition is a challenging task that demands speed, accuracy, and scalability. Current approaches test on small, controlled datasets and do not scale well to Internet-sized scenarios such as tagging friends or finding celebrities. Focusing on web-scale face identification, we introduce a new 800,000 face dataset gathered from the Facebook social network that models real-world situations where specific faces must be recognized and unknown identities rejected. We propose a novel sparse algorithm LASRC that harnesses the speed of least-squares and the robustness of SRC. Our efficient LASRC algorithm achieves comparable performance to SRC with a 100-250 times speedup and surpasses SVMs at high precision with 100 times faster training. Extensive tests demonstrate our proposed approach outperforms current state-of-the-art algorithms on large-scale, uncontrolled datasets.

Results

Our goal was develop a fast, realtime algorithm that achieves the performance of slow algorithms such as Sparse Representation Classification (SRC) or Support Vector Machines (SVMs) – even when used with large, Internet-sized datasets. While the full results will be available when our paper is published, we would like to highlight a precision and recall curve on our 800,000 Facebook dataset:

Non-realtime algorithms are marked with an asterisk (*). Our LASRC approach performs very similarly to non-realtime algorithms such SRC or SVMs but has the advantage of being real-time. In fact, LASRC trains 100X faster than SVMs and classifies 250X faster than SRC. Compared to other real-time methods, LASRC outperforms state-of-the-art least squares, sparse, and max-margin classifiers.