How does Facebook recognize you and your friends in photos? After all, each photo is different. Some are group pics, others are selfies. Some are yearbook portraits, others are vacation photos. The lighting varies. Your expression changes. The angle of your face is different. And yet, Facebook can almost always identify you from your likeness. It may seem like magic, but it’s just technology-with a little help from your friends.
Meet DeepFace, Your New Best Friend
Each day more than 350 million photos are uploaded to Facebook. Friend stag friends, matching a face with a name. This gives Facebook an enormous database it can use to perfect its secret algorithms. With this user-generated data-and a new artificial intelligence system called “DeepFace” -Facebook can now recognize you almost as well as your own mother can.
How accurate is DeepFace? After testing the program against industry benchmarks, Facebook claims that its accuracy is 97.25%. Human accuracy is 97.5%-not much difference at all. With the pace of technological advancement, Facebook could soon surpass the ability of humans to recognize even their closest friends.
Amazingly, Facebook can identify you in a photo even if you’re not facing the camera directly. This is a significant technological advance. The DeepFace program can even recognize that you are the same person in photos taken at different ages. Your high school graduation photos and your engagement photos may have been taken years apart, but DeepFace understands that both of those people are you. Whether you’re laughing or crying, or looking a little haggard, it’s still you-and Facebook knows it.
Facebook has grown into much more than an online book of faces, but the social networking giant remains the ideal testing ground for facial recognition technology. Researchers at the Facebook Al Group in Menlo Park, California, have used their growing database of user-tagged Facebook photos to create what they call the “Social Face Classification” (SFC) data set: 4.4 million face photos of more than 4,000 people. Each person in this data set appears in about 1,000 images which allows DeepFace to learn the essential features of each face.
The SFC is what researchers call a “deep neural network” that connects data nodes like the brain connects neurons. It has complex interconnections and layered hierarchies that DeepFace can train itself on-in a process known as “deep learning”-to better identify the person in an image. One of the pioneers of neural network computer science, Yann LeCun, now runs Facebook’s artificial intelligence lab in New York. With arguably the biggest and best data set in the world, and with many of the top researchers and programmers in the field, Facebook continues to push the limits of the technology.
With DeepFace, it wasn’t enough for Facebook to have a massive deep neural network and a clever algorithm. A key problem remained: how to recognize a person whose face was captured not from the front but at an angle? The answer, researchers discovered, was to take that side view and analyze it from the front, using a new technique called “frontalization.”
To reorient an image, DeepFace uses 3D facial models and sophisticated alignment techniques. There are three key steps. The first step is 2D alignment: the facial image is marked with six points to identify the center of the eyes, the tip of the nose, and the location of the mouth. The second step is 3D alignment: the 2D facial image is warped onto a generic 3D facial model. The third step is frontalization, a process that reorients the 3D image so that it faces front, as if looking directly into a camera lens.
It is the combination of these three techniques-a deep neural network, deep learning, and frontalization-that allows DeepFace to recognize you with an accuracy that rivals that of your friends and family.
Welcome to the brave new world of facial recognition.