What facial recognition actually does

Facial recognition is software that identifies or verifies who you are by analyzing the shape and features of your face. A camera captures an image of your face, the software measures distances between your eyes, nose, cheekbones, and jaw, and then compares those measurements against a database of known faces. If the measurements match someone in the database closely enough, the system says "that's person X."

The technology does not read your emotions, your intentions, or your health status — despite what you might see in movies or marketing materials. It measures physical geometry. That is all it does. But that straightforward measurement has real consequences for how your face can be used, tracked, and stored.

Facial recognition exists in two main forms. Verification answers the question "Is this the person they claim to be?" — like unlocking your phone with your face. Identification answers "Who is this person?" — like searching a database of millions of mugshots to find a match. Verification is generally faster and more accurate because it is comparing one face to one known image. Identification is slower and more error-prone because it is comparing one face to thousands or millions.

Key Takeaways

  • Facial recognition measures the distance between facial features and compares those measurements to a database; it does not read emotions or intentions.
  • Verification (unlocking your phone) is more accurate than identification (finding someone in a crowd), because one-to-one matching is easier than one-to-many.
  • Law enforcement, airports, banks, and social media platforms all use facial recognition, but the rules about how they can use it vary widely by location.
  • Your face is collected and stored by companies and governments in ways you may not know about, and once stored, that data can be used for purposes you did not consent to.
  • You have some control over facial recognition on your own devices, but almost no control over how businesses and police use your face in public.

Where facial recognition is already in use

Your phone likely uses it. Apple's Face ID and Android's face unlock both use facial recognition to let you open your device without a password. Banks use it to verify your identity when you log in online. Airports in the United States use it to match your face against your passport photo at security checkpoints and departure gates.

Law enforcement agencies use facial recognition to search mugshot databases, driver's license photos, and passport images. The FBI maintains a database of over 600 million photos — mostly from driver's licenses and passport records — and can search it against photos from surveillance cameras or submitted images. State and local police have their own systems. Some use them carefully; others have made arrests based on false matches.

Social media platforms like Facebook and Google Photos use facial recognition to organize your photos, tag people automatically, and build databases of faces. Retailers use it in stores to track which customers enter and how long they stay. China uses it at a massive scale for surveillance and social control. The technology is everywhere, and the rules about how it can be used are still being written.

Why accuracy matters and where it fails

Facial recognition is not perfect. The accuracy depends on the quality of the camera image, the lighting, the angle of your face, whether you are wearing glasses or a mask, and the diversity of the database it is comparing against. A clear, straight-on photo in good light will match more reliably than a blurry image from a security camera at an angle.

The technology also has documented bias. Studies have shown that facial recognition systems are more accurate at identifying white men than Black women, Asian people, or other groups. This happens because many of the databases used to train these systems contained more images of white faces. When the system learns from biased data, it performs worse on faces it has seen less often. This is not a minor problem — it means innocent people from underrepresented groups are more likely to be falsely matched and investigated.

Law enforcement has arrested people based on facial recognition matches that turned out to be wrong. In 2020, a Black man in Detroit was arrested after facial recognition incorrectly matched him to a surveillance photo from a shoplifting case. He spent 30 hours in custody before the error was discovered. Similar cases have happened in other cities. The technology is treated as a lead-generation tool by police — a way to narrow down suspects — but it is sometimes treated as proof, which it is not.

What companies and governments collect and keep

When you unlock your phone with your face, that facial data usually stays on your device. Apple and Google say they do not send your face template to their servers. But when a retailer scans your face in a store, or when police photograph you at an arrest, or when you submit a photo to a social media platform, that data is collected and stored somewhere.

Governments keep facial data from driver's licenses, passport photos, and mugshots. Some states have passed laws limiting how police can use these databases; others have not. The federal government can access state driver's license photos through the FBI's system. Once your face is in a government database, you have limited ability to remove it or control how it is used.

Companies keep facial data for different reasons. Facebook and Google use it to organize photos and build advertising profiles. Retailers use it to track foot traffic and customer behavior. Data brokers buy and sell facial recognition data. Unlike government databases, there is often no law requiring companies to tell you they are collecting your face or to let you see what they have stored. You may never know your face is in their system.

Your control over facial recognition on your own devices

On phones and computers you own, you have real choices. You can turn off Face ID or face unlock in your settings and use a password instead. You can turn off photo organization features that use facial recognition. You can delete photos from cloud storage. You can adjust privacy settings on social media to limit facial recognition tagging.

On an iPhone, go to Settings > Face ID & Passcode and turn off "iPhone Unlock" or "App and iTunes Purchases." On Android, go to Settings > Security > Face Unlock and disable it. On Facebook, go to Settings > Privacy > Face Recognition and turn it off. On Google Photos, go to Settings > Privacy Controls > Face Recognition and disable it. These steps take less than a minute and give you direct control over whether the technology is used on your own data.

The tradeoff is convenience. Face unlock is faster than typing a password. Photo organization is genuinely useful. But if you value privacy over speed, the option exists. The choice is yours on your own device.

What you cannot control: facial recognition in public

You have almost no control over facial recognition used by businesses or police in public spaces. A retailer can scan your face when you walk into a store without telling you or asking permission. A police officer can photograph you and run your face through a database. A surveillance camera can capture your image and feed it into facial recognition software. You cannot opt out of these uses because they do not require your consent.

Some cities and states have passed laws limiting police use of facial recognition. San Francisco, Boston, and Portland have banned it for law enforcement. California requires a warrant before police can search facial recognition databases. But many places have no restrictions. And even where restrictions exist, they usually only explore to police — not to private companies or federal agencies.

If you are concerned about facial recognition in public, your options are limited. You can avoid cameras when possible. You can wear sunglasses or a hat, though this may draw more attention. You can contact your city council or state representative and ask them to pass restrictions on facial recognition. But you cannot prevent the technology from being used on you in most places.

The difference between your face and your fingerprint

Your face is not like your fingerprint. Your fingerprint is unique to you and does not change. Your face changes constantly — you age, you gain or lose weight, you grow facial hair, you wear makeup, you get injured. A facial recognition system trained on a photo of you from five years ago may not recognize you today.

More importantly, your face is visible. Anyone with a camera can capture it without your knowledge or consent. Your fingerprint requires you to touch something. Your face is on display every time you leave your house. This makes facial recognition fundamentally different from other biometric technologies. It can be collected at scale, from a distance, without your participation.

This is why facial recognition raises privacy concerns that fingerprints do not. A government or company can build a database of millions of faces without asking anyone for permission. They can use that database to track where you go, who you spend time with, and what you do in public. That power is new, and the rules about how it can be used are still being decided.

Frequently Asked Questions

Can facial recognition work if I am wearing a mask?

It depends on the system and the mask. Some systems can match faces with surgical masks or cloth masks covering the lower half of the face. Others cannot. N95 masks and full-face coverings are harder to match. During the COVID-19 pandemic, many facial recognition systems became less accurate because people were wearing masks. The technology is improving, but it is not yet reliable with masks in all situations.

If I delete a photo from my phone, does the company still have my face data?

Deleting a photo from your phone does not delete it from company servers if you uploaded it to cloud storage or social media. If you posted a photo on Facebook, Facebook still has your face data even if you delete the photo from your phone. You need to delete it from the cloud service or social media platform itself. Check your cloud storage settings and social media privacy controls to see what is stored where.

Can I request that my face be removed from a government database?

It depends on which database and which state or country. You can request removal of your mugshot from some police databases if your case was dismissed or you were acquitted, but the process varies. Driver's license photos are harder to remove because they are tied to your license. Federal databases like the FBI's are even harder to access or modify. Contact your state's attorney general or privacy office to learn what options exist in your location.

Does facial recognition work better on some races than others?

Yes. Studies have documented that facial recognition systems are significantly more accurate on white faces than on Black, Asian, or other non-white faces. This bias exists because the training data used to build these systems contained more white faces. The problem is being addressed by some companies, but it has not been solved. If you are from a group where the technology is less accurate, you are at higher risk of being misidentified.

What should I do if I think I was misidentified by facial recognition?

If you were arrested based on facial recognition, contact a lawyer when ready. Bring any evidence that contradicts the match — your location at the time, witnesses, timestamps, anything that shows you were not at the scene. If you were misidentified by a company or service, contact their customer service and request correction. Document everything. The technology is not perfect, and you have the right to challenge a match that is wrong.