Who's Afraid of Facial Recognition? Me.


Image Recognition Technology is Ubiquitous and Powerful

Image recognition technology has been around a while. It's extremely helpful in our lives in ways that a lot of other tech enhancements cannot match. Today I'm going to talk about why I'm finally scared that computers see so much more accurately than we do.

My day job is in cyber-security and one of my tech interests outside of work is computer vision and AI. I've been tinkering with image recognition and other machine vision for almost 15 years. Image recognition has matured quite a bit in the last decade while also becoming much more accessible. First it was digital cameras finding and highlighting human faces (and interesting leaves that looked like human faces), but grew to become telling you a list of all the things the computer could see in an image. Including the cat you never noticed was there or the motorcycle parked behind the SUV in the back corner of your photo.

Now though, computer vision is very mature and we've begun to see the power it can have. If you use Google Photos, you may notice how easily it can identify people in photos of someone even as they change with age or even wear costumes. How you can search for 'black and white cat' and it will show you a surprisingly accurate list of your pictures with black and white cats in them (for me this is hundreds).

You Best Start Believing in Cyberpunk Dystopias

The first time I started to worry about facial recognition technology was in Daniel Suarez's third novel: Kill Decision. Daniel Suarez writes amazing technology thrillers with a very high believability and you should check out his books. In Kill Decision, weaponized drones can pick a target out of a crowd with facial recognition and quickly murder the target with a highly accurate headshot. The book then goes on to talk about how automated warfare means that one person can make an unpopular decision to start or escalate conflict without having to rely on the loyalty of a large number of soldiers, meaning war can be waged for more selfish reasons. This theme inspired the Mindshare Metaverse powered soldier drones in Robo.

This year, I began to see posts online about a proof-of-concept for I-XRAY and it was more than a little scary to watch. I-XRAY is a set of smart lenses set in glasses which run your video feed through facial recognition and facial databases to attempt to instantly identify people you see out in the real world. Then, using that identification, it can feed you information about the person such as their name and where they work. I've included the video below, which has a demonstration by the creators:

As he says in the video, the facial recognition and camera glasses are only part of the process. First, the glasses use facial recognition to locate the face of the person you're looking at. Second, the picture of their face is sent to a service like PimEyes, which uses face matching to find other pictures which are likely to be that person on the internet. Third, a bot takes all of the text information from those websites and sends it to a Large Language Model AI which summarizes that text into relevant details found on the websites like names, addresses, and personal data. Finally, that information is fed into a personal inforamtion database to fill in missing parts of your data using the key pieces that were found previously.

All of this is already possible today, but I-XRAY and any tools which follow suit will be centralizing them and speeding up the research process. Automating this means that a picture of a crowd at a protest rally can quickly become an itemized list of everyone who was present, what their signs said, how they are registered to vote, and where they live. It can mean that a camera on the side of the road can identify the drivers who pass by in a day and a network of these devices could track where we go and spend our time.

As if those two examples aren't frightening enough, remember that it's been more than 12 years since the news story about how shopping and web browsing data just at Target was able to correctly determine a high school girl was pregnant. We are tracked and recorded constantly in our online lives, but this technology would now bring our daily commute or time at the coffee shop under the same microscope of data collection. This even shows up in another Daniel Suarez novel, Change Agent, where the people are constantly hounded by personalized ads whenever they pass a camera that gets a look at their face.

Fighting Back Against Tracking

So what can you do to protect yourself? The video above mentions opting out of the services used by I-XRAY. This will stop that particular chain, but you may not be able to opt out of illegal databases or those used by law enforcement or law enforcement adjacent companies. Similarly, sometimes data about you is stolen like in major personal information breaches at AT&T and Ticketmaster earlier this year.

To protect yourself against facial recognition technology in public, there are lots of clever ways to fool the software. In my reading about the subject, I found that one of the easiest ways is to wear a mask or face covering. This is a simple and "normal" way to avoid the tracking without expensive custom solutions.

Unfortunately, some governments are already trying to make wearing a mask in public illegal. Provisions for health considerations and discussions of masked protestors surrounding these laws show that there is already a path and a will to use technology like this to identify protestors. It's become a lot easier to automatically create lists of people who are trying to oppose the actions of those in power. With advancements like I-XRAY this moved quickly out of the realm of cyberpunk fiction and into our reality.

Make your social media accounts private if you can. Remove metadata from your photos and don't tag people without asking them. Opt out of personal information databases and check for data breaches that might affect you. Wear a mask if you're going to a protest. Turn off ad tracking in your phone.

Thanks for reading and good luck out there.