Uber’s facial recognition locks out Indian drivers from their accounts

by James · December 6, 2022

Uber checks that a driver’s face matches what the company has on file through a program called “Real-Time ID Check.” It was rolled out in the US in 2016, in India in 2017, and then in other markets. “This prevents fraud and protects drivers’ accounts from being compromised. It also protects riders by building a new layer of accountability into the app to ensure the right person is behind the wheel,” Joe Sullivan, Uber’s chief security officer, said in a statement in 2017.

But the company’s driver verification procedures are far from seamless. Adnan Taqi, an Uber driver in Mumbai, ran into trouble with that when the app prompted him to take a selfie around dusk. He was banned for 48 hours, a major dent in his work schedule – he says he drives 18 hours straight, sometimes as much as 24 hours, to make ends meet. Days later, he took a selfie that locked him out of his account again, this time for a whole week. That time, Taqi suspects, it came down to hair: “I hadn’t shaved for a few days, and my hair had also grown out a bit,” he says.

More than a dozen drivers interviewed for this story detailed instances of having to find better lighting to avoid being banned from their Uber accounts. “Whenever Uber asks for a selfie in the evening or at night, I have had to stop and go under a street lamp to click a clear picture – otherwise there are chances of rejection,” said Santosh Kumar, an Uber driver from Hyderabad .

Others have struggled with scratches on their cameras and budget smartphones. The problem is not unique to Uber. Drivers with Ola, which is backed by SoftBank, face similar problems.

Some of these struggles can be explained by inherent limitations in facial recognition technology. The software starts by converting your face into a set of points, explains Jernej Kavka, an independent technology consultant with access to Microsoft’s Face API, which is what Uber uses to power real-time ID checks.

Adnan Taqi holds up the phone in the driver’s seat of the car. Variations in lighting and facial hair likely caused him to lose access to the app.

“With excessive facial hair, the points change and it may not recognize where the chin is,” says Kavka. The same happens when there is little light or the phone’s camera does not have good contrast. “This makes it difficult for the computer to detect edges,” he explains.

But the software can be particularly fragile in India. In December 2021, technology policy researchers Smriti Parsheera (a fellow with the CyberBRICS project) and Gaurav Jain (an economist at the International Finance Corporation) published a preprint paper that audited four commercial facial processing tools—Amazon’s Recognition, Microsoft Azure’s Face, Face++, and FaceX—for their achievements on Indian faces. When the software was applied to a database of 32,184 election candidates, Microsoft’s Face failed to detect the presence of a face in more than 1,000 images, producing an error rate of more than 3% — the worst of the four.

It could be that the Uber app is failing drivers because the software was not trained on a variety of Indian faces, says Parsheera. But she says there can be other problems as well. “There could be a number of other contributing factors such as lighting, angle, effects of aging, etc.,” she explained in writing. “But the lack of transparency around the use of such systems makes it difficult to give a more concrete explanation.”