Cindy Ord | Getty Images
Face detection algorithms from Los Angeles startup TrueFace are good enough that the U.S. Air Force uses them to speed up security checks at base entrances. However, CEO Shaun Moore faces a new question: How good is TrueFace technology when people wear face masks?
"It's something we don't know yet because it wasn't deployed in this environment," says Moore. His engineers test their technology on masked faces and rush to collect images of masked faces to match their machine learning algorithms to pandemic times.
Face recognition has become more widespread and more accurate in recent years as an artificial intelligence technology called deep learning makes computers much better at interpreting images. Governments and private companies use facial recognition to identify people in workplaces, schools, and airports, among other things, although some algorithms work less well for women and people with darker skin tones. Now the face recognition industry is trying to adapt to a world in which many people keep their faces covered to prevent the spread of disease.
Facial recognition experts say algorithms are generally less accurate when a face is covered, be it by an obstacle, camera angle, or mask, because less information is available for comparison. "If you have fewer than 100,000 people in the database, you won't feel the difference," said Alexander Khanin, CEO and co-founder of VisionLabs, a startup based in Amsterdam. Accuracy is significantly reduced in 1 million people and the system may need to be adjusted depending on how it is used.
Some face recognition providers and users say that the technology works well enough on masked faces. "We can identify a person wearing a balaclava or medical mask and hat on his forehead," says Artem Kuharenko, founder of NtechLab, a Russian company whose technology is used on 150,000 cameras in Moscow. He says the company has experience with face masks through contracts in Southeast Asia, where masks are worn to curb colds and flu. Its technology can identify masked faces, according to U.S. Customs and Border Guard, who uses facial recognition on travelers making international flights to U.S. airports.
However, Anil Jain, a professor at Michigan State University who deals with facial recognition and biometrics, says that such claims cannot be easily verified. "Companies can provide internal numbers, but we don't have a trustworthy database or rating to verify this," he says. "There is no third party validation."
A U.S. government laboratory at the National Institute of Standards and Technology, which acts as a global arbiter for the accuracy of facial recognition algorithms, hopes to enable this external validation – but is held back by the same pandemic that triggered the project.
Patrick Grother, a computer scientist who heads NIST's facial recognition test program, says his group is preparing tests to quantify how accurately algorithms identify people with masks. NIST plans to digitally add masks to its existing inventory of photos and test algorithms that have previously undergone a test to see if one photo matches another, similar to a border guard who checks passports. Companies are later asked to submit new algorithms for face masks. However, according to Grother, the timing of the project is uncertain as NIST has made fewer staff available due to the Covid 19 crisis.
Chinese and Russian companies typically dominate NIST's widely watched leaderboards to ensure face recognition accuracy. Stricter data protection regulations and widespread acceptance of surveillance make it easier for these companies to collect the data and operational experience needed to improve facial recognition algorithms. Companies from China and Russia claimed for the first time this year that their products were ready for a world of half-covered faces.
In early March, China's SenseTime, which became the world's most valuable AI startup, including through facial recognition from businesses and government agencies, announced that it had improved its product to control access to buildings and workplaces for face masks. The software takes care of uncovered facial features such as eyes, eyebrows and the bridge of the nose, a spokesman said. The United States restricted sales to SenseTime and other Chinese AI companies last year for allegedly providing technology to suppress Uyghur Muslims in northwest China.
China's facial recognition providers faced the challenge of identifying masked faces earlier and more generally, as the country is both the source of the novel corona virus and the world's most developed facial recognition market. Chinese citizens can use their faces to pay in shops or use ATMs, while government agencies are using the technology to get interested people off the streets and crowds.
Reports from China about the effectiveness of the systems with masks are mixed. A Beijing resident told WIRED that she appreciated the convenience of not having to remove her mask to use Alipay, China's leading mobile payment network, which has updated its facial recognition system. But Daniel Sun, a Gartner analyst also in Beijing, says he has to step out of the crowd to pull off his mask and use facial recognition to make payments. Still, he believes facial recognition will continue to grow, which may be supported by an interest in more hygienic, touchless transactions. "I don't think Covid-19 will stop the increasing use of this technology in China," said Sun.
The Japanese conglomerate NEC, which provides the facial recognition technology used by customs and border guards at US airports, is carefully discussing the technology's ability to mask faces. Benji Hutchinson, vice president of the U.S. division of NEC, says the company's laboratories in Japan that develop its algorithms have always been tested on face masks, which are often worn during the flu season in Asia. But the company has now started new test rounds because masks are to become the norm. "Masks are nothing new to us, but that doesn't mean that everything is perfect," says Hutchinson. He says the company is advising customers like CBP to make their own technology decisions for now.
Although international passengers are currently rare, a CBP spokesman said face recognition is still used at more than two dozen US airports and that the technology works with masks. "CBP's biometric facial comparison technology can match travelers wearing masks with photos from their travel documents," the spokesman said.
The CBP system checks the faces of the passengers at the departure gate for “facial prints” of photos that the Department of Homeland Security has saved for the people listed on this flight. Although the agency says passengers are always free to sign out, some people have found it difficult to do so. According to the CBP, a person wearing a face mask can retain the technology if the technology fails while a person manually checks their passport.
Will Knight contributed to the reporting.
This story originally appeared on wired.com.