Facebooks AI tools are the only thing standing between users and the growing rush of hatred and misinformation on the platform. The company's researchers have developed some new features for the systems that keep the enemy at bay. They identify COVID-19 related misinformation and hateful language disguised as memes.
Detecting and removing misinformation about the virus is obviously a priority right now, as Facebook and other social media are not only breeding grounds for ordinary speculation and discussion, but also malicious intervention through organized campaigns aimed at sowing discord and Spread pseudosciences.
"We have seen a huge change in behavior on the website due to COVID-19, a huge increase in misinformation that we consider dangerous," said Mike Schroepfer, CTO of Facebook, in a press call today.
The company has contracts with dozens of fact-checking organizations around the world. Aside from how effective collaboration is, misinformation can quickly mutate and even remove a single image or link a complex issue.
For example, take a look at the following three sample images:In a way, they're almost identical, with the same background image, the same colors, the same font, etc. But the second one is a little different – you might see something like this when someone takes a screenshot and shares it instead of the original. The third is visually the same, but the words have the opposite meaning.
An unsophisticated computer vision algorithm would either rate these images as completely different due to these small changes (they lead to different hashes) or because of the overwhelming visual similarity they would all be the same. Of course, we see the differences immediately, but it is very difficult to train an algorithm to do it reliably. And the way things spread on Facebook may lead to thousands of variations, not a handful.
"We want to be able to recognize these things as identical because they are the same for one person," said Schröder. “Our previous systems were very accurate, but they were very fragile and brittle, even with very small changes. If you change a small number of pixels, we were too nervous to be different, so we would mark it as different and not remove it. In the past two and a half years we have built a neural network-based similarity detector with which we can again detect a greater variety of these variants with very high accuracy. "
Fortunately, analyzing images at these scales is a Facebook specialty. The infrastructure is used to compare photos and to search for functions such as faces and less desirable things. it just had to be taught what to look for. The result – from years of work, should be said – is SimSearchNet, a system dedicated to finding and analyzing nearly duplicates of a given image by scrutinizing key features (which may not match what you imagined).
SimSearchNet is currently checking every image uploaded to Instagram and Facebook – billions a day.
The system also monitors the Facebook marketplace, where people who try to circumvent the rules upload the same picture of an item for sale (e.g. an N95 face mask), but it has been edited slightly to avoid that the system marks it as not allowed. With the new system, the similarities between newly colored or otherwise edited photos are determined and sales are stopped.
Hateful memes and ambiguous skunks
Another problem that Facebook has addressed is hate speech – and sibling's loosely defined hate speech. One area that has proven particularly difficult for automated systems is memes.
The problem is that the meaning of these contributions often results from the interplay of images and text. Words that would be perfectly appropriate or ambiguous in themselves become clearer through the image on which they appear. In addition, there are infinite variations of images or phrases that can subtly change (or not change) the resulting meaning. See below:
Every single piece of the puzzle is fine in some contexts and offensive in others. How can a machine learning system learn to say what is good and what is bad? This “multimodal hate speech” is a non-trivial problem due to the way AI works. We have developed systems to understand language and classify images, but how these two things are connected is not a simple problem.
The Facebook researchers find that there is "surprisingly little" research on the subject, so their mission is more of an exploration mission than a solution. The technique they came to had several steps. First, they let people annotate a large collection of meme-type images as hateful or not hateful, creating the Hateful Memes record. Next, a machine learning system was trained on this data, but with a decisive difference to the existing one.
Almost all such image analysis algorithms, when presented with text and an image at the same time, classify one and then the other and then try to relate the two to each other. But that has the weakness mentioned above, that the text and images of hateful memes can be completely harmless regardless of the context.
The Facebook system combines the information from text and image earlier in the pipeline, which is referred to as "early fusion", to distinguish it from the traditional "late fusion" approach. This is more like how people do it – by looking at all the components of a medium before evaluating its meaning or tone.
Currently, the resulting algorithms are not ready for deployment in general – with an overall accuracy of around 65-70%, although Schroepfer has pointed out that the team uses "the most difficult of the most difficult problems" to assess the effectiveness. Some multimodal hate speeches are trivial to label as such, while others are difficult to assess even for people.
To promote the art, Facebook is holding a "Hateful Memes Challenge" as part of the NeurIPS AI conference later this year. This usually happens with difficult machine learning tasks, as new problems like this are like catnip for researchers.
The changing role of AI in Facebook politics
Facebook announced its plans to rely more on AI in the early days of the COVID 19 crisis to support moderation. In a press conference in March, Mark Zuckerberg said the company expects more “false positives” with the company's fleet of 15,000 moderation companies with paid vacations at home – cases of content that are flagged when they shouldn't.
At about the same time, YouTube and Twitter have also shifted a larger part of their content moderation to AI and issued similar warnings as an increased dependence on automated moderation can lead to content that does not violate platform rules being incorrectly labeled.
Despite its AI efforts, Facebook has sought to get its human content reviewers back into the office. In mid-April, Zuckerberg announced a schedule for when employees are expected to return to the office, and found that content reviewers were high on Facebook's list of "critical employees" who were marked for the earliest return.
While Facebook has warned that its AI systems may remove content too aggressively, hate speech, violent threats, and misinformation on the platform continue to increase as the coronavirus crisis spreads. Facebook has recently come under fire for spreading a viral video that is preventing people from wearing face masks or looking for vaccines as soon as they become available – a clear violation of the platform's rules against misinformation about health.
The video, an excerpt from an upcoming pseudo-documentation called "Plandemic", was originally published on YouTube. However, researchers found that Facebook's thriving conspiratorial group ecosystem shared it across the platform and brought it into mainstream online discourse. The 26-minute video full of conspiracies is also a perfect example of the kind of content that an algorithm can hardly understand.
On Tuesday, Facebook also published a report on community standards enforcement that lists moderation efforts in categories such as terrorism, harassment, and hate speech. While the results only cover a period of one month during the pandemic, we can assume that the shift from Facebook to AI moderation will have a greater impact next time.
In a call to the company's moderation efforts, Zuckerberg noted that the pandemic had made the “part of human review” of moderation significantly more difficult because of concerns about user privacy and employee mental health made remote work a challenge make for the auditor but navigated to a challenge for the company now. Facebook confirmed to theinformationsuperhighway that the company is now voluntarily returning a small portion of the full-time content reviewers to the office. Integrity Guy Rosen’s Facebook vice president, “the majority” of his contract reviewers can now do this from home. "People will continue to be a really important part of the equation," said Rosen.