Alphabet’s Jigsaw expands AI-powered toxic comment detection technology to Spanish

Alphabet’s in-house incubator, Jigsaw, has revealed that it is opening up its artificial intelligence (AI)-powered abuse-detection technology to more languages, starting with Spanish.

Anyone who has spent time reading the comments section on websites will know all too well that they can be unpleasant places, with abuse and trolling comments commonplace. That is why Google’s Counter Abuse Technology team collaborated with Jigsaw last year to launch Perspective, an API for publishers to use on their platforms that automatically detects toxic comments. “Toxic” is defined as: “… a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion.”

Perspective kicked off last year in English, starting with the New York Times, and it later expanded to other outlets, including the Guardian, the Economist, and Wikipedia. Now Jigsaw said it’s working with Spanish-language newspaper El País to “improve conversations” on its website.

“This represents the first time that Perspective, technology that uses machine learning to spot abuse, is being used to analyze comments in Spanish,” Jigsaw’s Marie Pellat and Patricia Georgiou wrote in a blog post.

Jigsaw said in the coming months it will open up Perspective’s Spanish-language machine learning smarts to developers to “use and experiment with,” while over the next year it plans to expand Perspective to cover additional languages.

Spot abuse

In a nutshell, Perspective is trained via a human-generated database of comments that have already been labeled as toxic. The Perspective API essentially allows publishers to connect their own comments systems to this database, with Perspective rating each comment based on how similar it is to previously flagged comments.

Above: Perspective

Perspective is designed to work in tandem with human moderators, as it automatically sorts comments by their toxicity score, making it easy to start by approving or deleting comments with the highest ratings.

Interestingly, Perspective can also be a useful tool for commenters, giving them real-time feedback on how likely their comment is to be perceived as toxic. So if someone types a profanity-laden response to an article, they can see before they hit “publish” how likely their comment is to contravene community guidelines. And that is exactly how El País is using it.

“It highlights another way to use the information that Perspective provides —  a measurement of toxicity in language  —  in ways other than helping moderators sort comments or letting readers select which comments they see,” Jigsaw said. “Studies have shown that when people receive real-time feedback that their comments might be perceived as toxic, commenters often opt to rephrase their comments.”

Many online comment systems already have structures in place to help moderators manage user-generated responses, such as community-led “upvotes” and profanity filters. Using machine learning to teach a system based on historical data should go some way toward improving this process. However, Perspective has not been without controversy, with a number of false positives flagged, according to reports last year. One example showed that a phrase such as “I am a gay black woman” was deemed “toxic” by Perspective.

Investments

Companies have been investing heavily in technology to make their platforms more palatable, with the likes of Twitter and Microsoft rolling out various abuse and troll-detection tools in recent years. AI is playing a growing role in helping such companies manage content on their platforms at scale — back in October, Facebook revealed that it had used AI to remove nearly 9 million images of child nudity in the previous quarter alone. And Google recently released a new AI-powered Content Safety API to shield human moderators from exposure to child abuse images.

In the buildup to the launch with El País, Jigsaw said it has been working with the Spanish publication for a year to analyze historical public comments on its website.

“The process for training Perspective to work in new languages is identical to training it in English, but training machine learning models requires substantial datasets  —  in this case, lots of public online comments in Spanish,” Jigsaw said. “Those Spanish-language comments helped us train our machine learning models to understand how to spot toxicity in Spanish, as well as the linguistic nuances of that language.”