Using artificial intelligence to expand fact-checking

Reporters' Lab projects are harnessing machine learning to assist fact-checking journalism

By Andrew Donohue – September 16, 2019 | Print this article

As news organizations adapt to the digital age, they’re turning to artificial intelligence to help human journalists produce the content consumers need. This is especially true in fact-checking.

Because politicians often repeat claims – even after they have been debunked – AI can help hold the politicians accountable by quickly finding relevant fact-checks. This technology can also search through vast amounts of content for fact-checkable claims, saving journalists time. 

“Fact-checking is uniquely suited to the use of AI,” said Bill Adair, director of the Duke Reporters’ Lab. 

The Reporters’ Lab uses AI to build tools like Squash, a system under development that fact-checks video of politicians as they speak. The goal is to display related fact-checks on viewers’ screens in a matter of seconds.

Squash listens to what politicians say and transcribes their words, making them searchable text. It then compares that text to
previously published fact-checks to look for matches. 

 “We’ve made some huge advancements in the past three years,” Adair said. “Squash has improved in accuracy since we demo’ed it at the State of the Union back in February.”

A screenshot of Squash, a fully automated fact-checking tool under development at The Reporters’ Lab.

This fall, the Squash team is refining its claim-matching technology. Its performance is inconsistent because people can make similar claims using different language.

Reporters’ Lab researchers hope to use more advanced machine learning techniques to help Squash become smarter at recognizing similar meaning even when the words don’t match. That will take time.

 “We’re dependent on technological processes improving,” Adair said. “Voice-to-text and matching algorithms are two big things we’re reliant on and those are continually improving, but still have a long way to go.” 

The Reporters’ Lab is also running user experience testing with Squash this fall to learn more about the most effective ways to display fact-checks on screens. Media researcher Jessica Mahone recently joined the lab to help develop a more effective user experience.   

 Squash could be the first step to a future where instant fact-checking is broadly available on broadcast TV, cable news and even web browsers, all thanks to the power of AI. Eventually viewers of all live political speeches and debates could benefit from Squash.

All of this is part of a larger movement within journalism starting to take advantage of AI’s possibilities. Outlets such as the Associated Press publish stories about sports and earning reports entirely written by computers. Xinhua, the Chinese’ state news agency, is experimenting with producing news broadcasts with virtual news anchors.

The Reporters’ Lab is one of the leading organizations in the world applying AI to fact-checking,  along with outlets FullFact in England and Chequeado in Argentina. The Lab’s Tech & Check Alerts, for instance, use AI to find and share checkable claims for fact-checking journalists around the country, so they do not have to spend time looking themselves. The Alerts have often shared claims that journalists have fact-checked.

It works like this: bots developed by Duke student researchers scrape Twitter posts and CNN transcripts daily to start the hunt for checkable claims. That content is fed to the ClaimBuster algorithm developed at the University of Texas, Arlington, which identifies potentially promising claims for fact-checkers. 

“Reading transcripts and watching TV looking for factual claims takes humans hours, but ClaimBuster can do it in seconds,” Adair said.

The Reporters’ Lab just last week debuted a new alert, The Best of the Bot, intended to flag the best of what the bots dig up. 

“We needed Best of the Bot because our Alerts had become so successful in finding claims that fact-checkers didn’t even have time to read them,” Adair said. “I think of it as a back-to-the-future approach. We now need a human to read the great work of the bot.”