“Automated fact-checking”

At Global Fact V: A celebration of community

More than 200 people attended the fifth meeting of the world's fact-checkers in Rome, which was organized by the International Fact-Checking Network.

By Bill Adair – June 25, 2018 | Print this article

My opening remarks at Global Fact V, the fifth annual meeting of the world’s fact-checkers, organized by the International Fact-Checking Network, held June 20-22 in Rome.

A couple of weeks ago, a photo from our first Global Fact showed up in my Facebook feed. Many of you will remember it: we had been all crammed into a classroom at the London School of Economics. When we went outside for a group photo, there were about 50 of us.

To show how our conference has grown, I posted that photo on Twitter along with one from our 2016 conference that had almost twice as many people. I also posted a third photo that showed thousands of people gathered in front of the Vatican. I said that was our projected crowd for this conference.

I rate that photo Mostly True.

What all of our conferences have in common is that they are really about community. It all began in that tiny classroom at the London School of Economics when we realized that whether we were from Italy or the U.K. or Egypt, we were all in this together. We discovered that even though we hadn’t talked much before or in many cases even met, we were facing the same challenges — fundraising and finding an audience and overcoming partisanship.

It was also a really powerful experience because we got a sense of how some fact-checkers around the world were struggling under difficult circumstances — under governments that provide little transparency, or, much worse, governments that oppress journalists and are hostile toward fact-checkers.

Throughout that first London conference there was an incredible sense of community. We’d never met before, but in just a couple of days we formed strong bonds. We vowed to keep in touch and keep talking and help each other.

It was an incredibly powerful experience for me. I was at a point in my career where I was trying to sort out what I would do in my new position in academia. I came back inspired and decided to start an association of fact-checkers – and hold these meetings every year.

The next year we started the IFCN and Poynter generously agreed to be its home. And then we hired Alexios as the leader.

Since then, there are have been two common themes. One you hear so often that it’s become my mantra: Fact-checking keeps growing. Our latest census of fact-checking in the Reporters’ Lab shows 149 active fact-checking projects and I’m glad to see that number keep going up and up.

The other theme, as I noted earlier, is community. I thought I’d focus this morning on a few examples.

Let’s start with Mexico, where more than 60 publishers, universities and civil society organizations have started Verificado 2018, a remarkable collaboration. It was originally focused largely on false news, but they’ve put more emphasis on fact-checking because of public demand. Daniel Funke wrote a great piece last week about how they checked a presidential debate.

In Norway, an extraordinary team of rivals has come together to create Faktisk, which is Norwegian for “actually” and “factually.” It launched nearly a year ago with four of the country’s biggest news organizations — VG, Dagbladet, NRK and TV 2 – and it’s grown since then. My colleague Mark Stencel likened it to the New York Times, The Washington Post and PBS launching a fact-checking project together.

 

At Duke, both of our big projects are possible because of the fact-checkers’ commitment to help each other. The first, Share the Facts and the creation of the ClaimReview schema, grew out of an idea from Glenn Kessler, the Washington Post Fact Checker, who suggested that Google put “fact-check” tags on search results.

That idea became our Duke-Google-Schema.org collaboration that created what many of you now use so search engines can find your work. And one unintended consequence: it makes automated fact-checking more possible. It all started because of one fact-checker’s sense of community.

Also, FactStream, the new app of our Tech & Check Cooperative, has been a remarkable collaboration between the big US fact-checkers — the Post, FactCheck.org and PolitiFact. All three took part in the beta test of the first version, our live coverage of the State of the Union address back in January. Getting them together on the same app was pretty remarkable. But our new version of the app –which we’re releasing this week – is even cooler. It’s like collaboration squared, or collaboration to the second power!

It took Glenn’s idea, which created the Share the Facts widget, and combined it with an idea from Eugene Kiely, the head of FactCheck.org, who said we should create a new feature on FactStream that shows the latest U.S. widgets every day.

So that’s what we did. And you know what: it’s a great new feature that reveals new things about our political discourse. Every day, it shows the latest fact-checks in a constant stream and users can click through, driving new traffic to the fact-checking sites. I’ll talk more about it during the automated demo session on Friday. But it wouldn’t be possible if it weren’t for the commitment to collaboration and community by Glenn and Eugene.

We’ve got a busy few days ahead, so let’s get on with it. There sure are a lot of you!

As we know from the photographs: fact-checking keeps growing.

 

Back to top

Tech & Check Alerts

Tech & Check Alerts aim to ease the workload of fact-checkers

Student-created tool can peruse political transcripts and find claims most likely to contain falsehoods

By Sydney McKinney – April 6, 2018 | Print this article

Students in the Duke Reporters’ Lab have built a bot that is like an intern who watches TV around the clock.

Asa Royal, a junior at Duke University, and Lucas Fagan, a freshman, have created Tech & Check Alerts, a new tool in a series of innovations the Reporters’ Lab is creating to help simplify the fact-checking process.

Using Tech & Check Alerts, the Lab can identify check-worthy claims in television news transcripts and send them to fact-checkers in daily email alerts.

“We’re going to save fact-checkers a lot of time and help them find things that they would otherwise miss,” said Mark Stencel, co-director of the Reporters’ Lab.

Though the fact-checking industry is growing worldwide, the organizations doing that work are typically small, even one-person enterprises, and the workload can be burdensome. Fact-checkers often have to sift through pages of text to find claims to check. This time-consuming process can create a substantial time gap between when statements are made and when fact-checks are available to viewers or readers.

The Tech & Check Alerts automate that process. Royal and Fagan, who are both computer science majors, created a program that scans transcripts of TV news channels, such as CNN, for claims that fact-checkers may want to investigate. It then compiles the check-worthy claims and sends them in a daily email to fact-checkers at The Washington Post, PolitiFact, the Associated Press, FactCheck.org and The New York Times, among others. Thus far, there have been seven fact-checks performed based on these alerts.

“Journalists don’t have to watch 15 hours of CNN or read the entire congressional report,” Royal said. “We’ll do it for them.”

Royal and Fagan created Tech & Check Alerts using ClaimBuster, an algorithm created by computer scientist Chengkai Li from the University of Texas at Arlington. ClaimBuster scans blocks of text and identifies “check-worthy” claims, based on indicators such as past-tense verbs, numbers, dates or statistics. It ranks statements from 0 to 1.0 based on how likely they are to be checkable; any statements that score a 0.7 or higher are typically considered check-worthy.

According to Royal, Li’s technology had yet to be used much outside of academia, so leaders of the Tech & Check Cooperative decided to utilize it for daily alerts.

“There’s already software that can find factual claims, and there are already fact-checkers who can check them,” Royal said. “We’re just solving the last-mile problem.”

The creation of Tech & Check Alerts is an important step for the Duke Tech & Check Cooperative, a two-year research project funded by the John S. and James L. Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation.

The broader purpose of this initiative is to bring together journalists, academics and computer scientists from across the country to innovate and automate the fact-checking industry. Over the course of two years, the Reporters’ Lab will develop tools that ease the job of fact-checkers and make fact-checking more accessible to consumers. Another tool the Lab is currently working on is FactStream, an app that provides instant fact-checking during live events.

Alongside other student researchers, Fagan and Royal are working to improve Tech & Check Alerts to include additional sources such as daily floor speeches and debates from the Congressional Record, and social media feeds from endangered incumbents running in this year’s closest House and Senate races. Fact-checkers will have input on how these additional alerts will be deployed.

Fagan is also building a web interface that would give fact-checking partners a way to dig deeper into these feeds and perhaps even customize certain alerts. Freshman Helena Merk, another student researcher in the Lab, is building a tool that would deliver the daily alerts directly to a channel on Slack, a communication platform used in many newsrooms.

Once these improvements are completed, and Tech & Check Alerts are deployed more widely, they should help fact-checkers across the country.

“This project is a stepping stone in our process of using real-time claims and existing fact-checks to automate fact-checking in real time,” Stencel said.

Back to top

Tech & Check Conference

Journalists, computer scientists gather for Tech & Check Conference at Duke

Members of the fact-checking community convened March 29-30 on Duke University's campus to tackle pressing issues

By Rebecca Iannucci – March 30, 2018 | Print this article

About 40 fact-checkers, journalists, computer scientists and academics gathered at Duke University March 29-30 for the Tech & Check Conference, a meeting hosted by the Reporters’ Lab.

As part of its Tech & Check Cooperative, the Reporters’ Lab is serving as a hub for automated fact-checking to connect journalists and technologists around the world. The conference gave them an opportunity to demonstrate current projects and discuss the big challenges of automation.

Some highlights of the conference:

Tech & Check Conference* Eleven demos of past and current projects.  Technologists and computer scientists showed off projects they’ve been developing to either automate fact-checking or improve the flow of accurate information on the internet.

Topics included new tools such as Chequeabot, an automated service that detects factual claims for the Argentinian fact-checker Chequeado; the Bad Idea Factory’s update of the Truth Goggles tool; and the perils of misinformation, including a real-life example from Penn State professor S. Shyam Sundar, whose research project about fake news was inaccurately described in widespread news coverage.

Tech & Check Conference

* Two Q&A panels. Alexios Mantzarlis, director of the International Fact-Checking Network, led a discussion with three fact-checkers about the potential tools and processes that could make fact-checking more efficient in the future.

Reporters’ Lab co-director Bill Adair moderated a conversation about challenges in automated fact-checking, including the pitfalls of voice-to-text technology and natural language processing.

Attendees also participated in breakout sessions to discuss ways to develop international standards and consistent terminology.

Photos by Colin Huth.

Back to top

FactStream

New Tech & Check projects will provide pop-up fact-checking

With advances in artificial intelligence and the growing use of the ClaimReview schema, Reporters' Lab researchers are developing a new family of apps that will make pop-up fact-checking a reality

By Julianna Rennie – January 16, 2018 | Print this article

For years, fact-checkers have been working to develop automated “pop-up” fact-checking. The technology would enable users to watch a political speech or a campaign debate while fact-checks pop onto their screens in real time.

That has always seemed like a distant dream. A 2015 report on “The Quest to Automate Fact-Checking” called that innovation “the Holy Grail” but said it “may remain far beyond our reach for many, many years to come.”

Since then, computer scientists and journalists have made tremendous progress and are inching closer to the Holy Grail. Here in the Reporters’ Lab, we’ve received $1.2 million in grants to make automated fact-checking a reality.

The Duke Tech & Check Cooperative, funded by Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation, is an effort to use automation to help fact-checkers research factual claims and broaden the audience for their work. The project will include about a half-dozen pop-up apps that will provide fact-checking on smartphones, tablets and televisions.

One key to the pop-up apps is a uniform format for fact-checks called the ClaimReview schema. Developed through a partnership of Schema.org, the Reporters’ Lab, Jigsaw and Google, it provides a standard tagging system for fact-checking articles that makes it easier for search engines and apps to identify the details of a fact-check. ClaimReview, which can be created using the Share the Facts widget developed by the Reporters’ Lab, will enable future apps to quickly find relevant fact-checking articles.

“Now, I don’t need to scrape 10 different sources and try to wrangle permission because there’s this database that will be growing increasingly,” says Dan Schultz, senior creative technologist at the Internet Archive.

This works because politicians repeat themselves. For example, many politicians and analysts have claimed that the United States has the highest corporate tax rate.

FactStreamThe Reporters’ Lab is developing several pop-up apps that will deliver fact-checking in real time. The apps will include:

  • FactStream, which will display relevant fact-checks on mobile devices during a live event. The first version, to be tested this month during the State of the Union address Jan. 30, will be a “manual” version that will rely on fact-checkers. When they hear a claim that they’ve checked before, the fact-checkers will compose a message containing the URL of the fact-check or a brief note about the claim. That message will appear in the FactStream app on a phone or tablet.
  • FactStream TV, which will use platforms such as Chromecast or Apple TV for similar pop-up apps on television. The initial versions will also be manual, enabling fact-checkers to trigger the notifications.

Another project, Truth Goggles, will be a plug-in for a web browser that will automatically scan a page for content that users should think about more carefully. Schultz, who developed a prototype of Truth Goggles as a grad student at the MIT Media Lab, will use the app to experiment with different ways to present accurate information and help determine which methods are most valuable for readers.

The second phase of the pop-up apps will take the human fact-checker out of the equation. For live events, the apps will rely on voice-to-text software and then match with the database of articles marked with ClaimReview.

The future apps will also need natural language processing (NLP) abilities. This is perhaps the biggest challenge because NLP is necessary to reflect the complexities of the English language.

“Human brains are very good at [NLP], and we’re pretty much the only ones,” says Chris Guess, the Reporters’ Lab’s chief technologist for Share the Facts and the Tech & Check Co-op. Programming a computer to understand negation or doublespeak, for instance, is extremely difficult.

Another challenge comes from the fact that there are few published fact-checks relative to all of the claims made in conversation or articles. “The likelihood of getting a match to the 10,000 or so stored fact-checks will be low,” says Bill Adair, director of the Reporters’ Lab.

Ideally, computers will eventually research and write the fact checks, too. “The ultimate goal would be that it could pull various pieces of information out, use that context awareness to do its own research into various data pools across the world, and create unique and new fact-checks,” Guess says.

The Reporters’ Lab is also developing tools that can help human fact-checkers. The first such tool uses ClaimBuster, an algorithm that can find claims fact-checkers might want to examine, to scan transcripts of newscasts and public events and identify checkable claims.

“These are really hard challenges,” Schultz says. “But there are ways to come up with creative ways around them.”

Back to top

Knight Foundation, Facebook and Craig Newmark provide funding to launch Duke Tech & Check Cooperative

New automated fact-checking project will build apps and coordinate with other researchers around the world

By Bill Adair – September 25, 2017 | Print this article

The John S. and James L. Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation are awarding grants to the Duke University Reporters’ Lab for a $1.2 million project to automate fact-checking.

The Duke Tech & Check Cooperative will bring together teams from universities and the Internet Archive to develop new ways to automate fact-checking and broaden the audience for this important new form of journalism.

During the two-year project, computer scientists and journalism faculty from Duke, the University of Texas at Arlington and Cal Poly-San Luis Obispo will build a variety of new tools and apps. Some will help journalists with time-consuming reporting tasks, such as mining transcripts, media streams and social feeds for the most important factual claims. Others will provide instant pop-up fact-checking during live events.

The Reporters’ Lab will also coordinate and share its automation efforts with journalists and computer scientists across the country and around the world. The Tech & Check Cooperative will connect the leaders of similar projects through its relationships with the International Fact-Checking Network, the global association of fact-checkers, and awardees of Knight Prototype Fund grants to address misinformation. The Lab will host an annual meeting and will hold regular video conferences.

Knight has provided $800,000 for the project and the Facebook Journalism Project has contributed $200,000. The Newmark Foundation has pledged $200,000.

A multitude of people and solutions are required to tackle the problem of misinformation in the digital age. The Reporters’ Lab is tackling the issue through an effective, multi-pronged approach, bringing together a network of journalists and technologists to build new tools that will promote the flow of accurate news, while strengthening their connections with major technology companies,” said Jennifer Preston, the vice president for journalism at Knight Foundation.

“The Duke Tech & Check Cooperative will tap into the power of technology to improve and expand fact-checking on a global scale,” said Campbell Brown, head of news partnerships at Facebook. “This important initiative will bring together some of the most respected experts in the industry along with new digital innovations to create practical and efficient tools for journalists and newsrooms.”

 “News consumers like me want the truth, which requires more and better fact-checking,” said Newmark, founder of craigslist and the Craig Newmark Foundation. “The Duke University Tech & Check Cooperative will soon become a vital part of the fact-checking network, and I’m excited to work with them to help build a system of information we can trust.”

The Tech & Check Cooperative will incorporate technology and content developed in Share the Facts, a Duke Reporters’ Lab partnership with the Google News Lab and Jigsaw. Share the Facts provides a way for the world’s fact-checkers to identify their articles for search engines and apps.

“Automated fact-checking is no longer just a dream,” said Bill Adair, the Knight Professor of the Practice of Journalism and Public Policy at Duke and the leader of the Tech & Check Cooperative. “Advances in artificial intelligence will soon make it possible to provide people with real-time information about what’s true and what’s not.”

Partners in the Tech & Check Cooperative include:

● The University of Texas at Arlington, which has developed ClaimBuster, a tool that can mine lengthy transcripts for claims that fact-checkers might want to examine.

● The Internet Archive, which will help develop a “Talking Point Tracker” that will identify factual claims that are used repeatedly by politicians and pundits.

● Truth Goggles, a project created by developer Dan Schultz and the Bad Idea Factory to provide pop-up fact-checking for articles on the web.

● Digital Democracy, an initiative of the Institute for Advanced Technology and Public Policy at Cal Poly-San Luis Obispo, which will develop ways to identify factual claims from video of legislative proceedings in California.

About the John S. and James L. Knight Foundation

Knight Foundation is a national foundation with strong local roots. We invest in journalism, in the arts, and in the success of cities where brothers John S. and James L. Knight once published newspapers. Our goal is to foster informed and engaged communities, which we believe are essential for a healthy democracy. For more, visit  knightfoundation.org.

About Facebook

Founded in 2004, Facebook’s mission is to give people the power to build community and bring the world closer together. People use Facebook to stay connected with friends and family, to discover what’s going on in the world, and to share and express what matters to them.

The Facebook Journalism Project was created in January 2017 to establish stronger ties between Facebook and the news industry.  FJP focuses on three pillars: collaborative development of new products; tools and trainings for journalists; and tools and trainings for people.

About Craig Newmark

Craig Newmark is a Web pioneer, philanthropist, and leading advocate on behalf of trustworthy journalism, voting rights, veterans and military families, and other civic and social justice causes. In 2017, he became a founding funder and executive committee member of the News Integrity Initiative, administered by the CUNY Graduate School of Journalism, which seeks to advance news literacy and increase trust in journalism.

About the Reporters’ Lab

The Duke Reporters’ Lab is a project of the DeWitt Wallace Center for Media & Democracy at the Sanford School of Public Policy. The Lab conducts research into fact-checking and explores how automation can be used to help journalists and broaden audiences for their work.

Back to top

PolitiFact Truth-O-Meter

Want to test FactPopUp? Here’s how to install and use the fact-checking tool

During Tuesday's speech, Google Chrome users will have another opportunity to experiment with our real-time fact-checking tool

By Gautam Hathi – February 23, 2017 | Print this article

UPDATE. 10 a.m. Feb. 28: We’ve discovered that some users get a black window when they should be seeing a photo of Trump on the livestream page. If that happens to you, close all your Chrome windows and relaunch Chrome. Email us at factpopup@gmail.com if you have problems and we’ll troubleshoot.

On Tuesday, Feb. 28, the Duke Reporters’ Lab will conduct another test of FactPopUp, our real-time fact-checking tool, during President Trump’s address to a joint session of Congress. The free Chrome extension will provide users with a livestream of the event, along with occasional pop-up notifications of fact-checks from PolitiFact, which will be checking Trump’s statements live.

Previous tests of FactPopUp have been encouraging, with more than 500 people successfully using the tool to receive fact-checks during the third presidential debate and President Trump’s inauguration.

If you would like to be a part of this test, here are a few simple steps to follow.

1. Go here and click the “Add to Chrome” button.

FactPopUp

2. Click the “Add extension” button on the prompt that comes up.

FactPopUp

3. Click the “Open live stream” button in the page that opens after the extension installs. This should open a web page with a full-screen livestream of the event.

FactPopUp

4. If you have to close the stream window before the event, click on the FactPopUp icon to the right of your Chrome address bar and then click “Open live stream”:

FactPopUp

5. If the FactPopUp icon doesn’t show up next to the address bar, find it in the Chrome menu.

FactPopUp

If you take part, we’d love to hear your feedback. Send your comments and/or questions to factpopup@gmail.com.

Back to top

PolitiFact Pants on Fire

Improved version of FactPopUp, our real-time fact-checking tool, now available

The Chrome extension, which fact-checks political events as they happen, is now more reliable and easier to use

By Gautam Hathi – January 26, 2017 | Print this article

We’ve made some improvements to FactPopUp, our real-time fact-checking tool.

FactPopUp allows fact-checking organizations to provide live fact-checks via Twitter to users watching a live stream of a political event on their computers.

The new version is more reliable and easier to use. A Twitter account is no longer required to use the extension, and FactPopUp can be easily configured to receive fact-check tweets from any Twitter account.

Powering the new version of FactPopUp is a significantly revised architecture. Users still download a simple Chrome extension from the Chrome webstore. However, fact-checking organizations will now run a simple server which uses Azure Notification Hubs to check for new tweets and send them to the extension clients.

The new version of FactPopUp was successfully tested by PolitiFact in order to provide live fact-checking for the presidential inauguration. More than 500 people have now participated in tests of FactPopUp, both during the inauguration and the 2016 election debates.

The code for the new version of FactPopUp is now available on GitHub, allowing anyone to set up and experiment with their own version of the system. In addition, FactPopUp is now configured so that the Reporters’ Lab can explore working with a range of fact-checking organizations to leverage FactPopUp for use during major political events around the world.

The Reporters’ Lab will continue to test and iterate FactPopUp, with the goal of eventually creating a universal live fact-checking solution that works on all major platforms and for all political events.

Back to top

Fact-checking comes to the Amazon Echo

New skill from the Duke Reporters’ Lab allows users to 'ask the fact-checkers'

By Erica Ryan – October 21, 2016 | Print this article

The Duke Reporters’ Lab has created a new fact-checking app for the Amazon Echo.

The app is a spin-off of Share the Facts, a project that has expanded the reach of fact-checking. The launch partners are PolitiFact, The Washington Post’s Fact Checker and FactCheck.org.

With the new Share the Facts skill, owners of the Echo and other Alexa-enabled devices, including the Tap and the Dot, can “ask the fact-checkers” about claims they hear from presidential candidates Hillary Clinton and Donald Trump, as well as other candidates and politicians who have been checked.

Share the Facts is now available in the Skills section of the Alexa app. (To find it, open the Alexa app on your smartphone, click on the left navigation panel, and then select “Skills.” From there, you can search for “Share the Facts” and select “Enable Skill.”)

We encourage you to try checking candidates’ claims from your couch after watching a campaign ad or during a discussion around the dinner table.

To begin a query, say: “Alexa, ask the fact-checkers.” (If you’re using the Tap, you’ll need to press the microphone button first and then say, “Ask the fact-checkers.”)

We have found it often works best if you wait for Alexa to reply, “Welcome to Share the Facts. We consolidate fact-checks from some of the most respected journalists in the U.S. Ask me to check a fact you’re wondering about” — and then ask your question, such as:

  • “Did Donald Trump oppose the war in Iraq?”
  • “Was Hillary Clinton right that her email practices were allowed?”
  • “Is it true that 300,000 Floridians have lost their health insurance because of Obamacare?”

Try to use the most important keywords in your question, following the examples above.

Share the Facts uses natural speech recognition to analyze and answer your questions from our database of roughly 2,000 professionally curated fact-checks. We scale our results so that they are timely and have the most consensus among our partners.

We welcome your feedback on our new Echo skill Share the Facts. Please send your thoughts to Share the Facts project manager Erica Ryan.

Back to top

Lessons from our first experiment with pop-up fact-checking

More than 380 people tried our pop-up fact-checking during the presidential debate.

By Bill Adair – October 20, 2016 | Print this article

Several hundred people who watched the presidential debate last night got a preview of what fact-checking will look like in the future.

More than 380 beta testers of our Reporters’ Lab browser extension saw fact-checks pop onto their screens in near-real time. The pop-ups were actually tweets from PolitiFact that said when statements from Hillary Clinton or Donald Trump were true, false or somewhere in between.

The pop-ups provided instant fact-checks from PolitiFact's Twitter feed.
The pop-ups provided instant fact-checks from PolitiFact’s Twitter feed.

The “FactPopUp” tool, built by Reporters’ Lab student Gautam Hathi, used the notification feature in the Chrome browser to display the PolitiFact tweets. The pop-ups appeared over a livestream of the debate. They were generated by Aaron Sharockman, a PolitiFact editor who was tweeting links to items that had been previously published.

Viewers saw 29 pop-ups with messages such as “Trump said Clinton would double taxes for Americans. False.”

We had modest expectations. We knew that the success of the test depended on Sharockman’s speed finding previous fact-checks and the time delay for the NBC News livestream we were using. It turned out that Sharockman was quite fast and the NBC livestream was slightly delayed, which made the pop-ups appear very quickly for most users.

I was so pleased at one point that I tweeted:

Still, there are many lessons and plenty of things we can improve:

Ease of installation. We tried to make it as easy as possible, but we were relying on Chrome’s notifications and Twitter for the pop-ups, so the sign up and installation was more complicated than we wanted. Users needed a Twitter account to activate the browser extension. Next time, we probably should build it as a standalone tool that doesn’t rely on Chrome or Twitter.

Type size. This was another challenge because of our reliance on Chrome notifications. We’d like to increase the size of the type (we had to rely on the standard font size) and the length of the messages (we were limited to about 80 characters). Building it as a standalone tool would allow us to make the type larger and the fact-checks slightly longer.

Location of the pop-ups. We discovered a difference in the location of the pop-ups for Mac and PC users. On the Mac version, they appeared in the upper right, which sometimes blocked Hillary Clinton’s face when NBC was showing a split screen. On the PC version of Chrome, they appeared in the lower right, which seemed less obtrusive.

Dwell time. We had tinkered with how long the pop-ups stayed on the screen. They need to stay long enough that people can read and digest them, but not too long. When the pop-up is in the lower right, you could have a a slightly longer dwell time because it doesn’t block anyone’s face.

Other functions. It’s worth exploring other functions such as the ability to save a fact-check to read later or immediately share it with others. Two of our testers suggested synchronizing the pop-ups with a video recording so users could go back to them later or see where they occurred in the debate.

Overall, we’re quite pleased with our pioneering effort at pop-up fact-checking. It shows great promise not just for future debates but also for live events such as major speeches. We’ll be making it available in GitHub for others to try and build on.

We’d love to hear your feedback. You can write to me at bill.adair@duke.edu.

 

Back to top

Reporters’ Lab to experiment with pop-up fact-checking during debate

The beta test of the Chrome extension will allow viewers to see real-time fact-checking with a livestream of the debate.

By Bill Adair – October 17, 2016 | Print this article

On Wednesday, the Duke Reporters’ Lab will test a new tool that will provide on-screen fact-checking during the third presidential debate.

The tool, a free extension for Chrome browsers, will provide real-time fact-checking from PolitiFact. Users will see a livestream of the debate with occasional messages that will pop onto the screen showing PolitiFact’s Truth-O-Meter rating for statements by the candidates.

We welcome your feedback. To try the tool, you must have the Chrome browser and download the extension. Follow the directions to get a personal identification number to activate it.

The “FactPopUp” tool was built by Gautam Hathi, a Duke computer science student who works in the Reporters’ Lab. The pop-ups will be generated by a PolitiFact editor watching the debate. When the editor hears one of the candidates make a claim that PolitiFact has checked before, the editor will quickly publish a tweet with a summary and the Truth-O-Meter rating. The summary and the rating then pop up on the livestream.

Viewers who take part in the beta test of our new browser extension will see fact-checks appear on a livestream of the debate.
Viewers who try our new browser extension will see fact-checks appear over a livestream of the debate.

Our private tests in the first two debates have been encouraging. The fact-checks pop up relatively quickly after the candidate makes the claim. It is reminiscent of the VH1 show Pop Up Video, which provided sometimes irreverent annotation to rock videos in the 1990s.

Our Chrome tool has some limitations. You need to have your own Twitter account so you can receive the fact-checks from PolitiFact’s tweets. And the browser extension can only provide pop-ups for statements that PolitiFact has previously researched and rated. And its quickness depends on the editor’s knowledge of PolitiFact’s 500-plus fact-checks on Donald Trump and Hillary Clinton.

The extension is being released as a beta for public testing, not as a final product. Hathi is optimistic that the extension will work well, but cautions that in some tests there have been delays in showing the pop-up.

Still, we’re hopeful that this will be a small first step toward the “Holy Grail” of fully automated fact-checking. Once we work out the bugs, we will make the browser extension available as an open source tool that can be used by others. Please send us feedback at factpopup@gmail.com.

The Reporters’ Lab, which is part of the DeWitt Wallace Center for Media & Democracy in Duke’s Sanford School of Public Policy, is exploring a variety of ways to automate fact-checking and expand the audience for this growing form of journalism. Earlier this year we unveiled the Share the Facts, a widget that provides a new way for readers to share fact-check articles and spread them virally across the Internet..

Back to top