UPDATE. 10 a.m. Feb. 28: We’ve discovered that some users get a black window when they should be seeing a photo of Trump on the livestream page. If that happens to you, close all your Chrome windows and relaunch Chrome. Email us at email@example.com if you have problems and we’ll troubleshoot.
On Tuesday, Feb. 28, the Duke Reporters’ Lab will conduct another test of FactPopUp, our real-time fact-checking tool, during President Trump’s address to a joint session of Congress. The free Chrome extension will provide users with a livestream of the event, along with occasional pop-up notifications of fact-checks from PolitiFact, which will be checking Trump’s statements live.
Previous tests of FactPopUp have been encouraging, with more than 500 people successfully using the tool to receive fact-checks during the third presidential debate and President Trump’s inauguration.
If you would like to be a part of this test, here are a few simple steps to follow.
We’ve made some improvements to FactPopUp, our real-time fact-checking tool.
FactPopUp allows fact-checking organizations to provide live fact-checks via Twitter to users watching a live stream of a political event on their computers.
The new version is more reliable and easier to use. A Twitter account is no longer required to use the extension, and FactPopUp can be easily configured to receive fact-check tweets from any Twitter account.
Powering the new version of FactPopUp is a significantly revised architecture. Users still download a simple Chrome extension from the Chrome webstore. However, fact-checking organizations will now run a simple server which uses Azure Notification Hubs to check for new tweets and send them to the extension clients.
The new version of FactPopUp was successfully tested by PolitiFact in order to provide live fact-checking for the presidential inauguration. More than 500 people have now participated in tests of FactPopUp, both during the inauguration and the 2016 election debates.
The code for the new version of FactPopUp is now available on GitHub, allowing anyone to set up and experiment with their own version of the system. In addition, FactPopUp is now configured so that the Reporters’ Lab can explore working with a range of fact-checking organizations to leverage FactPopUp for use during major political events around the world.
The Reporters’ Lab will continue to test and iterate FactPopUp, with the goal of eventually creating a universal live fact-checking solution that works on all major platforms and for all political events.
The Duke Reporters’ Lab has created a new fact-checking app for the Amazon Echo.
The app is a spin-off of Share the Facts, a project that has expanded the reach of fact-checking. The launch partners are PolitiFact, The Washington Post’s Fact Checker and FactCheck.org.
With the new Share the Facts skill, owners of the Echo and other Alexa-enabled devices, including the Tap and the Dot, can “ask the fact-checkers” about claims they hear from presidential candidates Hillary Clinton and Donald Trump, as well as other candidates and politicians who have been checked.
Share the Facts is now available in the Skills section of the Alexa app. (To find it, open the Alexa app on your smartphone, click on the left navigation panel, and then select “Skills.” From there, you can search for “Share the Facts” and select “Enable Skill.”)
We encourage you to try checking candidates’ claims from your couch after watching a campaign ad or during a discussion around the dinner table.
To begin a query, say: “Alexa, ask the fact-checkers.” (If you’re using the Tap, you’ll need to press the microphone button first and then say, “Ask the fact-checkers.”)
We have found it often works best if you wait for Alexa to reply, “Welcome to Share the Facts. We consolidate fact-checks from some of the most respected journalists in the U.S. Ask me to check a fact you’re wondering about” — and then ask your question, such as:
“Did Donald Trump oppose the war in Iraq?”
“Was Hillary Clinton right that her email practices were allowed?”
“Is it true that 300,000 Floridians have lost their health insurance because of Obamacare?”
Try to use the most important keywords in your question, following the examples above.
Share the Facts uses natural speech recognition to analyze and answer your questions from our database of roughly 2,000 professionally curated fact-checks. We scale our results so that they are timely and have the most consensus among our partners.
We welcome your feedback on our new Echo skill Share the Facts. Please send your thoughts to Share the Facts project manager Erica Ryan.
Several hundred people who watched the presidential debate last night got a preview of what fact-checking will look like in the future.
More than 380 beta testers of our Reporters’ Lab browser extension saw fact-checks pop onto their screens in near-real time. The pop-ups were actually tweets from PolitiFact that said when statements from Hillary Clinton or Donald Trump were true, false or somewhere in between.
The “FactPopUp” tool, built by Reporters’ Lab student Gautam Hathi, used the notification feature in the Chrome browser to display the PolitiFact tweets. The pop-ups appeared over a livestream of the debate. They were generated by Aaron Sharockman, a PolitiFact editor who was tweeting links to items that had been previously published.
Viewers saw 29 pop-ups with messages such as “Trump said Clinton would double taxes for Americans. False.”
We had modest expectations. We knew that the success of the test depended on Sharockman’s speed finding previous fact-checks and the time delay for the NBC News livestream we were using. It turned out that Sharockman was quite fast and the NBC livestream was slightly delayed, which made the pop-ups appear very quickly for most users.
Still, there are many lessons and plenty of things we can improve:
Ease of installation. We tried to make it as easy as possible, but we were relying on Chrome’s notifications and Twitter for the pop-ups, so the sign up and installation was more complicated than we wanted. Users needed a Twitter account to activate the browser extension. Next time, we probably should build it as a standalone tool that doesn’t rely on Chrome or Twitter.
Type size. This was another challenge because of our reliance on Chrome notifications. We’d like to increase the size of the type (we had to rely on the standard font size) and the length of the messages (we were limited to about 80 characters). Building it as a standalone tool would allow us to make the type larger and the fact-checks slightly longer.
Location of the pop-ups. We discovered a difference in the location of the pop-ups for Mac and PC users. On the Mac version, they appeared in the upper right, which sometimes blocked Hillary Clinton’s face when NBC was showing a split screen. On the PC version of Chrome, they appeared in the lower right, which seemed less obtrusive.
Dwell time. We had tinkered with how long the pop-ups stayed on the screen. They need to stay long enough that people can read and digest them, but not too long. When the pop-up is in the lower right, you could have a a slightly longer dwell time because it doesn’t block anyone’s face.
Other functions. It’s worth exploring other functions such as the ability to save a fact-check to read later or immediately share it with others. Two of our testers suggested synchronizing the pop-ups with a video recording so users could go back to them later or see where they occurred in the debate.
Overall, we’re quite pleased with our pioneering effort at pop-up fact-checking. It shows great promise not just for future debates but also for live events such as major speeches. We’ll be making it available in GitHub for others to try and build on.
We’d love to hear your feedback. You can write to me at firstname.lastname@example.org.
On Wednesday, the Duke Reporters’ Lab will test a new tool that will provide on-screen fact-checking during the third presidential debate.
The tool, a free extension for Chrome browsers, will provide real-time fact-checking from PolitiFact. Users will see a livestream of the debate with occasional messages that will pop onto the screen showing PolitiFact’s Truth-O-Meter rating for statements by the candidates.
We welcome your feedback. To try the tool, you must have the Chrome browser and download the extension. Follow the directions to get a personal identification number to activate it.
The “FactPopUp” tool was built by Gautam Hathi, a Duke computer science student who works in the Reporters’ Lab. The pop-ups will be generated by a PolitiFact editor watching the debate. When the editor hears one of the candidates make a claim that PolitiFact has checked before, the editor will quickly publish a tweet with a summary and the Truth-O-Meter rating. The summary and the rating then pop up on the livestream.
Our private tests in the first two debates have been encouraging. The fact-checks pop up relatively quickly after the candidate makes the claim. It is reminiscent of the VH1 show Pop Up Video, which provided sometimes irreverent annotation to rock videos in the 1990s.
Our Chrome tool has some limitations. You need to have your own Twitter account so you can receive the fact-checks from PolitiFact’s tweets. And the browser extension can only provide pop-ups for statements that PolitiFact has previously researched and rated. And its quickness depends on the editor’s knowledge of PolitiFact’s 500-plus fact-checks on Donald Trump and Hillary Clinton.
The extension is being released as a beta for public testing, not as a final product. Hathi is optimistic that the extension will work well, but cautions that in some tests there have been delays in showing the pop-up.
Still, we’re hopeful that this will be a small first step toward the “Holy Grail” of fully automated fact-checking. Once we work out the bugs, we will make the browser extension available as an open source tool that can be used by others. Please send us feedback at email@example.com.
The Reporters’ Lab, which is part of the DeWitt Wallace Center for Media & Democracy in Duke’s Sanford School of Public Policy, is exploring a variety of ways to automate fact-checking and expand the audience for this growing form of journalism. Earlier this year we unveiled the Share the Facts, a widget that provides a new way for readers to share fact-check articles and spread them virally across the Internet..
The Reporters’ Lab and Poynter’s International Fact-Checking Network will host “Tech & Check”, the first conference to explore the promise and challenges of automated fact-checking.
Tech & Check, to be held March 31-April 1 at Duke University, will bring together experts from academia, journalism and the tech industry. The conference will include:
Demos and presentations of current research that automates fact-checking
Discussions about the institutional challenges of expanding the automated work
Discussions on new areas for exploration, such as live fact-checking and automated annotation.
Research in computational fact-checking has been underway for several years, but has picked up momentum with a flurry of new projects.
While automating fact-checking entirely is still the stuff of science fiction, parts of the fact-checking process such as gathering fact-checkable claims or matching them with articles already published seem ripe for automation. As natural language processing (NLP) and other artificial intelligence tools become more sophisticated, the potential applications for fact-checking will increase.
Indeed, around the world several projects are exploring ways to make fact-checking faster and smarter through the use of technology. For example, at Duke University, an NSF-funded project uses computational power to help fact-checkers verify common claims about the voting records of members of Congress. The University of Texas-Arlington has developed a tool called ClaimBuster that can analyze long transcripts of debates and suggest sentences that could be fact-checked. At Indiana University, researchers have experimented with a tool that uses Wikipedia and knowledge networks to verify simple statements. Fact-checkers in France, Argentina, the U.K. and Italy are also doing work in this field.
The conference is made possible with support by, among others, the Park Foundation. More details will be published in the coming weeks.
Researchers and journalists interested in attending the conference should contact the International Fact-Checking Network at firstname.lastname@example.org
Two projects from the Duke Reporters’ Lab were featured at the 2015 Computation + Journalism Symposium, which was held over the weekend at Columbia University in New York.
The two-day conference included presentations about Structured Stories NYC, an experiment that involved three Duke students covering events in New York, and a separate project that is exploring new ways to automate fact-checking.
Structured Stories, which uses a unique structured journalism approach to local news, was the topic of a presentation by David Caswell, a fellow at the Reynolds Journalism Institute.
Structured Stories NYC is one of the boldest experiments of structured journalism because it dices the news into short events that can be reassembled in different ways by readers. The site is designed to put readers in charge by allowing them to adjust the depth of story coverage.
On the second day of the conference, Reporters’ Lab Director Bill Adair and Naeemul Hassan, a Ph.D. student in computer science at the University of Texas-Arlington, made a presentation that Adair said was “a call to arms” to automate fact-checking. It was based on a paper called The Quest to Automate Fact-Checking that they co-authored with Chengkai Li and Mark Tremayne of the University of Texas-Arlington, Jun Yang of Duke, James Hamilton of Stanford University and Cong Yu of Google.
Adair spoke about the need for more research to achieve the “holy grail” of fully automated, instant fact-checking. Hassan gave a presentation about ClaimBuster, a tool that analyzes text and predicts which sentences are factual claims that fact-checkers might want to examine.
The Reporters’ Lab is working with computer scientists and researchers from UT-Arlington, Stanford and Google on the multi-year project to explore how computational power can assist fact-checkers.