Category: Fact-Checking News
Fact-Checking News
A better ClaimReview to grow a global fact-check database
Google and the Reporters’ Lab have developed an easier method of labelling and logging fact-check articles.
By Sanha Lim - April 18, 2019
It’s now much easier for fact-checkers to use ClaimReview, a tagging tool that logs fact-checks published around the world into one database. The tool helps search engines — and readers — find non-partisan fact-checks published globally. It also organizes fact-check content into structured data that automated fact-checking will require.
Currently, only half of the roughly 160 fact-checking organizations that the Duke Reporters’ Lab tracks globally use ClaimReview. In response, Google and the Duke Reporters’ Lab have developed an easier method of labelling the articles to help both recruit more users and expand a vital fact-check data set.
ClaimReview was created in 2015 after a conversation between staff at Google and Glenn Kessler, the Washington Post fact-checker. Kessler wanted Google to highlight fact-checks in its search results. Bill Adair, director of the Duke Reporters’ Lab, was soon brought in to help.
Dan Brickley from Schema.org, Justin Kosslyn from Google and Adair developed a tagging system based on the schemas maintained by Schema.org, an organization that develops structured ways of organizing information. They created a universal system for fact-checkers to label their articles to include the claim checked, who said it and a ruling on its accuracy. “It’s the infrastructure that provides the atomic unit of fact-checking to search engines,” Adair said.
Initially, ClaimReview produced a piece of code that fact-checkers copy and pasted into their online content management system. Google and other search engines look for the code when crawling content. Next, Chris Guess of Adair’s team developed a ClaimReview widget called Share the Facts, a content box summarizing fact-checks that PolitiFact, FactCheck.org and the Washington Post can publish online and share on social media.
The latest version of ClaimReview no longer requires users to copy and paste the code, which can behave inconsistently on different content management systems. Instead, fact-checkers only have to fill out Google form fields similar to what they used previously to produce the code.
While the concept of ClaimReview is simple, it opens to the door to more innovation in fact-checking. It organizes data in ways that can be reused. By “structuring journalism, we can present content in more valuable ways to people,” said Adair.
By labeling fact-checks, the creators effectively created a searchable database of fact-checks, numbering about 24,000 today. The main products under development at the Reporters’ Lab, from FactStream to Squash, rely on fact-check databases. Automated fact-checking especially requires a robust database to quickly match untrue claims to previously published fact-checks.
The database ClaimReview builds offers even more possibilities. Adair hopes to tweak the fields fact-checkers fill in to provide better summaries of the fact-checks and provide more information to readers. In addition, Adair envisions ClaimReview being used to tag types of misinformation, as well as authors and publishers of false content. It could also tag websites that have a history of publishing false or misleading articles.
The tagging already is already benefiting some fact-check publishers. “ClaimReview helps to highlight and surface our fact-checks on Google, more than the best SEO skills or organic search would be able to achieve,” said Laura Kapelari, a journalist with Africa Check. ClaimReview has increased traffic on Africa Check’s website and helped the smaller Africa Check compete with larger media houses, she said. It also helps fact-checkers know which facts have already been investigated, which reduces redundant checks.
Joel Luther, the ClaimReview project manager in the Reporters’ Lab, expects this new ClaimReview format will save fact-checkers time and decrease errors when labeling fact-checks. However, there is still room to grow. Kapelari wishes there was a way for the tool to automatically grab key fields such as names in order to save time.
The Reporters’ Lab has a plan to promote ClaimReview globally. Adair is already busy on that front. Early this month, a group of international fact-checkers and technologists met in Durham for Tech & Check 2019, an annual conference where people on this quest share progress on automated fact-checking projects intended to fight misinformation. Adair, an organizer of Tech & Check, emphasized new developments with ClaimReview, as well as its promise for automating fact-checking.
Not much would be possible without this tool, he stressed. “It’s the secret sauce.”
During State of the Union, a failure…and a glimpse of Squash
Our app crashed under unusual traffic. But our tests of a new automated tool were a success.
By Bill Adair - February 6, 2019
We tested two fact-checking products during the State of the Union address. One failed, the other showed great promise.
The failure was FactStream, our iPhone app. It worked fine for the first 10 minutes of the speech. Users received two timely “quick takes” from Washington Post Fact Checker Glenn Kessler, but then the app crashed under an unusual surge of heavy traffic that we’re still investigating. We never recovered.
The other product is a previously secret project we’ve code-named Squash. It’s our first attempt at fully automated fact-checking. It converts speech to text and then searches our database of fact-checks from the Post, FactCheck.org and PolitiFact. When it finds a match, a summary of the fact-check pops onto the screen.
We’ve been testing Squash for the last few weeks with mixed results. Sometimes it finds exactly the right fact-checks. Other times the results are hilariously bad. But that’s what progress looks like.
We went into last night’s speech with very modest expectations. I said before the speech I’d be happy if the speech simply triggered fact-checks to pop up, even if it was a poor match.
But Squash actually performed pretty well. It had 20 pop-ups and six of them were in the ballpark.
Overall, the results were stunning. It gave us a glimpse of how good automated fact-checking can be.
We’ll have more to share once we’ve reviewed the results, so stay tuned.
As for FactStream, it now has lots of timely fact-checks from the State of the Union on the main home screen, which continues to function well. We will fix any problems we identify with the live event feature and plan to be back in action for real-time coverage for campaign events later this year.
Live fact-checking of the State of the Union address with our FactStream app
We're partnering with FactCheck.org, PolitiFact and Washington Post Fact Checker Glenn Kessler to provide real-time checks.
By Bill Adair - February 3, 2019
UPDATE, Feb. 5, 11 p.m.: Our FactStream app failed during the State of the Union address. We apologize for the problems. We are still sorting out what happened, but it appears we got hit with an unexpected surge of traffic that overwhelmed our servers and our architecture.
As we noted at the bottom of this post, this was a test – only our second of the app. We’ll fix the problems and be better next time.
—————-
The Reporters’ Lab is teaming up with the Washington Post, PolitiFact and FactCheck.org to offer live fact-checking of the State of the Union address on Tuesday night on our new FactStream app.
Journalists from the Post, PolitiFact, and FactCheck.org will provide real-time updates throughout the speech in two forms:
Ratings – Links to previously published fact-checks with ratings when the president repeats a claim that has been checked before.
Quick takes – Instant updates about a statement’s accuracy. They will be labeled red, yellow and green to indicate their truthfulness.
Tuesday’s speech will be the second test of FactStream. The first test, conducted during last year’s State of the Union address, provided users with 32 updates. We got valuable feedback and have made several improvements to the app.
FactStream is part of the Duke Tech & Check Cooperative, a project to automate fact-checking that is funded by Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation. Additional support has been provided by Google.
FactStream is available for iPhone and iPad (sorry, no Android version yet!) and is a free download from the App Store.
The app has two streams. One, shown by the home symbol in the lower left of the screen, provides a constant stream of the latest fact-checks published every day throughout the year. The live event feature for the State of the Union address is marked by an icon of a calendar with a check mark.
Because this is a test, users could encounter a few glitches. We’d love to hear about any bugs you encounter and get your feedback at team@sharethefacts.org.
Reporters’ Lab students are fact-checking North Carolina politicians
Student journalists and computer scientists find claims and report articles for the N.C. Fact-Checking Project
By Catherine Clabby - November 20, 2018
Duke Reporters’ Lab students expanded vital political journalism during a historic midterm campaign season this fall with the North Carolina Fact-Checking Project.
Five student journalists reviewed thousands of statements that hundreds of North Carolina candidates vying for state and federal offices made online and during public appearances. They collected newsy and checkable claims from what amounted to a firehose of political claims presented as fact.
Duke computer science undergraduates with the Duke Tech & Check Cooperative applied custom-made bots and the ClaimBuster algorithm to scrape and sort checkable political claims from hundreds of political Twitter feeds.
Editors and reporters then selected claims the students had logged for most of the project’s 30 plus fact-checks and six summary articles that the News and Observer and PolitiFact North Carolina published between August and November.
Duke senior Bill McCarthy was part of the four-reporter team on the project, which the North Carolina Local News Lab Fund supported to expand local fact-checking during the 2018 midterms and beyond in a large, politically divided and politically active state.
“Publishing content in any which way is exciting when you know it has some value to voters, to democracy,” said McCarthy, who interned at PolitiFact in Washington, D.C. last summer. “It was especially exciting to get so many fact-checks published in so little time.”
Reporters found politicians and political groups often did not stick with the facts during a campaign election season that that fielded an unusually large number of candidates statewide and a surge in voter turnout.
“NC GOP falsely ties dozens of Democrats to single-payer health care plan,” read one project fact-check headline. “Democrat falsely links newly-appointed Republican to health care bill,” noted another. The fact-check “Ad misleads about NC governors opposing constitutional amendments” set the record straight about some Democratic-leaning claims about six proposed amendments to the state constitution.
Digging for the Truth
Work in the lab was painstaking. Five sophomores filled weekday shifts to scour hundreds of campaign websites, social media feeds, Facebook and Google political ads, televised debates, campaign mailers and whatever else they could put their eyes on. Often they recorded one politician’s attacks on an opponent that might, or might not, be true.
Students scanned political chatter from all over the state, tracking competitive state and congressional races most closely. The resulting journalism was news that people could use as they were assessing candidates for the General Assembly and U.S. Congress as well as six proposed amendments to the state constitution.
The Reporters’ Lab launched a mini news service to share each fact-checking article with hundreds of newsrooms across the state for free.
The Charlotte Observer, a McClatchy newspaper like the N&O, published several checks. So did smaller publications such as Asheville’s Citizen-Times and the Greensboro News and Record. Newsweek cited a fact-check report by the N&O’s Rashaan Ayesh and Andy Specht about a fake photo of Justice Kavanaugh’s accuser, Christine Blasey Ford, shared by the chairman of the Cabarrus County GOP, which WRAL referenced in a roundup.
Project fact-checks influenced political discourse directly too. Candidates referred to project fact-checks in campaign messaging on social media and even in campaign ads. Democrat Dan McCready, who lost a close race against Republican Mark Marris in District 9, used project fact-checks in two campaign ads promoted on Facebook and in multiple posts on his Facebook campaign page, for instance.
While N&O reporter Andy Specht was reporting a deceptive ad from the Stop Deceptive Amendments political committee, the group announced plans to change it.
The fact-checking project will restart in January, when North Carolina’s reconfigured General Assembly opens its first 2019 session.
Fact-checking course to be offered this spring
New class will be taught by Reporters' Lab co-directors Adair and Stencel
By Catherine Clabby - October 22, 2018
We rate this true: The DeWitt Wallace Center will offer a new course in the spring titled “Fact-Checking American Politics.”
The course, taught by Bill Adair and Mark Stencel, will examine the growth of political fact-checking by organizations such as the Washington Post, PolitiFact and FactCheck.org, along with dozens more across the United States and around the world. Students will learn advanced techniques for researching political claims by candidates and elected officials and will write fact-check articles.
Adair, the Knight Professor of the Practice of Journalism and Public Policy, is the founder of PolitiFact and the International Fact-Checking Network. He worked 24 years for the St. Petersburg Times (now Tampa Bay Times) and covered the White House, Congress and the U.S. Supreme Court. Adair and the PolitiFact team won the Pulitzer Prize for National Reporting in 2009.
Mark Stencel is co-director of the Reporters’ Lab at Duke, where he tracks the spread and impact of fact-checking and teaches courses in political journalism. His introduction to fact-checking was working as a political researcher at The Washington Post during the 1992 presidential campaign. He has been a senior editor and media executive at The Post, Congressional Quarterly (now CQ-Roll Call) and National Public Radio.
The course will focus on independent analysis and advanced research techniques, including the importance of obtaining original documents and relying on multiple sources. Students will learn how to analyze claims and determine ratings. They also will learn how to identify, track and rate campaign promises.
Adair and Stencel will emphasize clear, well-argued, persuasive writing and well-supported fact-check ratings. They also will examine the impact of fact-checking on politicians and political discourse.
The course is listed PJMS 390S – Special Topics in Journalism. It is cross-listed as PUBPOL 290S.
Reporters’ Lab to launch project to promote ClaimReview
Google-funded project will seek to expand number of fact-checkers using tagging system
By Erica Ryan - October 2, 2018
The Duke Reporters’ Lab is launching a global effort to get more publishers to adopt ClaimReview, a schema.org open standard tagging system or “markup” that search engines and other major digital platforms use to find and highlight fact-checking articles.
The ClaimReview project, funded by a $200,000 grant from the Google News Initiative, will include a partnership with the International Fact-Checking Network, the global alliance of fact-checking organizations based at the Poynter Institute.
The Reporters’ Lab will develop instructional materials about ClaimReview and assist publishers in adopting a new tool to create the markup more easily with help from Google and Data Commons. The Lab also will work to expand the number of publishers around the world that are using ClaimReview. The IFCN will produce webinars and conduct outreach and training sessions at fact-checking conferences around the world, including the group’s annual Global Fact conference.
ClaimReview was developed three years ago through a partnership of the Reporters’ Lab, Google, and schema.org. It provides a standard way for publishers of fact-checks to identify the claim being checked, the person or entity that made the claim, and the conclusion of the article. The standardization enables search engines and other platforms to highlight the fact-checks in search results.
Google and Bing, the Microsoft search engine, both use ClaimReview to highlight fact-checking articles in search results and their own news products. Facebook announced in the summer that it plans to use ClaimReview as part of its partnership with fact-checkers.
The Reporters’ Lab uses ClaimReview as a key element in the Tech & Check Cooperative, our ambitious effort to automate fact-checking. Projects such as the FactStream app for iPhone and iPad and a new app being developed for television rely on the markup.
“ClaimReview is one of the untold success stories of the fact-checking movement,” said Bill Adair, director of the Reporters’ Lab. “It’s helping people find the facts in search results and helping fact-checkers increase their audience and impact.”
Despite the success, the Reporters’ Lab team estimates that roughly half the fact-checkers in the world still are not using the tagging system. Some fact-checkers have found it cumbersome to create the ClaimReview markup in their own publishing systems, while others have been confused about the different options for making it.
The new project will help editors switch to the new Google / Data Commons markup tool, a simpler way of generating ClaimReview, and provide technical assistance when they need it.
“We think of this project as ClaimReview 2.0,” Adair said. “This should expand the number of publishers using it, which should broaden the audience for fact-checking around the world.”
Spreading the word on The NC Fact-Checking Project
Journalists from the Reporters' Lab and the News & Observer appeared on Spectrum News
By Catherine Clabby - August 16, 2018
Cathy Clabby of The Reporters’ Lab and Andy Specht of The News & Observer appeared on Spectrum Cable’s “Politics Tonight” this week to explain the newly announced NC Fact-Checking Project. The journalists briefed Spectrum News host Tim 
Boyum on the ambitious plan to fact-check claims by politicians statewide during federal and state campaigns this fall and into the 2019 General Assembly session.
In the project, Duke journalism students and faculty – assisted by Tech & Check bots – will scour campaign messaging and news reports to find newsworthy claims by politicians that can be verified. Journalists at The News & Observer will select statements to check, report their veracity and craft the fact-checks. The reporting will be shared for free with print, broadcast and digital newsrooms statewide. To watch, click here.
The number of fact-checkers around the world: 156… and growing
Collaboration, aggregation and networks add to the Reporters' Lab ongoing survey of fact-checking projects in more than 50 countries.
By Mark Stencel - August 7, 2018
The number of active fact-checking projects around the world now stands at 156, with steady growth driven by expanding networks and new media partnerships that focus on holding public figures and organizations accountable for what they say.
And elections this year in the United States and around the globe mean that number will likely increase even more by the time the Duke Reporters’ Lab publishes its annual census early next year. Our map of the fact-checkers now shows them in 55 countries.
There were 149 active fact-checking ventures in the annual summary we published in February, up from 44 when we started this count in 2014. And after this summer’s Global Fact summit in Rome — where the attendee list topped 200 and the waitlist was more than three times as long — we still have plenty of other possible additions to vet and review in the coming weeks. So check back for updates.
Among the most recent additions is Faktiskt, a Swedish media partnership that aggregates reporting from five news organizations — two newspapers, two public broadcasters and a digital news service. We’ve seen other aggregation partnerships like this elsewhere, such as Faktenfinder in Germany and SNU FactCheck in South Korea. (This is a different model from the similarly named Faktisk partnership in Norway, where six news organizations operate a jointly funded fact-checking team whose work is made freely available as a public service to other media in the country.)
As we prepare for our annual fact-checking census, we plan to look more closely at the output of each contributor to these aggregation networks to see which of them we should also count as standalone fact-checkers. Our goal is to represent the full range of independent and journalistic fact-checking, including clusters of projects in particular countries and local regions, as well as ventures that find ways to operate across borders.
Along those lines, we also added checkmarks to our map for Africa Check‘s offices in Kenya and Nigeria. We had done the same previously for the South Africa-based project’s office in Senegal, which covers francophone countries in West Africa. The new additions have been around awhile too: The Kenya office has been in business since late 2016 and the Nigeria office opened two months later.
Meanwhile, our friends at Africa Check regularly help us identify other standalone fact-checking projects, including two more new additions to our database: Dubawa in Nigeria and ZimFact in Zimbabwe. The fast growth of fact-checking across Africa is one reason the International Fact-Checking Network’s sixth Global Fact summit will be in Cape Town next summer.
One legacy of these yearly summits is IFCN’s code of principles, and the code has established an independent evaluation process to certify that each of its signatories adheres to those ethical and journalistic standards. Our database includes all 58 signatories, including the U.S.-based (but Belgium-born) hoax-busting site Lead Stories; Maldita’s “Maldito Bulo” (or “Damned Hoax”) in Spain; and the “cek facta” section of the Indonesian digital news portal Liputan6. All three are among our latest additions.
There’s more to come from us. We plan to issue monthly updates as we try to keep our heads and arms around this fast-growing journalism movement. I’ll be relying heavily on Reporters’ Lab student researcher Daniela Flamini, who has just returned from a summer fact-checking internship at Chequeado in Argentina. Daniela takes over from recently graduated researcher Riley Griffin, who helped maintain our database for the past year.
Take a look at the criteria we use to select the fact-checkers we include in this database and let us know if you have any additions to suggest.
Reporters’ Lab joins N&O, UNC Reese News Lab on major fact-checking project
With grant from the N.C. Local News Lab Fund, partnership will expand non-partisan fact-checking throughout the state
By Catherine Clabby - August 1, 2018
The Duke Reporters’ Lab is joining McClatchy Carolinas and the UNC Reese News Lab in an ambitious project to expand non-partisan fact-checking throughout North Carolina.
With a $50,000 grant from the North Carolina Local News Lab Fund, the North Carolina Fact-Checking Project will build on the existing work at The News & Observer, add the work of student journalists and take advantage of new automated tools from the Duke Tech & Check Cooperative.
The project will evaluate statements by state and federal candidates in the 2018 election as well as lawmakers in the General Assembly session that begins in January 2019. The fact-checks will be produced by N&O journalists as part of PolitiFact North Carolina and will be made available for free to any news organization in the state for use online and in print.
The project will get support from the new TruthBuzz program of the International Center for Journalists (ICFJ), which is hiring an engagement fellow based on Raleigh to promote the North Carolina fact-checking.
The North Carolina Fact-Checking Project will put special emphasis on claims by politicians in rural parts of the state. Students in the Reporters’ Lab will scour news coverage and campaign ads for factual claims made by state, local and congressional candidates. The Lab will build new versions of its Tech & Check Alerts that use automated bots to find statements by politicians in social media that could be of interest to the North Carolina fact-checkers.
The Duke students and bots will provide daily suggestions of possible claims to The News & Observer, which will select which statements to research.
The UNC Reese News Lab will co-host a student seminar on fact-checking and help select a student journalist to work on the project. Representatives from Duke, TruthBuzz and the News & Observer will hold outreach sessions around the state to promote fact-checking and encourage news organizations to publish the project’s work.
About the partners:
The North Carolina Local News Lab Fund is a collaborative fund at the North Carolina Community Foundation established by a group of local and national funders who believe in the power of local journalism, local stories, and local people to strengthen our democracy.
The Duke Reporters’ Lab at the Sanford School of Public Policy is a center of research on fact-checking and automated journalism. The Lab tracks the growth of fact-checking around the world, conducts studies on important topics and develops tools to help journalists.
McClatchy Carolinas is the McClatchy division that publishes three newspapers in North Carolina, The News & Observer, The Charlotte Observer and The Herald-Sun. The N&O has a strong team of political reporters and has been the state’s PolitiFact partner for the last two years.
The Reese News Lab is an experimental media and research project based at the School of Media and Journalism at the University of North Carolina-Chapel Hill.
At Global Fact V: A celebration of community
More than 200 people attended the fifth meeting of the world's fact-checkers in Rome, which was organized by the International Fact-Checking Network.
By Bill Adair - June 25, 2018
My opening remarks at Global Fact V, the fifth annual meeting of the world’s fact-checkers, organized by the International Fact-Checking Network, held June 20-22 in Rome.
A couple of weeks ago, a photo from our first Global Fact showed up in my Facebook feed. Many of you will remember it: we had been all crammed into a classroom at the London School of Economics. When we went outside for a group photo, there were about 50 of us.
To show how our conference has grown, I posted that photo on Twitter along with one from our 2016 conference that had almost twice as many people. I also posted a third photo that showed thousands of people gathered in front of the Vatican. I said that was our projected crowd for this conference.
I rate that photo Mostly True.
What all of our conferences have in common is that they are really about community. It all began in that tiny classroom at the London School of Economics when we realized that whether we were from Italy or the U.K. or Egypt, we were all in this together. We discovered that even though we hadn’t talked much before or in many cases even met, we were facing the same challenges — fundraising and finding an audience and overcoming partisanship.
It was also a really powerful experience because we got a sense of how some fact-checkers around the world were struggling under difficult circumstances — under governments that provide little transparency, or, much worse, governments that oppress journalists and are hostile toward fact-checkers.
Throughout that first London conference there was an incredible sense of community. We’d never met before, but in just a couple of days we formed strong bonds. We vowed to keep in touch and keep talking and help each other.
It was an incredibly powerful experience for me. I was at a point in my career where I was trying to sort out what I would do in my new position in academia. I came back inspired and decided to start an association of fact-checkers – and hold these meetings every year.
The next year we started the IFCN and Poynter generously agreed to be its home. And then we hired Alexios as the leader.
Since then, there are have been two common themes. One you hear so often that it’s become my mantra: Fact-checking keeps growing. Our latest census of fact-checking in the Reporters’ Lab shows 149 active fact-checking projects and I’m glad to see that number keep going up and up.
The other theme, as I noted earlier, is community. I thought I’d focus this morning on a few examples.
Let’s start with Mexico, where more than 60 publishers, universities and civil society organizations have started Verificado 2018, a remarkable collaboration. It was originally focused largely on false news, but they’ve put more emphasis on fact-checking because of public demand. Daniel Funke wrote a great piece last week about how they checked a presidential debate.
In Norway, an extraordinary team of rivals has come together to create Faktisk, which is Norwegian for “actually” and “factually.” It launched nearly a year ago with four of the country’s biggest news organizations — VG, Dagbladet, NRK and TV 2 – and it’s grown since then. My colleague Mark Stencel likened it to the New York Times, The Washington Post and PBS launching a fact-checking project together.
At Duke, both of our big projects are possible because of the fact-checkers’ commitment to help each other. The first, Share the Facts and the creation of the ClaimReview schema, grew out of an idea from Glenn Kessler, the Washington Post Fact Checker, who suggested that Google put “fact-check” tags on search results.
That idea became our Duke-Google-Schema.org collaboration that created what many of you now use so search engines can find your work. And one unintended consequence: it makes automated fact-checking more possible. It all started because of one fact-checker’s sense of community.
Also, FactStream, the new app of our Tech & Check Cooperative, has been a remarkable collaboration between the big US fact-checkers — the Post, FactCheck.org and PolitiFact. All three took part in the beta test of the first version, our live coverage of the State of the Union address back in January. Getting them together on the same app was pretty remarkable. But our new version of the app –which we’re releasing this week – is even cooler. It’s like collaboration squared, or collaboration to the second power!
It took Glenn’s idea, which created the Share the Facts widget, and combined it with an idea from Eugene Kiely, the head of FactCheck.org, who said we should create a new feature on FactStream that shows the latest U.S. widgets every day.
So that’s what we did. And you know what: it’s a great new feature that reveals new things about our political discourse. Every day, it shows the latest fact-checks in a constant stream and users can click through, driving new traffic to the fact-checking sites. I’ll talk more about it during the automated demo session on Friday. But it wouldn’t be possible if it weren’t for the commitment to collaboration and community by Glenn and Eugene.
We’ve got a busy few days ahead, so let’s get on with it. There sure are a lot of you!
As we know from the photographs: fact-checking keeps growing.