At Tech & Check, some new ideas to automate fact-checking

Journalists and technologists met at Duke to dream up new ways that technology can help fact-checkers.

By Bill Adair – April 4, 2016 | Print this article

Last week, journalists and technologists gathered at Duke to dream up new ways that automation could help fact-checking.

The first Tech & Check conference, sponsored by the Duke Reporters’ Lab and Poynter’s International Fact-Checking Network, brought together about 50 journalists, students and computer scientists. The goal was to showcase existing projects and inspire new ones.

Tech and Check photo
At Tech & Check, groups of students, journalists and technologists dreamed up new ideas to automate fact-checking.

The participants included representatives of Google, IBM, NBC News, PolitiFact, Full Fact, FactCheck.org and the WRAL-TV. From the academic side, we had faculty and Ph.D students from Duke, the University of North Carolina, University of Texas-Arlington, Indiana University and the University of Michigan.

The first day featured presentations about existing projects that automate some aspect of fact-checking; the second day, attendees formed groups to conceive new projects.

The presentations showcased a wide variety of tools and research projects. Will Moy of the British site Full Fact did a demo of his claim monitoring tool that tracks the frequency of talking points, showing how often politicians said the phrase over time. Naeemul Hassan of the University of Texas at Arlington showed ClaimBuster, a project I’ve worked on, that can ingest huge amounts of text and identify factual claims that journalists might want to check.

IBM’s Ben Fletcher showed one of the company’s new projects known as Watson Angles, a tool that extracts information from Web articles and distills it into a summary that includes key players and a timeline of events. Giovanni Luca Ciampaglia, a researcher at Indiana University, showed a project that uses Wikipedia to fact-check claims.

On the second day, we focused on the future. The attendees broke into groups to come up with new ideas for research. The groups had 75 minutes to create three ideas for tools or further research. The projects showed the many ways that automation can help fact-checking.

One promising idea was dubbed “Parrot Score,” a website that could build on the approach that Full Fact is exploring for claim monitoring. It would track the frequency of claims and then calculate a score for politicians who use canned phrases more often. Tyler Dukes, a data journalist from WRAL-TV in Raleigh, N.C., said Parrot Score could be a browser extension that showed the origin of a claim and then tracked it through the political ecosystem.

Despite the focus on the digital future of journalism, we used Sharpies and a lot of Post-It notes.
Despite the focus on the digital future of journalism, we used Sharpies and a lot of Post-It notes.

Two teams proposed variations of a “Check This First” button that would allow people to verify the accuracy of a URL before they post it on Facebook or in a chat. One team dubbed it “ChatBot.” Clicking it would bring up information that would help users determine if the article was reliable.

Another team was assigned to focus on ways to improve public trust in fact-checkers. The team came up with several interesting ideas, including more transparency about the collective ratings for individual writers and editors as well as a game app that would simulate the process that journalists use to fact-check a claim. The app could improve trust by giving people an opportunity to form their own conclusions as well as demonstrating the difficult work that fact-checkers do.

Another team, which was focused on fact-checker tools, came up with some interesting ideas for tools. One would automatically detect when the journalists were examining a claim they had checked before.  Another tool would be something of a “sentence finisher” that, when a journalist began typing something such as “The unemployment rate last month…” would finish the sentence with the correct number.

The conference left me quite optimistic about the potential for more collaboration between computer scientists and fact-checkers. Things that never seemed possible, such as checking claims against the massive Wikipedia database, are increasingly doable. And many technologists are interested in doing research and creating products to help fact-checking.

Back to top

From ‘Baloney’ to ‘Screaming Lies’: the extreme ratings of the world’s fact-checkers

Our 2015 census of fact-checkers reveals the odd names they use for the most ridiculous falsehoods.

By Claire Ballentine – February 5, 2015 | Print this article

FactCheckEU calls them “Insane Whoppers.” The Voice of San Diego uses “Huckster Propaganda.” Honolulu Civil Beat refers to them as “Screaming Lies.”

From Rome to Hawaii and everywhere in between, the growth of political fact-checking has spawned new rating systems that use catchy names for the most ridiculous falsehoods.

While conducting our census of fact-checking sites around the world, we encountered some amusing ratings. Here is a sampling:

  • Canada’s Baloney Meter measures the accuracy of politicians’ statements based of how much “baloney” they contain. This ranges from “No Baloney” (the statement is completely accurate) to “Full of Baloney” (completely inaccurate).
  • FactCheckEU, which rates statements by politicians in Europe, uses a rating system that includes “Rather Daft” and “Insane Whopper.”
  • The Washington Post Fact Checker, written by reporter Glenn Kessler, utilizes the classic tale of Pinocchio to rate the claims made by politicians, political candidates and diplomats. A rating of one Pinocchio indicates some shading of the facts, while two Pinocchios means there were significant omissions or exaggerations. A rating of four Pinocchios simply means  “whoppers.” The French site Les Pinocchios uses a similar scale.
  • In Australia, ABC Fact Check uses a wide range of labels that are often tailored to the specific fact-check. They include “Exaggerated,” “Far-fetched,” “Cherrypicking” and “More to the Story.”
  • PolitiFact, the fact-checking venture of the Tampa Bay Times, uses the Truth-O-Meter, which rates statements from “True” to “Pants on Fire” (a rating reserved for the most ridiculous falsehoods).
  • The Honolulu Civil Beat rates the most outrageous statements as “Screaming Lies.”

    From The Hound in Mexico
    A false rating from The Hound in Mexico
  • Mexico’s new site The Hound rates statements from “Verdadero” (true) to “Ridiculo” (ridiculous), accompanied by images of dogs wearing detective hats. Uruguay’s UYCheck uses a similar scale. Argentina’s Chequeado also uses a “Verdadero” to “Falso” scale, plus ratings for “Exagerado” (exaggerated) and “Enganoso” (deceitful/misleading).
  • In California, the local website Voice of San Diego uses a system modeled after PolitiFact’s Truth-O-Meter. But instead of “Pants on Fire,” it uses “Huckster Propaganda.”
  • Denver’s NBC 9 Truth Test gives verdicts such as “Needs Context” and “Deceptive.”
  • In California, the Sacramento Bee’s Ad Watch uses a scale from “True” to “Outright Lie.”
  • Instead of words, WRAL in Raleigh uses traffic lights. Green is “go ahead, run with it”; red means “stop right there.”
  • Italy’s Pagella Politica labels its most far-fetched statements as “Panzana Pazzesca,” which loosely translates as “crazy fib” or “insane whopper.”
  • Australia’s Crikey Get Fact site named its fact-checking meter the Fib-O-Matic. Ratings range from true to “Rubbish.”
Back to top

A Viewer’s Guide to the N.C. Senate Debates

A preview of the upcoming NC Senate debate between Senator Kay Hagan and Speaker Thom Tillis

By Kyra Noonan – October 7, 2014 | Print this article

Political debates aren’t scripted, but the candidates usually come armed with some familiar talking points.

To help you sort out the truth in the talking points in this week’s debates for the U.S. Senate seat from North Carolina, the Duke Reporters’ Lab has compiled a viewer’s guide from fact-checking done in the past year by WRAL-TV, PolitiFact, the Washington Post FactChecker, FactCheck.org and McClatchy.

The debates are being held tonight and Thursday, with Democratic Sen. Kay Hagan and Republican Thom Tillis tonight and Libertarian Sean Haugh joining them Thursday.

ABC’s George Stephanopoulos will moderate tonight’s one-hour debate, which starts at 7 p.m. on TV and radio stations throughout the state. It will be streamed live at http://abcnews.go.com/live. On Thursday, the debate will also be held at 7 p.m. and broadcast on TV stations around the state.

Below are some of the lines and subjects you might hear, with links to the fact-checks. We’ve also posted a full list of all the fact-checks in the race so you can browse and search them.


In the past few weeks, the candidates have attacked each other about the rise of the terrorist group known variously as ISIS, ISIL or the Islamic State. Hagan has said Tillis has dodged questions about how he’d respond to the group, while Tillis has said Hagan is out of touch on the issue because she missed so many meetings of the Senate Armed Services Committee.

In one Tillis ad, the narrator says, “In January, President Obama refers to the Islamic State as a ‘JV team.’ Days later the Armed Services Committee holds a hearing on new global threats. Sen. Kay Hagan? Absent. In fact, Hagan’s missed half the Armed Services Committee hearings this year.”

PolitiFact checked Hagan’s attendance and found that Hagan had attended nine of 22 public meetings. It was not possible to determine if she attended the closed hearings, so PolitiFact rated Tillis’ claim Mostly True.


The biggest state issue in the campaign has been education, with Hagan and teacher groups attacking Tillis about teacher pay, while Tillis has claimed that he’s boosted education funding.

A TV ad from Women Vote, an arm of EMILY’s List, a political action committee that supports Democratic womenwho favor of abortion rights, said that Tillis “cut almost $500 million from education, causing crowded classrooms and forcing teachers to pay out-of-pocket for school supplies, while Tillis protected tax breaks for yachts and jets.”

PolitiFact checked the claim about the $500 million education cut and rated it Half True. The Washington Post’s FactChecker gave the claim a rating of “2 Pinocchios” and said that EMILY’s List “exaggerates the extent and impact of reductions in state funding for education last year – while ignoring the fact that the education budget is being bolstered this year”.

Obamacare/Health care

Denying CoverageOne of the largest issues both nationally and in the North Carolina Senate Race has been Obamacare. Tillis has claimed there would be a loss of jobs due to the Affordable Care Act, while Hagan and pro-Democratic groups have said that Tillis supports a plan which would make it difficult to obtain health care.

An ad by Patriot Majority USA, a pro-Democratic social welfare group, said in an ad that Tillis “sides with health insurance companies. He’d let them deny coverage for pre-existing conditions and raise rates for women needing mammograms. Tillis supports a plan that would end Medicare as we know it and force seniors to spend up to $1,700 more for prescriptions. Thom Tillis, he’s with the special interests, hurting North Carolina families.”

PolitiFact checked the claim that tillis would allow health insurance companies to “deny coverage for preexisting conditions” and rated it Mostly True. WRAL Fact Check gave this claim a “provisional yellow light”, as Tillis has yet to “fully articulate” his position on health care.

As for the claim that Tillis supports Paul Ryan’s plan that would “end Medicare as we know it”, PolitiFact rated it Mostly False on its Truth-O-Meter. PolitiFact said that “while Tillis has acknowledged supporting aspects of Ryan’s budget plan, he hasn’t specifically said whether or not he supports the original Medicare provision that would have made significant and mandatory changes to the program.”

Lost JobsIn response to Hagan’s support of Obamacare, Tillis has repeated a Republican talking point in an ad that the “Congressional Budget Office estimates 2 million lost jobs due to Obamacare.”

Fact-checkers have found that to be wrong. The Washington Post Fact Checker reviewed the report by the CBO and found that the “loss of jobs” was not stated, and could not have been given a numeric value, earning the claim a rating of “Three Pinocchios”. FactCheck.org also checked the claim and found that while “it has been a popular Republican talking point, it’s innacurate”. The CBO said more than 2 million people will “decide not to work, or will decide to work less, due to the law.”


To portray Tillis as out of touch with ordinary North Carolina voters, liberal groups have made a fuss about tax breaks for yachts and “private jets.”

The Senate Majority PAC ran an ad against Tillis that said, ”Speaker Tillis gave tax cuts to the wealthy and big corporations, even kept breaks for private yachts. He raised taxes on 80 percent of North Carolina.”

The Washington Post FactChecker reviewed the claim and gave it 3 Pinocchios finding that, “35 percent of the people appear to face a tax increase, including some of the wealthiest people in the state–not 80 percent, all at the bottom.”WRAL Fact Check checked the claim and gave it a “red light”. Politifact rated the claim by the Senate Majority PAC as “False”. And FactCheck, org writes that “…the ads’ central claim — that Tillis passed a whopping tax increase that hit 80 percent of North Carolinians” — is wrong.'”

A similar claim that said Tillis “gives tax breaks to yacht and jet owners” earned a False from PolitiFact, which said the group suggests Tillis created the tax breaks, but the law had been around for 23 years and Tillis just left it unchanged.

Hagan’s voting record

As the general election nears, candidates have tried to position themselves as moderates who can appeal to voters from both parties, with Hagan claiming that she is “the most moderate senator in the nation.”

WRAL Fact Check found this claim to be true backed by a National Journal report, and gave it a “Green Light”. McClatchy said, “”In February, the National Journal did rank her as the Senate’s ‘most moderate’ member’” but noted that “critics argue that the rankings are subjective, determined by the votes the Journal chooses to analyze.”

Tillis, meanwhile, has countered that in 2013, “Hagan voted with Obama 96 percent of the time.”

McClatchy found Tillis’ statement to be true and wrote “In 2013, the most recent year analyzed, Hagan voted in line with Obama’s preference 96 percent of the time.” WRAL gave this claim a “green light.”

Back to top