“UYCheck”

Study explores new questions about quality of global fact-checking

The University of Wisconsin study examined fact-checks from Africa, India, Mexico, the United States, Uruguay and the United Kingdom.

By Bill Adair – August 11, 2015 | Print this article

How long should fact-checks be? How should they attribute their sources — with links or a detailed list? Should they provide a thorough account of a fact-checker’s work or distill it into a short summary?

Those are just a few of the areas explored in a fascinating new study by Lucas Graves, a journalism professor at the University of Wisconsin. He presented a summary of his research last month at the 2015 Global Fact-Checking Summit in London.

Lucas Graves
Lucas Graves

The pilot project represents the first in-depth qualitative analysis of global fact-checking. It was funded by the Omidyar Network as part of its grant to the Poynter Institute to create a new fact-checking organization. The study, done in conjunction with the Reporters’ Lab, lays the groundwork for a more extensive analysis of additional sites in the future.

The findings reveal that fact-checking is still a new form of journalism with few established customs or practices. Some fact-checkers write long articles with lots of quotes to back up their work. Others distill their findings into short articles without any quotes. Graves did not take a position on which approach is best, but his research gives fact-checkers some valuable data to begin discussions about how to improve their journalism.

Graves and three research assistants examined 10 fact-checking articles from each of six different sites: Africa Check, Full Fact in the United Kingdom, FactChecker.in in India, PolitiFact in the United States, El Sabueso in Mexico and UYCheck in Uruguay. The sites were chosen to reflect a wide range of global fact-checking, as this table shows:

Screen Shot 2015-08-11 at 3.26.38 PM
Click on the chart for more detail, then click browser “back” arrow to return to article.

Graves and his researchers found a surprising range in the length of the fact-checking articles. UYCheck from Uruguay had the longest articles, with an average word count of 1,148, followed by Africa Check at 1,009 and PolitiFact at 983.

The shortest were from Full Fact, which averaged just 354 words. They reflected a very different approach by the British team. Rather than lay out the factual claims and back them up with extensive quotes the way Screen Shot 2015-08-11 at 3.37.21 PMmost other sites do, the Full Fact approach is to distill them down to summaries.

Graves also found a wide range of data visualization in the articles sampled for each site. For example, Africa Check had three data visualizations in its 10 articles, while there were 11 in the Indian site FactChecker.in.

Graves found some sites used lots of data visualizations; others used relatively few.
Graves found some sites used lots of data visualizations; others used relatively few.

The Latin American sites UYCheck and El Sabueso used the most infographics, while the other sites relied more on charts and tables.

Graves also found a wide range in the use of web links and quotes. Africa Check averaged the highest total of web links and quotes per story (18), followed by 12 for PolitiFact, while UYCheck and El Sabueso had the fewest (8 and 5, respectively). Full Fact had no quotes in the 10 articles Graves examined but used an average of 9 links per article.

Graves and his researchers also examined how fact-checkers use links and quotes — whether they were used to provide political context about the claim being checked, to explain the subject being analyzed or to provide evidence about whether the claim was accurate. They found some sites, such as Africa Check and PolitiFact, used links more to provide context for the claim, while UYCheck and El Sabueso used them more for evidence in supporting a conclusion.

The analysis of quotes yielded some interesting results. PolitiFact used the most in the 10 articles — 38 quotes — with its largest share from evidentiary uses. Full Fact used the fewest (zero), followed by UYCheck (23) and El Sabueso (26).

The study also examined what Graves called “synthetic” sources — the different authoritative sources used to explain an issue and decide the accuracy of a claim. This part of the analysis distilled a final list of institutional sources for each fact-check, regardless of whether sources were directly quoted or linked to. AfricaCheck led the list with almost nine different authoritative sources considered on average, more than twice as many as FactChecker.in and UYCheck. Full Fact, UYCheck, and El Sabueso relied mainly on government agencies and data, while PolitiFact and Africa Check drew heavily on NGOs and academic experts in addition to official data.

The study raises some important questions for fact-checkers discuss. Are we writing are fact-checks too long? Too short?

Are we using enough data visualizations to help readers? Should we take the time to create more infographics instead of simple charts and tables?

What do we need to do to give our fact-checks authority? Are links sufficient? Or should we also include quotes from experts?

Over the next few months, we’ll have plenty to discuss.

TAGGED: , , , , , , , ,

Back to top

From ‘Baloney’ to ‘Screaming Lies’: the extreme ratings of the world’s fact-checkers

Our 2015 census of fact-checkers reveals the odd names they use for the most ridiculous falsehoods.

By Claire Ballentine – February 5, 2015 | Print this article

FactCheckEU calls them “Insane Whoppers.” The Voice of San Diego uses “Huckster Propaganda.” Honolulu Civil Beat refers to them as “Screaming Lies.”

From Rome to Hawaii and everywhere in between, the growth of political fact-checking has spawned new rating systems that use catchy names for the most ridiculous falsehoods.

While conducting our census of fact-checking sites around the world, we encountered some amusing ratings. Here is a sampling:

  • Canada’s Baloney Meter measures the accuracy of politicians’ statements based of how much “baloney” they contain. This ranges from “No Baloney” (the statement is completely accurate) to “Full of Baloney” (completely inaccurate).
  • FactCheckEU, which rates statements by politicians in Europe, uses a rating system that includes “Rather Daft” and “Insane Whopper.”
  • The Washington Post Fact Checker, written by reporter Glenn Kessler, utilizes the classic tale of Pinocchio to rate the claims made by politicians, political candidates and diplomats. A rating of one Pinocchio indicates some shading of the facts, while two Pinocchios means there were significant omissions or exaggerations. A rating of four Pinocchios simply means  “whoppers.” The French site Les Pinocchios uses a similar scale.
  • In Australia, ABC Fact Check uses a wide range of labels that are often tailored to the specific fact-check. They include “Exaggerated,” “Far-fetched,” “Cherrypicking” and “More to the Story.”
  • PolitiFact, the fact-checking venture of the Tampa Bay Times, uses the Truth-O-Meter, which rates statements from “True” to “Pants on Fire” (a rating reserved for the most ridiculous falsehoods).
  • The Honolulu Civil Beat rates the most outrageous statements as “Screaming Lies.”

    From The Hound in Mexico
    A false rating from The Hound in Mexico
  • Mexico’s new site The Hound rates statements from “Verdadero” (true) to “Ridiculo” (ridiculous), accompanied by images of dogs wearing detective hats. Uruguay’s UYCheck uses a similar scale. Argentina’s Chequeado also uses a “Verdadero” to “Falso” scale, plus ratings for “Exagerado” (exaggerated) and “Enganoso” (deceitful/misleading).
  • In California, the local website Voice of San Diego uses a system modeled after PolitiFact’s Truth-O-Meter. But instead of “Pants on Fire,” it uses “Huckster Propaganda.”
  • Denver’s NBC 9 Truth Test gives verdicts such as “Needs Context” and “Deceptive.”
  • In California, the Sacramento Bee’s Ad Watch uses a scale from “True” to “Outright Lie.”
  • Instead of words, WRAL in Raleigh uses traffic lights. Green is “go ahead, run with it”; red means “stop right there.”
  • Italy’s Pagella Politica labels its most far-fetched statements as “Panzana Pazzesca,” which loosely translates as “crazy fib” or “insane whopper.”
  • Australia’s Crikey Get Fact site named its fact-checking meter the Fib-O-Matic. Ratings range from true to “Rubbish.”

TAGGED: , , , , , , , , , , , , , , , , ,

Back to top

Fact-Checking Census finds continued growth around the world

Our latest tally of fact-checking sites finds 30 new sites in places such as Turkey, Uruguay and South Korea.

By Bill Adair & Ishan Thakore – January 19, 2015 | Print this article

Fact-checking keeps growing around the world, with new sites in countries such as Turkey, Uruguay and South Korea.

The 2015 Fact-Checking Census from the Duke Reporters’ Lab found 89 that have been active in the past few years and 64 that are active today. That’s up from 59 total/44 active when we did our last count in May 2014. (We include inactive sites in our total count because sites come and go with election cycles. Some news organizations and journalism NGOs only fact-check during election years.)

Many of the additional sites have started in the last seven months, including UYCheck in Uruguay and Dogruluk Payi in Turkey. Others are sites that we didn’t find when we did our first count.

You can see the complete list on the fact-checking page of the Reporters’ Lab website, where you can browse by continent and country.

As with our last tally, the largest concentrations of fact-checking are in Europe and North America. We found 38 sites in Europe (including 27 active), 30 in North America (22 active) and seven in South America (five active). There are two new sites in South Korea.

The Truth or False Poll in South Korea enlists readers to help with fact-checking.
The Truth or False Poll in South Korea enlists readers to help with fact-checking.

The percentage of sites that use ratings continues to grow, up from about 70 percent in last year’s count to 80 percent today. Many rating systems use a true to false scale while others have devised more creative names. For example, ratings for the European site FactCheckEU include “Rather Daft” and “Insane Whopper.” Canada’s Baloney Meter rates statements from “No Baloney” to “Full of Baloney.”

We found that 56 of the 89 sites are affiliated with news organizations such as newspapers and television networks. The other 33 are sites that are dedicated to fact-checking such as FactCheck.org in the United States and Full Fact in Great Britain.

Almost one-third of the sites (29 of the 89) track the campaign promises of elected officials. Some, such as the Rouhani Meter for Iran’s President Hassan Rouhani, only track campaign promises. Others, such as PolitiFact in the United States, do promise-tracking in addition to fact-checking.

For more information about the Reporters’ Lab database, contact Bill Adair at  bill.adair@duke.edu

TAGGED: , , , , , , , , ,

Back to top