Category: Tech & Check Cooperative
Tech & Check Cooperative
“We need to disrupt the lies”
At the Simón Bolívar National Journalism Awards, a call for a more aggressive approach to fact-checking
By Bill Adair - November 16, 2018
My remarks for the Simón Bolívar National Journalism Awards, Bogota, Colombia, Nov. 15, 2018.
This is a critical moment for journalism around the world, when the path ahead seems uncertain. We lived through the dawn of the Information Age and saw the great promise of the internet; but it now seems like we are in a darker time.
When I started PolitiFact in 2007, I was filled with hope about what the digital revolution would bring. There was a belief the internet could make information more widely available, bring people together and help us hold power accountable. Like many of you, I am wondering if those hopes were misplaced. But we can’t get despondent about how things have turned out. We need to reimagine our roles as journalists and harness the power of technology to combat misinformation.
A little history: I started PolitiFact out of my own guilt. I had been covering the White House and Congress for the St. Petersburg Times, a Florida newspaper that is now called the Tampa Bay Times. I had grown tired of hearing politicians’ false claims and felt I had been complicit by publishing them in my news articles without scrutiny. The internet offered a new way for us to hold politicians accountable for what they said. I went to my editors with a crazy idea: instead of having me cover the 2008 campaign like all the other political journalists, how about if I started a fact-checking website?
Sure, they said.
In fact, they liked the crazy idea so much, they let me start a team with some of the most talented reporters and editors at the paper. They also let us break the rules. We built our own content management system and took some bold steps that most newspaper editors would never allow. We invented the Truth-O-Meter, which rated politicians’ claims from True to “Pants on Fire.” It made substantive articles about policy accessible to a wider audience.
We created a unique form of journalism. Instead of publishing traditional articles, we published fact-checks of politicians’ statements in a new structured form that could be collected on report card pages and tallied to tell people how many true, half true or Pants on Fire statements a politician had earned. PolitiFact was truly new journalism in the internet age.
A key to PolitiFact’s success was the culture of my newspaper. My editors were not only willing to let me try my unusual idea, they encouraged that kind of creative approach in everything we did. They also had a deep commitment to accountability reporting. They believed – they still believe – that holding power accountable is one of the fundamental missions of journalism.
PolitiFact became part of a growing community that included FactCheck.org and the Washington Post Fact-Checker in the United States, Full Fact in Britain and Chequeado in Argentina.
The digital revolution made it all possible. I remember those early years as an exciting time filled with promise. It was the honeymoon for journalism on the internet, as reporters and editors around the world discovered we could use the web to do powerful, important things.
News stories became interactive, enabling readers to engage with content. Data could be presented as vivid graphics that made numbers come alive. And design was transformed: I remember when we first saw Snowfall, the New York Times story about an avalanche, and we saw how the web could be used for powerful storytelling.
Then came smartphones, which enabled our readers and viewers to get the news all the time, wherever they were, and use thousands of apps to read articles and watch videos. It seemed like the future was infinitely bright and there were lots more great things ahead.
But the honeymoon ended.
The internet got loud and crowded. Partisan voices began to dominate the discussions and people began shouting at each other in ALL CAPITAL LETTERS. Twitter became a thing, but we realized that some of the “people” on Twitter weren’t really human — they were bots. And they were programmed to pump up the partisan propaganda to drive apart the real people.
Partisans took advantage of the internet to build digital fortresses where they could isolate themselves from opinions they disliked. They hid behind the walls and lobbed attacks against their enemies. These fortresses are home to more propaganda than discourse and they provide refuge for extremists. If you’re inside one, your side is always right.
The iPhone had changed the landscape when it was introduced in 2007. It created a whole a new platform for apps, which offered promising new ways that people could connect. I remember when a friend in Chile showed me WhatsApp and how he used it to communicate with his friends and family.
But it didn’t take long for people who want to spread misinformation to discover they could use WhatsApp without getting much scrutiny from journalists. I know that here in Colombia, you saw lots of misinformation spread through WhatsApp about the peace deal in the 2016 election.
Pablo Medina Uribe, the editor of the fact-checking site Colombia Check, reminded me recently how WhatsApp is a fertile ground for falsehoods. People are more likely to believe WhatsApp messages because they’re sent by people they trust. But the nature of many mobile data plans here give people unlimited data on WhatsApp and Facebook but not for their internet browser or other apps.
So, on a broad scale, what can we do? More specifically, what can journalists and the tech community do?
We need to find new ways to harness technology to get accurate information to people when they need it. We need to be as aggressive and cunning as the people and groups who are spreading the misinformation. And we need to change our thinking.
First, the technology part. At Duke University, we’ve launched a project called the Tech & Check Cooperative that has an ambitious goal: to use automation to monitor politicians’ speeches and debates and provide live fact-checking sourced from existing fact-checks.
Five or six years ago, I thought this kind of automated fact-checking was a long time away. But advances in technology and the dividends from a partnership we started with Google have created remarkable momentum. So we have already created the first fact-checking app for the Amazon Echo called Share the Facts. It lets you query Alexa and get an instant fact-check.
It’s impressive: you can ask Alexa a question and, if the fact-checkers have published something on it, she will reply telling you that the Washington Post or FactCheck.org or PolitiFact checked it and what they concluded.
We are now building a similar product for TV and the web. It’s a much harder product to develop than the Amazon Echo, but the idea is that when our app hears someone say a statement that fact-checkers have examined, the app will pop up a related fact-check right on the screen of your smartphone or TV.
We’ve made significant progress in the last six months. Over the summer, our students created a rough framework for our app that converts a live speech to text, then filters out sentences that aren’t checkable using our ClaimBuster tool, and then uses an algorithm to look for matches from our database of previously published articles. We are still some months away from a finished product, but we are getting closer every day.
We also just completed the first user testing of instant fact-checking on TV. We had people watch specially modified videos of State of the Union speeches that had pop-up fact-checks. The viewers had helpful feedback for us about what they wanted on the screen, and they were unanimous about the concept: They all want real-time fact-checking on their TV.
And you don’t need an army of computer science students to create something valuable. We need more projects like the “lie detector” developed by La Silla Vacia here in Colombia, which pioneered fact-checking on WhatsApp. People send a screenshot of a message they would like checked and then the journalists check it and encourage people to share it back through WhatsApp.
But regardless of whether we’re building big projects for a television screen, or smaller ones for WhatsApp, it requires that we think differently about the role of a journalist.
Most political reporters – and nearly all fact-checkers – have traditionally thought of themselves as neutral players in the political discourse. We published the information in the daily paper and left it on your doorstep. Or we just put it on our website or on the nightly news.
We were passive. We left it up to you to seek out the information that you needed and draw any connections or conclusions about the information on your own. I’ve been asked many times if it bothered me that politicians kept lying after we fact-checked them, and I would say, “Our job is just to provide the information.”
That strategy worked fine 10 years ago. But now, politicians and propagandists have learned how to spread misinformation at light speed. We can no longer sit back and wait for readers to come to us. We must become more aggressive and take the facts to the people. We need to disrupt the lies.
We can no longer be passive when our readers and viewers are being swamped with misinformation. We need to be more energetic and inventive in getting the information to people at the moment they first hear the claim.
Some of these solutions will be high-tech, like the automated fact-checking apps we’re building at Duke. Others can take more of a grass-roots approach, like the WhatsApp “lie detector” developed by La Silla Vacia.
But we need all kinds of these efforts, big and small, simple and complex, to adapt how we provide information in these fast-changing times. We need to build new apps for your phone and create new ways to provide the facts while you’re watching a speech on TV. And while print is still around, we should put the truth in ink and paper.
Yes, the honeymoon for the internet is over and sometimes it seems like we’re in a dark time. But I’m encouraged by the progress we’re making. I see promising efforts all around the world. And through it all I see great spirit and creativity.
The mission of the journalist remains the same: to give people the vital information they need to make sense of their world and hold their government accountable. We’ll continue to do that. We’ll just do it in new and creative ways.
Duke students tackle big challenges in automated fact-checking
Trio assembled promising building blocks needed for live fact-checking
By Catherine Clabby - October 8, 2018
Three Duke computer science majors advanced the quest for what some computer scientists say is the Holy Grail in fact-checking this summer.
Caroline Wang, Ethan Holland and Lucas Fagan tackled major challenges to creating an automated system that can both detect factual claims while politicians speak and instantly provide fact-checks.
That required finding and customizing state-of-art computing tools that most journalists would not recognize. A collective fondness for that sort of challenge helped, a lot.
“We had a lot of fun discussing all the different algorithms out there, and just learning what machine learning techniques had been applied to natural language processing,” said Wang, a junior also majoring in math.
Wang and her partners took on the assignment for a Data+ research project. Part of the Information Initiative at Duke, Data+ invites students and faculty to find data-driven solutions to research challenges confronting scholars on campus.
The fact-checking team convened in a Gross Hall conference from 9 am to 4 pm every weekday for 10 weeks to help each other figure out how to help achieve live fact-checking, a goal of Knight journalism professor Bill Adair and other practitioners of accountability journalism.
Their goal was to do something of a “rough cut” of end-to-end automated fact-checking: to convert a political speech to text, identify the most “checkable” sentences in the speech and then match them with previously published fact-checks.
The students concluded that Google Cloud Speech-to-Text API was the best available tool to automate audio transcriptions. They then submitted the sentences to ClaimBuster, a project at the University of Texas at Arlington that the Duke Tech & Check Cooperative uses to identify statements that merit fact-checking. ClaimBuster acted as a helpful filter that reduced the number of claims submitted to the database, which in turn reduced processing time.
They chose Google Cloud speech-to-text because it can infer where punctuation belongs, Holland said. That yields text divided into complete thoughts. Google speech-to-text also shares transcription results while it processes the audio, rather than waiting until translation is done. That speeds up how fast the new text can get moved to the next steps along a fact-checking pipeline.
“Google will say: This is my current take and this is my current confidence that take is right. That lets you cut down on the lag,” said Holland, a junior whose second major is statistics.
Their next step was finding ways to match the claims from that speech with the database of fact-checks that came from the Lab’s Share the Facts project. (The database contains thousands of articles published by the Washington Post, FactCheck.org and PolitiFact, each checking an individual claim.)
To do that, the students adapted an algorithm that the open-source research outfit OpenAI released in June, after the students started working together. The algorithm builds on The Transformer, a new neural network computing architecture that Google researchers published just six months prior.
The architecture alters how computers organize trying to understand written language. Instead of translating a sentence word by word, The Transformer weighs the importance of each word to the meaning of every other word. Over time that system helps machines discern meaning in more and more sentences more quickly.
“It’s a lot more like learning English. You grow up hearing it and your learn it,” said Fagan, a sophomore also majoring in math.
Work by Wang, Holland and Fagan is expected to help jumpstart a Bass Connections fact-checking team that started this fall. Students on that team will continue the hunt for better strategies to find statements that are good fact-check candidates, produce pop-up fact-checks and create apps to deliver this accountability journalism to more people.
Tech & Check has $1.2 million in funding from the John S. and James L. Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation to tackle that job.
FactStream app now shows latest fact-checks from Post, FactCheck.org and PolitiFact
New version features alerts for Pants on Fire and Four Pinocchio ratings
By Bill Adair - October 7, 2018
FactStream, our iPhone/iPad app, has a new feature that displays the latest fact-checks from FactCheck.org, PolitiFact and The Washington Post.
FactStream was conceived as an app for live fact-checking during debates and speeches. (We had a successful beta test during the State of the Union address in January.) But our new “daily stream” makes the app valuable every day. You can check it often to get summaries of the newest fact-checks and then click through to the full articles.
By viewing the work of the nation’s three largest fact-checkers in the same stream, you can spot trends, such as which statements and subjects are getting checked, or which politicians and organizations are getting their facts right or wrong.
The new version of the app includes custom notifications so users can get alerts for every new fact-check or every “worst” rating, such as Four Pinocchios from Washington Post Fact Checker Glenn Kessler, a False from FactCheck.org or a False or Pants on Fire from PolitiFact.
The new daily stream was suggested by Eugene Kiely, the director of FactCheck.org. The app was built by our lead technologist Christopher Guess and the Durham, N.C., design firm Registered Creative. It gets the fact-check summaries from ClaimReview, our partnership with Google that has created a global tagging system for fact-checking. We plan to expand the daily stream to include other fact-checkers in the future.
The app also allows users to search the latest fact-checks by the name of the person or group making the statement, by subject or keyword.
FactStream is part of the Duke Tech & Check Cooperative, a $1.2 million project to automate fact-checking supported by Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation.
FactStream is available as a free download from the App Store.
Spreading the word on The NC Fact-Checking Project
Journalists from the Reporters' Lab and the News & Observer appeared on Spectrum News
By Catherine Clabby - August 16, 2018
Cathy Clabby of The Reporters’ Lab and Andy Specht of The News & Observer appeared on Spectrum Cable’s “Politics Tonight” this week to explain the newly announced NC Fact-Checking Project. The journalists briefed Spectrum News host Tim 
Boyum on the ambitious plan to fact-check claims by politicians statewide during federal and state campaigns this fall and into the 2019 General Assembly session.
In the project, Duke journalism students and faculty – assisted by Tech & Check bots – will scour campaign messaging and news reports to find newsworthy claims by politicians that can be verified. Journalists at The News & Observer will select statements to check, report their veracity and craft the fact-checks. The reporting will be shared for free with print, broadcast and digital newsrooms statewide. To watch, click here.
Reporters’ Lab joins N&O, UNC Reese News Lab on major fact-checking project
With grant from the N.C. Local News Lab Fund, partnership will expand non-partisan fact-checking throughout the state
By Catherine Clabby - August 1, 2018
The Duke Reporters’ Lab is joining McClatchy Carolinas and the UNC Reese News Lab in an ambitious project to expand non-partisan fact-checking throughout North Carolina.
With a $50,000 grant from the North Carolina Local News Lab Fund, the North Carolina Fact-Checking Project will build on the existing work at The News & Observer, add the work of student journalists and take advantage of new automated tools from the Duke Tech & Check Cooperative.
The project will evaluate statements by state and federal candidates in the 2018 election as well as lawmakers in the General Assembly session that begins in January 2019. The fact-checks will be produced by N&O journalists as part of PolitiFact North Carolina and will be made available for free to any news organization in the state for use online and in print.
The project will get support from the new TruthBuzz program of the International Center for Journalists (ICFJ), which is hiring an engagement fellow based on Raleigh to promote the North Carolina fact-checking.
The North Carolina Fact-Checking Project will put special emphasis on claims by politicians in rural parts of the state. Students in the Reporters’ Lab will scour news coverage and campaign ads for factual claims made by state, local and congressional candidates. The Lab will build new versions of its Tech & Check Alerts that use automated bots to find statements by politicians in social media that could be of interest to the North Carolina fact-checkers.
The Duke students and bots will provide daily suggestions of possible claims to The News & Observer, which will select which statements to research.
The UNC Reese News Lab will co-host a student seminar on fact-checking and help select a student journalist to work on the project. Representatives from Duke, TruthBuzz and the News & Observer will hold outreach sessions around the state to promote fact-checking and encourage news organizations to publish the project’s work.
About the partners:
The North Carolina Local News Lab Fund is a collaborative fund at the North Carolina Community Foundation established by a group of local and national funders who believe in the power of local journalism, local stories, and local people to strengthen our democracy.
The Duke Reporters’ Lab at the Sanford School of Public Policy is a center of research on fact-checking and automated journalism. The Lab tracks the growth of fact-checking around the world, conducts studies on important topics and develops tools to help journalists.
McClatchy Carolinas is the McClatchy division that publishes three newspapers in North Carolina, The News & Observer, The Charlotte Observer and The Herald-Sun. The N&O has a strong team of political reporters and has been the state’s PolitiFact partner for the last two years.
The Reese News Lab is an experimental media and research project based at the School of Media and Journalism at the University of North Carolina-Chapel Hill.
At Global Fact V: A celebration of community
More than 200 people attended the fifth meeting of the world's fact-checkers in Rome, which was organized by the International Fact-Checking Network.
By Bill Adair - June 25, 2018
My opening remarks at Global Fact V, the fifth annual meeting of the world’s fact-checkers, organized by the International Fact-Checking Network, held June 20-22 in Rome.
A couple of weeks ago, a photo from our first Global Fact showed up in my Facebook feed. Many of you will remember it: we had been all crammed into a classroom at the London School of Economics. When we went outside for a group photo, there were about 50 of us.
To show how our conference has grown, I posted that photo on Twitter along with one from our 2016 conference that had almost twice as many people. I also posted a third photo that showed thousands of people gathered in front of the Vatican. I said that was our projected crowd for this conference.
I rate that photo Mostly True.
What all of our conferences have in common is that they are really about community. It all began in that tiny classroom at the London School of Economics when we realized that whether we were from Italy or the U.K. or Egypt, we were all in this together. We discovered that even though we hadn’t talked much before or in many cases even met, we were facing the same challenges — fundraising and finding an audience and overcoming partisanship.
It was also a really powerful experience because we got a sense of how some fact-checkers around the world were struggling under difficult circumstances — under governments that provide little transparency, or, much worse, governments that oppress journalists and are hostile toward fact-checkers.
Throughout that first London conference there was an incredible sense of community. We’d never met before, but in just a couple of days we formed strong bonds. We vowed to keep in touch and keep talking and help each other.
It was an incredibly powerful experience for me. I was at a point in my career where I was trying to sort out what I would do in my new position in academia. I came back inspired and decided to start an association of fact-checkers – and hold these meetings every year.
The next year we started the IFCN and Poynter generously agreed to be its home. And then we hired Alexios as the leader.
Since then, there are have been two common themes. One you hear so often that it’s become my mantra: Fact-checking keeps growing. Our latest census of fact-checking in the Reporters’ Lab shows 149 active fact-checking projects and I’m glad to see that number keep going up and up.
The other theme, as I noted earlier, is community. I thought I’d focus this morning on a few examples.
Let’s start with Mexico, where more than 60 publishers, universities and civil society organizations have started Verificado 2018, a remarkable collaboration. It was originally focused largely on false news, but they’ve put more emphasis on fact-checking because of public demand. Daniel Funke wrote a great piece last week about how they checked a presidential debate.
In Norway, an extraordinary team of rivals has come together to create Faktisk, which is Norwegian for “actually” and “factually.” It launched nearly a year ago with four of the country’s biggest news organizations — VG, Dagbladet, NRK and TV 2 – and it’s grown since then. My colleague Mark Stencel likened it to the New York Times, The Washington Post and PBS launching a fact-checking project together.
At Duke, both of our big projects are possible because of the fact-checkers’ commitment to help each other. The first, Share the Facts and the creation of the ClaimReview schema, grew out of an idea from Glenn Kessler, the Washington Post Fact Checker, who suggested that Google put “fact-check” tags on search results.
That idea became our Duke-Google-Schema.org collaboration that created what many of you now use so search engines can find your work. And one unintended consequence: it makes automated fact-checking more possible. It all started because of one fact-checker’s sense of community.
Also, FactStream, the new app of our Tech & Check Cooperative, has been a remarkable collaboration between the big US fact-checkers — the Post, FactCheck.org and PolitiFact. All three took part in the beta test of the first version, our live coverage of the State of the Union address back in January. Getting them together on the same app was pretty remarkable. But our new version of the app –which we’re releasing this week – is even cooler. It’s like collaboration squared, or collaboration to the second power!
It took Glenn’s idea, which created the Share the Facts widget, and combined it with an idea from Eugene Kiely, the head of FactCheck.org, who said we should create a new feature on FactStream that shows the latest U.S. widgets every day.
So that’s what we did. And you know what: it’s a great new feature that reveals new things about our political discourse. Every day, it shows the latest fact-checks in a constant stream and users can click through, driving new traffic to the fact-checking sites. I’ll talk more about it during the automated demo session on Friday. But it wouldn’t be possible if it weren’t for the commitment to collaboration and community by Glenn and Eugene.
We’ve got a busy few days ahead, so let’s get on with it. There sure are a lot of you!
As we know from the photographs: fact-checking keeps growing.
Meet the bot builders: How our student team is automating fact-checkers’ work
A team of Duke students is building tools that automate the most tedious task for fact-checkers: finding claims to check
By Julianna Rennie - April 30, 2018
In a sunny corner office at Duke University, four students are building bots to do tasks that are too tedious for humans.
The project is part of the Duke Tech & Check Cooperative, a $1.2 million project to automate fact-checking. The students spend up to 10 hours each week in the Reporters’ Lab, a room in the Sanford School of Public Policy decorated with movie posters from All the President’s Men, Spotlight and The Post.
The bots are computer programs that perform the tasks often done by college interns. The programs scour long transcripts and articles online and identify sentences that journalists might want to fact-check.
The students are an eclectic bunch: a data enthusiast who guzzles nutritional drinks; a Cameron Crazy who is spending his summer solving computer science problems; a Silicon Valley resident who writes code to help animals; and a snowboarder who helps run student businesses.
Lucas Fagan, the puzzler
Lucas Fagan is a political junkie. He joined the Lab so he could have a role in fact-checking and debunking fake news, which he says are critical in politics today.
Fagan, a first-year from Morristown, New Jersey, is building a bot to identify checkable claims from Facebook. The program will gather content from posts written by politicians in close races.
Fagan is considering majoring in computer science and mathematics. He writes for Duke Political Review and competes with the debate team. When he’s in the Lab, he enjoys playing devil’s advocate with other staff members about anything from Duke basketball to the Russia probe.
Though he enjoys interacting with other Lab students, Fagan finds he is most productive in his dorm, where he has multiple monitors set up for coding. “I honestly enjoy the work that we’re doing, so when I need a break from homework, I’ll do work for the Lab,” he says.
Fagan feeds off the problem-solving involved in coding. “I enjoy trying to face the CS challenges more than anything else,” he says. That’s why he’s staying in Durham this summer for Data and Technology for Fact-Checking, a 10-week research program through which he will tackle natural language processing and machine learning problems.
Helena Merk, coding for causes
Helena Merk competes in hack-a-thons across the country, so she has ample experience designing creative tech projects. Though coding has many applications, she chooses to apply her skills to social causes.
Merk, a first-year from Palo Alto, California, is writing a program that would enable the Lab to send its daily list of checkable claims through Slack, a messaging tool utilized by newsrooms.
This year, Merk helped organize Duke Blueprint, a conference aiming “to inspire disruptive innovation for future-focused global change.” She also works remotely as the lead mobile app developer for AdoptMeApp, an app that connects shelter dogs with potential owners.
Merk applied to the Pratt School of Engineering, planning to study biomedical engineering. But in her first semester she realized there were other disciplines at Duke that could combine technology and health.
Now, Merk is taking computer science and global health classes. She’s spending most of the summer in Madagascar working with Duke engineers to provide water systems for local communities. “Computer science is powerful in how versatile it is,” she says. “I want to make things that help people.”
Naman Agarwal, the entrepreneur
Naman Agarwal started coding in high school while he was interning for a local politician’s campaign. After spending weeks entering donor information into a spreadsheet, he built an app to automate the process.
Agarwal, a first-year from Palatine, Illinois, is now building a bot for Tech & Check to identify checkable tweets from politicians in competitive races.
Agarwal works at Campus Enterprises, a student-run LLC that provides food delivery, custom apparel and other services to Duke students. He is also a drummer and producer for Duke’s student-run record label, Small Town Records. On the weekends, he travels to snowboarding competitions with the Duke Ski Team.
Agarwal studies computer science and economics. He says he hopes to find a job that emulates his experience at the Lab. “I don’t want to contribute to work that’s just throwing money around,” he says. “I want to work at a company that has a soul.” (Photos by Evan Nicole Bell)
Tech & Check Alerts aim to ease the workload of fact-checkers
Student-created tool can peruse political transcripts and find claims most likely to contain falsehoods
By Sydney McKinney - April 6, 2018
Students in the Duke Reporters’ Lab have built a bot that is like an intern who watches TV around the clock.
Asa Royal, a junior at Duke University, and Lucas Fagan, a freshman, have created Tech & Check Alerts, a new tool in a series of innovations the Reporters’ Lab is creating to help simplify the fact-checking process.
Using Tech & Check Alerts, the Lab can identify check-worthy claims in television news transcripts and send them to fact-checkers in daily email alerts.
“We’re going to save fact-checkers a lot of time and help them find things that they would otherwise miss,” said Mark Stencel, co-director of the Reporters’ Lab.
Though the fact-checking industry is growing worldwide, the organizations doing that work are typically small, even one-person enterprises, and the workload can be burdensome. Fact-checkers often have to sift through pages of text to find claims to check. This time-consuming process can create a substantial time gap between when statements are made and when fact-checks are available to viewers or readers.
The Tech & Check Alerts automate that process. Royal and Fagan, who are both computer science majors, created a program that scans transcripts of TV news channels, such as CNN, for claims that fact-checkers may want to investigate. It then compiles the check-worthy claims and sends them in a daily email to fact-checkers at The Washington Post, PolitiFact, the Associated Press, FactCheck.org and The New York Times, among others. Thus far, there have been seven fact-checks performed based on these alerts.
“Journalists don’t have to watch 15 hours of CNN or read the entire congressional report,” Royal said. “We’ll do it for them.”
Royal and Fagan created Tech & Check Alerts using ClaimBuster, an algorithm created by computer scientist Chengkai Li from the University of Texas at Arlington. ClaimBuster scans blocks of text and identifies “check-worthy” claims, based on indicators such as past-tense verbs, numbers, dates or statistics. It ranks statements from 0 to 1.0 based on how likely they are to be checkable; any statements that score a 0.7 or higher are typically considered check-worthy.
According to Royal, Li’s technology had yet to be used much outside of academia, so leaders of the Tech & Check Cooperative decided to utilize it for daily alerts.
“There’s already software that can find factual claims, and there are already fact-checkers who can check them,” Royal said. “We’re just solving the last-mile problem.”
The creation of Tech & Check Alerts is an important step for the Duke Tech & Check Cooperative, a two-year research project funded by the John S. and James L. Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation.
The broader purpose of this initiative is to bring together journalists, academics and computer scientists from across the country to innovate and automate the fact-checking industry. Over the course of two years, the Reporters’ Lab will develop tools that ease the job of fact-checkers and make fact-checking more accessible to consumers. Another tool the Lab is currently working on is FactStream, an app that provides instant fact-checking during live events.
Alongside other student researchers, Fagan and Royal are working to improve Tech & Check Alerts to include additional sources such as daily floor speeches and debates from the Congressional Record, and social media feeds from endangered incumbents running in this year’s closest House and Senate races. Fact-checkers will have input on how these additional alerts will be deployed.
Fagan is also building a web interface that would give fact-checking partners a way to dig deeper into these feeds and perhaps even customize certain alerts. Freshman Helena Merk, another student researcher in the Lab, is building a tool that would deliver the daily alerts directly to a channel on Slack, a communication platform used in many newsrooms.
Once these improvements are completed, and Tech & Check Alerts are deployed more widely, they should help fact-checkers across the country.
“This project is a stepping stone in our process of using real-time claims and existing fact-checks to automate fact-checking in real time,” Stencel said.
Journalists, computer scientists gather for Tech & Check Conference at Duke
Members of the fact-checking community convened March 29-30 on Duke University's campus to tackle pressing issues
By Rebecca Iannucci - March 30, 2018
About 40 fact-checkers, journalists, computer scientists and academics gathered at Duke University March 29-30 for the Tech & Check Conference, a meeting hosted by the Reporters’ Lab.
As part of its Tech & Check Cooperative, the Reporters’ Lab is serving as a hub for automated fact-checking to connect journalists and technologists around the world. The conference gave them an opportunity to demonstrate current projects and discuss the big challenges of automation.
Some highlights of the conference:
* Eleven demos of past and current projects. Technologists and computer scientists showed off projects they’ve been developing to either automate fact-checking or improve the flow of accurate information on the internet.
Topics included new tools such as Chequeabot, an automated service that detects factual claims for the Argentinian fact-checker Chequeado; the Bad Idea Factory’s update of the Truth Goggles tool; and the perils of misinformation, including a real-life example from Penn State professor S. Shyam Sundar, whose research project about fake news was inaccurately described in widespread news coverage.
* Two Q&A panels. Alexios Mantzarlis, director of the International Fact-Checking Network, led a discussion with three fact-checkers about the potential tools and processes that could make fact-checking more efficient in the future.
Reporters’ Lab co-director Bill Adair moderated a conversation about challenges in automated fact-checking, including the pitfalls of voice-to-text technology and natural language processing.
Attendees also participated in breakout sessions to discuss ways to develop international standards and consistent terminology.
Photos by Colin Huth.
Pop-up fact-checking app Truth Goggles aims to challenge readers’ biases
Created by Dan Schultz, Truth Goggles is a browser plugin that creates a personalized "credibility layer" for users
By Julianna Rennie - February 21, 2018
Dan Schultz, a technologist building a new fact-checking app for the Reporters’ Lab, says the app should be like a drinking buddy.
“You can have a friend who you fundamentally disagree with on a lot of things, but are able to have a conversation,” Schultz says. “You’re not thinking of the other person as a spiteful jerk who’s trying to manipulate you.”
Schultz, 31, is using that approach to develop a new version of Truth Goggles, an app he first built eight years ago at the MIT Media Lab, for the Duke Tech & Check Cooperative. His goal is to get to know users and find the most effective way to show them fact-checks. While other Tech & Check apps take a traditional approach by providing Truth-O-Meter ratings or Pinocchios to all users, Schultz plans to experiment with customized formats. He hopes that personalizing the interface will attract new audiences who are put off by fact-checkers’ rating systems.
Truth Goggles is a browser plugin that automatically scans a page for content that users might want fact-checked. Schultz hopes that this unique “credibility layer” will be like a gentle nudge to get people to consider fact-checks.
“The goal is to help people think more carefully and ideally walk away with a more accurate worldview from their informational experiences,” he says.
As a graduate student at the Media Lab, Schultz examined how people interact with media. His 150-page thesis paper concluded that when people are consuming information, they are protecting their identities.
Schultz learned that a range of biases make people less likely to change their minds when exposed to new information. Most people simply are unaware of how to consume online content responsibly, he says.
He then set out to use technology to short-circuit biased behavior and help people critically engage with media. The first prototype of Truth Goggles used fact-checks from PolitiFact as a data source to screen questionable claims.
“The world will fall apart if we don’t improve the way information is consumed through technology.”
Schultz recently partnered with the Reporters’ Lab to resume working on Truth Goggles. This time, Truth Goggles will be integrated with Share the Facts, so it can access all fact-checking articles formatted using the ClaimReview schema.
Schultz also is exploring creative ways to present the information to users. He says the interface must be effective in impeding biases and enjoyable for people to use. As a graduate student, one of Schultz’s initial ideas was to highlight verified claims in green and falsehoods in red. But he quickly realized this solution was not nuanced enough.
“I don’t want people to believe something’s true because it’s green,” he says.
The new version of Truth Goggles will use information about users’ biases to craft messages that won’t trigger their defenses. But Schultz doesn’t know exactly what this will look like yet.
“Can we use interfaces to have a reader challenge their beliefs in ways that just a blunt presentation of information wouldn’t?” Schultz says. “If the medium is the message, how can we shape the way that message is received?”
Born in Cheltenham, Pennsylvania, Schultz studied information systems, computer science and math at Carnegie Mellon University. As a sophomore, he won the Knight News Challenge, which provides grants for “breakthrough ideas in news and information.”
The News Challenge put him “on the path toward eventually applying to the Media Lab and really digging in,” he says.
After graduating from MIT, Schultz worked as a Knight-Mozilla Fellow at the Boston Globe and then joined the Internet Archive, where his title is senior creative technologist. He continues to develop side projects such as Truth Goggles through the Bad Idea Factory, a company with a tongue-in-cheek name that he started with friends. He says the company’s goal is “to make people ‘thinking face’ emoji” by encouraging its technologists to try out creative ideas. With Truth Goggles, he hopes to get people who may not already consume fact-checking content to challenge their own biases.
“The world will fall apart if we don’t improve the way information is consumed through technology,” Schultz says. “It’s sort of like the future of the universe as we know it depends on solving some of these problems.”