Talking Point Tracker: A project to spot hot topics as they flare up on TV news

Developers will debut tracker prototype at 2019 Tech & Check Conference.

By Sanha Lim – March 13, 2019 | Print this article

When fact-checking technologists and journalists gather in Durham for the 2019 Tech & Check Conference this month, they will share new tools intended to optimize and automate fact-checking.

Dan Schultz
Dan Schultz of the Bad Idea Factory is preparing to debut a version of Talking Point Tracker.

For Dan Schultz, one founder of the Bad Idea Factory software development collective, this will be a chance to debut a “mannequin” version of the Talking Point Tracker. Created in collaboration with the Duke Tech & Check Cooperative, the tracker is intended to “capture the zeitgeist” of television news by identifying trending topics.

Duke journalism professor Bill Adair, who runs Tech & Check, launched the project by asking Schultz how fact-checkers could capture hot topics on TV news as quickly as possible. That is a simple but powerful idea. TV news is a place of vast discourse, where millions of viewers watch traditional, nonpartisan newscasts and partisan broadcasters such as Sean Hannity and Rachel Maddow. Listening in would give insight into what Schultz calls a “driver or predictor of collective consciousness.”

But executing even simple ideas can be difficult. In this case, TV news programs broadcast dense flows of media: audio, video, text and images that are not simple to track. Luckily, network and cable news outlets produce closed-caption subtitles for news shows. Talking Pointer Tracker scans those subtitles to identify keywords used most frequently within blocks of time. It also puts the keywords in context by showing sentences and longer passages where the keywords were found. To deepen the context, the tracker shows related keywords that often appear with the trending words.

The eventual goal is to group keywords into clusters that better capture emerging conversations. “Our hope is that it will be a useful tool for journalists who want to write in the context of what’s being discussed,“ said Schultz, who is collaborating with Justin Reese, a front-end developer with the Bad Idea Factory, on the project.

More technically, Talking Point Tracker runs closed-caption transcripts through a natural language processing pipeline that cleans the text as well as it can. An application programming interface, an API, uses separate language processing algorithm to find the most common keywords. These are “named entities” — usually proper nouns that can be sorted into different categories such as places, organizations and locations.

Talking Point Tracker’s prototype, to be unveiled at Tech & Check, is dense with information. But the design Reese created for viewing on a computer screen makes it readable. There’s enough white space to be easy on the eyes and a color scheme of red, blue, black and yellow that organizes text.

Talking Point Tracker packs lots of data on the current version of its screen display.

A list of the most frequent keywords over a specified time period are listed in a column on the left. Next to that is a line graph highlights their frequency. Sentences from which the keywords are listed on the right. If you click there, the tool points you to longer passages of transcripts. On the bottom are related keywords that often appear in the same sentences as a given word.

Moving from a mannequin stage to a living stage for this project will be challenging, Schultz said. As much as natural language processing has evolved over the past decade, algorithms still have trouble understanding aspects of human language. One free, open-source system the Tracker relies on is an API called spaCy. But programs like spaCy don’t always recognize the same thing when they’re stated differently — say, the “Virginia legislature” and the “Virginia General Assembly.”

Another challenge is coping with the quality of news show transcripts, Schultz said. The transcripts can contain many typos, in addition to sometimes being either all caps or all lowercase, which the API can have trouble reading.

Talking Point Tracker’s logo

And the API doesn’t always know where sentences break. Too often, the system will return sentences that contain just “Mr.” because it concludes that a period signifies the end of the sentence. To get around this, Schultz is using another NLP technology to clean the transcripts he obtains.

To prepare for the Tech & Check Conference, Schultz is building better searching tools and further cleaning up the Tracker’s design. “It’s always good to have your feet close to the fire,” Schultz said.

The biggest question he hopes to get answered before leaving is whether Talking Point Tracker could be useful for journalists, he said.

“There’s a lot things we can gain from feedback. If we have the capacity and interest from whoever, we will continue to iterate and build on top of that,” Schultz said.

 

         

 

Back to top

FactStream

During State of the Union, a failure…and a glimpse of Squash

Our app crashed under unusual traffic. But our tests of a new automated tool were a success.

By Bill Adair – February 6, 2019 | Print this article

We tested two fact-checking products during the State of the Union address. One failed, the other showed great promise.

The failure was FactStream, our iPhone app. It worked fine for the first 10 minutes of the speech. Users received two timely “quick takes” from Washington Post Fact Checker Glenn Kessler, but then the app crashed under an unusual surge of heavy traffic that we’re still investigating. We never recovered.

The other product is a previously secret project we’ve code-named Squash. It’s our first attempt at fully automated fact-checking. It converts speech to text and then searches our database of fact-checks from the Post, FactCheck.org and PolitiFact. When it finds a match, a summary of the fact-check pops onto the screen.

We’ve been testing Squash for the last few weeks with mixed results. Sometimes it finds exactly the right fact-checks. Other times the results are hilariously bad. But that’s what progress looks like.

A screenshot of Squash, our fully automated fact-checking tool, in the live test.

We went into last night’s speech with very modest expectations. I said before the speech I’d be happy if the speech simply triggered fact-checks to pop up, even if it was a poor match.

But Squash actually performed pretty well. It had 20 pop-ups and six of them were in the ballpark.

Overall, the results were stunning. It gave us a glimpse of how good automated fact-checking can be.

We’ll have more to share once we’ve reviewed the results, so stay tuned.

As for FactStream, it now has lots of timely fact-checks from the State of the Union on the main home screen, which continues to function well. We will fix any problems we identify with the live event feature and plan to be back in action for real-time coverage for campaign events later this year.

Back to top

FactStream

Live fact-checking of the State of the Union address with our FactStream app

We're partnering with FactCheck.org, PolitiFact and Washington Post Fact Checker Glenn Kessler to provide real-time checks.

By Bill Adair – February 3, 2019 | Print this article

UPDATE, Feb. 5, 11 p.m.: Our FactStream app failed during the State of the Union address. We apologize for the problems. We are still sorting out what happened, but it appears we got hit with an unexpected surge of traffic that overwhelmed our servers and our architecture.

As we noted at the bottom of this post, this was a test – only our second of the app.  We’ll fix the problems and be better next time.

—————-

The Reporters’ Lab is teaming up with the Washington Post, PolitiFact and FactCheck.org to offer live fact-checking of the State of the Union address on Tuesday night on our new FactStream app.

FactStreamJournalists from the Post, PolitiFact, and FactCheck.org will provide real-time updates throughout the speech in two forms:

Ratings – Links to previously published fact-checks with ratings when the president repeats a claim that has been checked before.

Quick takes – Instant updates about a statement’s accuracy. They will be labeled red, yellow and green to indicate their truthfulness.

Tuesday’s speech will be the second test of FactStream. The first test, conducted during last year’s State of the Union address, provided users with 32 updates. We got valuable feedback and have made several improvements to the app.

FactStream is part of the Duke Tech & Check Cooperative, a project to automate fact-checking that is funded by Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation. Additional support has been provided by Google.

FactStream is available for iPhone and iPad (sorry, no Android version yet!) and is a free download from the App Store.

The live event feature for the State of the Union address is marked by an icon of calendar with a check mark.

The app has two streams. One, shown by the home symbol in the lower left of the screen, provides a constant stream of the latest fact-checks published every day throughout the year. The live event feature for the State of the Union address is marked by an icon of a calendar with a check mark.

Because this is a test, users could encounter a few glitches. We’d love to hear about any bugs you encounter and get your feedback at team@sharethefacts.org.

Back to top

Tech & Check in the news

Coverage of Duke Tech & Check Cooperative 's efforts to strengthen journalism

By Catherine Clabby – December 14, 2018 | Print this article

It’s been more than a year since the Reporters’ Lab received $1.2 million in grant funding to launch the Duke Tech & Check Cooperative.

Our goal is to link computer scientists and journalists to better automate fact-checking and expand how many people see this vital, accountability reporting.

Here’s a sampling of some of the coverage about the range of projects we’re tackling:

Tech & Check:

Associated Press, Technology Near For Real-Time TV Political Fact-Checks

Digital Trends, Real-time fact-checking is coming to live TV. But will networks use it?

Nancy Watzman, Tech & Check: Automating Fact-Checking
Poynter, Automated fact-checking has come a long way. But it still faces significant challenges.
MediaShift, The Fact-Checking Army Waging War on Fake News

FactStream:
NiemanLab, The red couch experiments, early lessons in pop-up fact-checking.
WRAL, Fake news? App will help State of the Union viewers sort out fact, fiction
Media Shift, An Experiment in Live Fact-Checking the State of the Union Speech by Trump
American Press Institute, President Trump’s first State of the Union address is Tuesday night. Here’s how to prepare yourself, factually speaking.
WRAL, App will help views sort fact, fiction in State of the Union
NiemanLab, Automated, live fact-checks during the State of the Union? The Tech & Check Cooperative’s first beta test hopes to pull it off
NiemanLab, FactStream debuted live fact-checking with last night’s SOTU. How’d it go?

Tech & Check Alerts:
Poynter, This Washington Post fact check was chosen by a bot

Truth Goggles:
NiemanLab, Truth Goggles are back! And ready for the next era of fact-checking

And …
NiemanLab, So what is that, er, Trusted News Integrity Trust Project all about? A guide to the (many, similarly named) new efforts fighting for journalism
MediaShift, Fighting Fake News: Key Innovations in 2017 from Platforms, Universities and More NiemanLab, With $4.5 million, Knight is launching a new commission — and funding more new projects — to address declining public trust in media
Poynter, Knight’s new initiative to counter misinformation includes more than $1.3 million for fact-checking projects
Axios, How pro-trust initiatives are taking over the internet
Recode, Why the Craig behind Craigslist gave big bucks to a journalism program
Digital News Report (with Reuters and Oxford), Understanding the Promise and Limits of Automated Fact-Checking
Democratic Minority Staff Report, U.S. House Committee on Science, Space & Technology, Old Tactics, New Tools: A Review of Russia’s Soft Cyber Influence Operations

Back to top

Reporters’ Lab students are fact-checking North Carolina politicians

Student journalists and computer scientists find claims and report articles for the N.C. Fact-Checking Project

By Catherine Clabby – November 20, 2018 | Print this article

Duke Reporters’ Lab students expanded vital political journalism during a historic midterm campaign season this fall with the North Carolina Fact-Checking Project.

Five student journalists reviewed thousands of statements that hundreds of North Carolina candidates vying for state and federal offices made online and during public appearances. They collected newsy and checkable claims from what amounted to a firehose of political claims presented as fact.

Duke computer science undergraduates with the Duke Tech & Check Cooperative applied custom-made bots and the ClaimBuster algorithm to scrape and sort checkable political claims from hundreds of political Twitter feeds.

Editors and reporters then selected claims the students had logged for most of the project’s 30 plus  fact-checks and six summary articles that the News and Observer and PolitiFact North Carolina published between August and November.

Duke senior Bill McCarthy

Duke senior Bill McCarthy was part of the four-reporter team on the project, which the North Carolina Local News Lab Fund supported to expand local fact-checking during the 2018 midterms and beyond in a large, politically divided and politically active state.

“Publishing content in any which way is exciting when you know it has some value to voters, to democracy,” said McCarthy, who interned at PolitiFact in Washington, D.C. last summer. “It was especially exciting to get so many fact-checks published in so little time.”

Reporters found politicians and political groups often did not stick with the facts during a campaign election season that that fielded an unusually large number of candidates statewide and a surge in voter turnout.

The N.C. Fact-Checking Project produces nonpartisan journalism

NC GOP falsely ties dozens of Democrats to single-payer health care plan,” read one project fact-check headline. “Democrat falsely links newly-appointed Republican to health care bill,” noted another.  The fact-check “Ad misleads about NC governors opposing constitutional amendments” set the record straight about some Democratic-leaning claims about six proposed amendments to the state constitution.

And on and on.

Digging for the Truth

Work in the lab was painstaking. Five sophomores filled weekday shifts to scour hundreds of campaign websites, social media feeds, Facebook and Google political ads, televised debates, campaign mailers and whatever else they could put their eyes on. Often they recorded one politician’s attacks on an opponent that might, or might not, be true.

Students scanned political chatter from all over the state, tracking competitive state and congressional races most closely. The resulting journalism was news that people could use as they were assessing candidates for the General Assembly and U.S. Congress as well as six proposed amendments to the state constitution.

The Reporters’ Lab launched a mini news service to share each fact-checking article with hundreds of newsrooms across the state for free.

One of more than 30 N.C. Fact-Checking Project articles

The Charlotte Observer, a McClatchy newspaper like the N&O, published several checks. So did smaller publications such as Asheville’s Citizen-Times  and the Greensboro News and Record. Newsweek cited  a fact-check report by the N&O’s Rashaan Ayesh and Andy Specht about a fake photo of Justice Kavanaugh’s accuser, Christine Blasey Ford, shared by the chairman of the Cabarrus County GOP, which WRAL referenced in a roundup.

Project fact-checks influenced political discourse directly too. Candidates referred to project fact-checks in campaign messaging on social media and even in campaign ads. Democrat Dan McCready, who lost a close race against Republican Mark Marris in District 9, used project fact-checks in two campaign ads promoted on Facebook and in multiple posts on his Facebook campaign page, for instance.

While N&O reporter Andy Specht was reporting a deceptive ad from the Stop Deceptive Amendments political committee, the group announced plans to change it.

The fact-checking project will restart in January, when North Carolina’s reconfigured General Assembly opens its first 2019 session.

 

Back to top

Lessons learned from fact-checking 2018 midterm campaigns

After monitoring political messaging, students see the need for accountability journalism more than ever

By Catherine Clabby – November 20, 2018 | Print this article

Five Duke undergraduates monitored thousands of political claims this semester during a heated midterm campaign season for the N.C. Fact-Checking Project.

That work helped expand nonpartisan political coverage in a politically divided state with lots of contested races for state and federal seats this fall. The effort resumes in January when the project turns its attention to a newly configured North Carolina General Assembly.

Three student journalists who tackled this work with fellow sophomores Alex Johnson and Sydney McKinney reflect on what they’ve learned so far.

Lizzie Bond

Lizzie Bond: After spending the summer working in two congressional offices on Capitol Hill, I began my work in the Reporters’ Lab and on the N.C. Fact-Checking Project with first-hand knowledge of how carefully elected officials and their staff craft statements in press releases and on social media. This practice derives from a fear of distorting the meaning or connotation of their words. And in this social media age where so many outlets are available for sharing information and for people to consume it, this fear runs deep.

Yet, it took me discovering one candidate for my perspective to shift on the value of our work with the N.C. Fact-Checking Project. That candidate, Peter Boykin, proved to be a much more complicated figure than any other politician whose social media we monitored. The Republican running to represent Greensboro’s District 58 in the General Assembly, Boykin is the founder of “Gays for Trump,” a former online pornography actor, a Pro-Trump radio show host, and an already controversial, far-right online figure with tens of thousands of followers. Pouring through Boykin’s nearly dozen social media accounts, I came across everything from innocuous self-recorded music video covers to contentious content, like hostile characterizations of liberals and advocacy of conspiracy theories, like one regarding the Las Vegas mass shooting which he pushed with little to no corroborating evidence.

When contrasting Boykin’s posts on both his personal and campaign social media accounts with the more cautious and mild statements from other North Carolina candidates, I realized that catching untruthful claims has a more ambitious goal that simply detecting and reporting falsehoods. By reminding politicians that they should be accountable to the facts in the first place, fact-checking strives to improve their commitment to truth-telling. The push away from truth and decency in our politics and toward sharp antagonism and even alternate realities becomes normalized when Republican leaders support candidates like Boykin as simply another GOP candidate. The N.C. Fact-Checking Project is helping to revive truth and decency in North Carolina’s politics and to challenge the conspiracy theories and pants-on-fire campaign claims that threaten the self-regulating, healthy political society we seek.

Ryan Williams

Ryan Williams: I came into the Reporters’ Lab with relatively little journalism experience. I spent the past summer working on social media outreach & strategy at a non-profit where I drafted tweets and wrote the occasional blog post. But I’d never tuned into writing with the immense brevity of political messages during an election season. The N.C. Fact-Checking Project showed me the importance of people who not only find the facts are but who report them in a nonpartisan, objective manner that is accessible to an average person.

Following the 2016 election, some people blamed journalists and pollsters for creating false expectations about who would win the presidency. I was one of those critics. In the two and a half months I spent fact-checking North Carolina’s midterm races, I learned how hard fact-checkers and reporters work. My fellow fact-checkers and I compiled a litany of checkable claims made by politicians this midterm cycle. Those claims, along with claims found by the automated claim-finding algorithm ClaimBuster were raw material for many fact-checks of some of North Carolina hottest races. Those checks were made available for voters ahead of polling.

Now that election day has come and gone, I am more than grateful for this experience in fact-finding and truth-reporting. Not only was I able to hone research skills, I gained a deeper understanding of the intricacies of political journalism. I can’t wait to see what claims come out of the next two years leading up to, what could be, the presidential race of my lifetime.

Jake Sheridan

Jake Sheridan: I’m a Carolina boy who has grown up on the state’s politics. I’ve worked on campaigns, went to the 2012 Democratic National Committee in my hometown of Charlotte and am the son of a long-time news reporter. I thought I knew North Carolina politics before working in the Reporter’s Lab. I was wrong.

While trying to wrap my head around the 300-plus N.C. races, I came to better understand the politics of this state. What matters in the foothills of the Piedmont, I found out, is different than what matters on the Outer Banks and in Asheville. I discovered that campaigns publicly release b-roll so that PACs can create ads for them and saw just how brutal attack ads can be. I got familiar with flooding and hog farms, strange politicians and bold campaign claims.

There was no shortage of checkable claims. That was good for me. But it’s bad for us. I trust politicians less now. The ease with which some N.C. politicians make up facts troubles me. Throughout this campaign season in North Carolina, many politicians lied, misled and told half truths. If we want democracy to work — if we want people to vote based on what is real so that they can pursue what is best for themselves and our country — we must give them truth. Fact-checking is essential to creating that truth. It has the potential to place an expectation of explanation upon politicians making claims. That’s critical for America if we want to live in a country in which our government represents our true best interests and not our best interests in an alternate reality.

 

Back to top

“We need to disrupt the lies”

At the Simón Bolívar National Journalism Awards, a call for a more aggressive approach to fact-checking

By Bill Adair – November 16, 2018 | Print this article

My remarks for the Simón Bolívar National Journalism Awards, Bogota, Colombia, Nov. 15, 2018.

This is a critical moment for journalism around the world, when the path ahead seems uncertain. We lived through the dawn of the Information Age and saw the great promise of the internet; but it now seems like we are in a darker time.

When I started PolitiFact in 2007, I was filled with hope about what the digital revolution would bring. There was a belief the internet could make information more widely available, bring people together and help us hold power accountable. Like many of you, I am wondering if those hopes were misplaced. But we can’t get despondent about how things have turned out. We need to reimagine our roles as journalists and harness the power of technology to combat misinformation.

A little history: I started PolitiFact out of my own guilt. I had been covering the White House and Congress for the St. Petersburg Times, a Florida newspaper that is now called the Tampa Bay Times. I had grown tired of hearing politicians’ false claims and felt I had been complicit by publishing them in my news articles without scrutiny. The internet offered a new way for us to hold politicians accountable for what they said. I went to my editors with a crazy idea: instead of having me cover the 2008 campaign like all the other political journalists, how about if I started a fact-checking website?

Sure, they said.

In fact, they liked the crazy idea so much, they let me start a team with some of the most talented reporters and editors at the paper. They also let us break the rules. We built our own content management system and took some bold steps that most newspaper editors would never allow. We invented the Truth-O-Meter, which rated politicians’ claims from True to “Pants on Fire.” It made substantive articles about policy accessible to a wider audience.

We created a unique form of journalism. Instead of publishing traditional articles, we published fact-checks of politicians’ statements in a new structured form that could be collected on report card pages and tallied to tell people how many true, half true or Pants on Fire statements a politician had earned. PolitiFact was truly new journalism in the internet age.

A key to PolitiFact’s success was the culture of my newspaper. My editors were not only willing to let me try my unusual idea, they encouraged that kind of creative approach in everything we did. They also had a deep commitment to accountability reporting. They believed – they still believe – that holding power accountable is one of the fundamental missions of journalism.

PolitiFact became part of a growing community that included FactCheck.org and the Washington Post Fact-Checker in the United States, Full Fact in Britain and Chequeado in Argentina.

The digital revolution made it all possible. I remember those early years as an exciting time filled with promise. It was the honeymoon for journalism on the internet, as reporters and editors around the world discovered we could use the web to do powerful, important things.

News stories became interactive, enabling readers to engage with content. Data could be presented as vivid graphics that made numbers come alive. And design was transformed: I remember when we first saw Snowfall, the New York Times story about an avalanche, and we saw how the web could be used for powerful storytelling.

Then came smartphones, which enabled our readers and viewers to get the news all the time, wherever they were, and use thousands of apps to read articles and watch videos.  It seemed like the future was infinitely bright and there were lots more great things ahead.

But the honeymoon ended.

The internet got loud and crowded. Partisan voices began to dominate the discussions and people began shouting at each other in ALL CAPITAL LETTERS. Twitter became a thing, but we realized that some of the “people” on Twitter weren’t really human — they were bots. And they were programmed to pump up the partisan propaganda to drive apart the real people.

Partisans took advantage of the internet to build digital fortresses where they could isolate themselves from opinions they disliked. They hid behind the walls and lobbed attacks against their enemies. These fortresses are home to more propaganda than discourse and they provide refuge for extremists. If you’re inside one, your side is always right.

The iPhone had changed the landscape when it was introduced in 2007. It created a whole a new platform for apps, which offered promising new ways that people could connect. I remember when a friend in Chile showed me WhatsApp and how he used it to communicate with his friends and family.

But it didn’t take long for people who want to spread misinformation to discover they could use WhatsApp without getting much scrutiny from journalists. I know that here in Colombia, you saw lots of misinformation spread through WhatsApp about the peace deal in the 2016 election.

Pablo Medina Uribe, the editor of the fact-checking site Colombia Check, reminded me recently how WhatsApp is a fertile ground for falsehoods. People are more likely to believe WhatsApp messages because they’re sent by people they trust. But the nature of many mobile data plans here give people unlimited data on WhatsApp and Facebook but not for their internet browser or other apps.

So, on a broad scale, what can we do? More specifically, what can journalists and the tech community do?

We need to find new ways to harness technology to get accurate information to people when they need it. We need to be as aggressive and cunning as the people and groups who are spreading the misinformation. And we need to change our thinking.

First, the technology part. At Duke University, we’ve launched a project called the Tech & Check Cooperative that has an ambitious goal: to use automation to monitor politicians’ speeches and debates and provide live fact-checking sourced from existing fact-checks.

Five or six years ago, I thought this kind of automated fact-checking was a long time away. But advances in technology and the dividends from a partnership we started with Google have created remarkable momentum. So we have already created the first fact-checking app for the Amazon Echo called Share the Facts. It lets you query Alexa and get an instant fact-check.

It’s impressive: you can ask Alexa a question and, if the fact-checkers have published something on it, she will reply telling you that the Washington Post or FactCheck.org or PolitiFact checked it and what they concluded.

We are now building a similar product for TV and the web. It’s a much harder product to develop than the Amazon Echo, but the idea is that when our app hears someone say a statement that fact-checkers have examined, the app will pop up a related fact-check right on the screen of your smartphone or TV.

We’ve made significant progress in the last six months. Over the summer, our students created a rough framework for our app that converts a live speech to text, then filters out sentences that aren’t checkable using our ClaimBuster tool, and then uses an algorithm to look for matches from our database of previously published articles. We are still some months away from a finished product, but we are getting closer every day.

We also just completed the first user testing of instant fact-checking on TV. We had people watch specially modified videos of State of the Union speeches that had pop-up fact-checks. The viewers had helpful feedback for us about what they wanted on the screen, and they were unanimous about the concept: They all want real-time fact-checking on their TV.

And you don’t need an army of computer science students to create something valuable.  We need more projects like the “lie detector” developed by La Silla Vacia here in Colombia, which pioneered fact-checking on WhatsApp. People send a screenshot of a message they would like checked and then the journalists check it and encourage people to share it back through WhatsApp.

But regardless of whether we’re building big projects for a television screen, or smaller ones for WhatsApp, it requires that we think differently about the role of a journalist.

Most political reporters – and nearly all fact-checkers – have traditionally thought of themselves as neutral players in the political discourse. We published the information in the daily paper and left it on your doorstep. Or we just put it on our website or on the nightly news.

We were passive. We left it up to you to seek out the information that you needed and draw any connections or conclusions about the information on your own. I’ve been asked many times if it bothered me that politicians kept lying after we fact-checked them, and I would say, “Our job is just to provide the information.”

That strategy worked fine 10 years ago. But now, politicians and propagandists have learned how to spread misinformation at light speed. We can no longer sit back and wait for readers to come to us. We must become more aggressive and take the facts to the people. We need to disrupt the lies.

We can no longer be passive when our readers and viewers are being swamped with misinformation. We need to be more energetic and inventive in getting the information to people at the moment they first hear the claim.

Some of these solutions will be high-tech, like the automated fact-checking apps we’re building at Duke. Others can take more of a grass-roots approach, like the WhatsApp “lie detector” developed by La Silla Vacia.

But we need all kinds of these efforts, big and small, simple and complex, to adapt how we provide information in these fast-changing times. We need to build new apps for your phone and create new ways to provide the facts while you’re watching a speech on TV. And while print is still around, we should put the truth in ink and paper.

Yes, the honeymoon for the internet is over and sometimes it seems like we’re in a dark time. But I’m encouraged by the progress we’re making. I see promising efforts all around the world. And through it all I see great spirit and creativity.

The mission of the journalist remains the same: to give people the vital information they need to make sense of their world and hold their government accountable. We’ll continue to do that. We’ll just do it in new and creative ways.

 

Back to top

Duke students tackle big challenges in automated fact-checking

Trio assembled promising building blocks needed for live fact-checking

By Catherine Clabby – October 8, 2018 | Print this article

Three Duke computer science majors advanced the quest for what some computer scientists say is the Holy Grail in fact-checking this summer.

Caroline Wang, Ethan Holland and Lucas Fagan tackled major challenges to creating an automated system that can both detect factual claims while politicians speak and instantly provide fact-checks.

That required finding and customizing state-of-art computing tools that most journalists would not recognize. A collective fondness for that sort of challenge helped, a lot.

Duke junior Caroline Wang

“We had a lot of fun discussing all the different algorithms out there, and just learning what machine learning techniques had been applied to natural language processing,” said Wang, a junior also majoring in math.

Wang and her partners took on the assignment for a Data+ research project. Part of the Information Initiative at Duke, Data+ invites students and faculty to find data-driven solutions to research challenges confronting scholars on campus.

The fact-checking team convened in a Gross Hall conference from 9 am to 4 pm every weekday for 10 weeks to help each other figure out how to help achieve live fact-checking, a goal of Knight journalism professor Bill Adair and other practitioners of accountability journalism.

Their goal was to do something of a “rough cut” of end-to-end automated fact-checking: to convert a political speech to text, identify the most “checkable” sentences in the speech and then match them with previously published fact-checks.

The students concluded that Google Cloud Speech-to-Text API was the best available tool to automate audio transcriptions. They then submitted the sentences to ClaimBuster, a project at the University of Texas at Arlington that the Duke Tech & Check Cooperative uses to identify statements that merit fact-checking. ClaimBuster acted as a helpful filter that reduced the number of claims submitted to the database, which in turn reduced processing time.

They chose Google Cloud speech-to-text because it can infer where punctuation belongs, Holland said. That yields text divided into complete thoughts. Google speech-to-text also shares transcription results while it processes the audio, rather than waiting until translation is done. That speeds up how fast the new text can get moved to the next steps along a fact-checking pipeline.

Duke junior Ethan Holland

“Google will say: This is my current take and this is my current confidence that take is right. That lets you cut down on the lag,” said Holland, a junior whose second major is statistics.

Their next step was finding ways to match the claims from that speech with the database of fact-checks that came from the Lab’s Share the Facts project. (The database contains thousands of articles published by the Washington Post, FactCheck.org and PolitiFact, each checking an individual claim.)

To do that, the students adapted an algorithm that the open-source research outfit OpenAI released in June, after the students started working together. The algorithm builds on The Transformer, a new neural network computing architecture that Google researchers published just six months prior.

Duke sophomore Lucas Fagan

The architecture alters how computers organize trying to understand written language. Instead of translating a sentence word by word, The Transformer weighs the importance of each word to the meaning of every other word. Over time that system helps machines discern meaning in more and more sentences more quickly.

“It’s a lot more like learning English. You grow up hearing it and your learn it,” said Fagan, a sophomore also majoring in math.

Work by Wang, Holland and Fagan is expected to help jumpstart a Bass Connections fact-checking team that started this fall. Students on that team will continue the hunt for better strategies to find statements that are good fact-check candidates, produce pop-up fact-checks and create apps to deliver this accountability journalism to more people.

Tech & Check has $1.2 million in funding from the John S. and James L. Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation to tackle that job.

Back to top

FactStream app now shows latest fact-checks from Post, FactCheck.org and PolitiFact

New version features alerts for Pants on Fire and Four Pinocchio ratings

By Bill Adair – October 7, 2018 | Print this article

FactStream, our iPhone/iPad app, has a new feature that displays the latest fact-checks from FactCheck.org, PolitiFact and The Washington Post.

FactStream was conceived as an app for live fact-checking during debates and speeches. (We had a successful beta test during the State of the Union address in January.) But our new “daily stream” makes the app valuable every day. You can check it often to get summaries of the newest fact-checks and then click through to the full articles.

The new version of FactStream lets users get notifications of the latest fact-checks.

By viewing the work of the nation’s three largest fact-checkers in the same stream, you can spot trends, such as which statements and subjects are getting checked, or which politicians and organizations are getting their facts right or wrong.

The new version of the app includes custom notifications so users can get alerts for every new fact-check or every “worst” rating, such as Four Pinocchios from Washington Post Fact Checker Glenn Kessler, a False from FactCheck.org or a False or Pants on Fire from PolitiFact.

The daily stream shows the latest fact-checks.

The new daily stream was suggested by Eugene Kiely, the director of FactCheck.org. The app was built by our lead technologist Christopher Guess and the Durham, N.C., design firm Registered Creative. It gets the fact-check summaries from ClaimReview, our partnership with Google that has created a global tagging system for fact-checking. We plan to expand the daily stream to include other fact-checkers in the future.

The app also allows users to search the latest fact-checks by the name of the person or group making the statement, by subject or keyword.

Users can get notifications on their phones and on their Apple Watch.

FactStream is part of the Duke Tech & Check Cooperative, a $1.2 million project to automate fact-checking supported by Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation.

FactStream is available as a free download from the App Store.

 

Back to top

Spreading the word on The NC Fact-Checking Project

Journalists from the Reporters' Lab and the News & Observer appeared on Spectrum News

By Catherine Clabby – August 16, 2018 | Print this article

Cathy Clabby of The Reporters’ Lab and Andy Specht of The News & Observer appeared on Spectrum Cable’s “Politics Tonight” this week to explain the newly announced NC Fact-Checking Project. The journalists briefed Spectrum News host Tim Boyum on the ambitious plan to fact-check claims by politicians statewide during federal and state campaigns this fall and into the 2019 General Assembly session.

In the project, Duke journalism students and faculty – assisted by Tech & Check  bots – will scour campaign messaging and news reports to find newsworthy claims by politicians that can be verified. Journalists at The News & Observer will select statements to check, report their veracity and craft the fact-checks. The reporting will be shared for free with print, broadcast and digital newsrooms statewide. To watch, click here.

 

 

Back to top