We tested two fact-checking products during the State of the Union address. One failed, the other showed great promise.
The failure was FactStream, our iPhone app. It worked fine for the first 10 minutes of the speech. Users received two timely “quick takes” from Washington Post Fact Checker Glenn Kessler, but then the app crashed under an unusual surge of heavy traffic that we’re still investigating. We never recovered.
The other product is a previously secret project we’ve code-named Squash. It’s our first attempt at fully automated fact-checking. It converts speech to text and then searches our database of fact-checks from the Post, FactCheck.org and PolitiFact. When it finds a match, a summary of the fact-check pops onto the screen.
We’ve been testing Squash for the last few weeks with mixed results. Sometimes it finds exactly the right fact-checks. Other times the results are hilariously bad. But that’s what progress looks like.
We went into last night’s speech with very modest expectations. I said before the speech I’d be happy if the speech simply triggered fact-checks to pop up, even if it was a poor match.
But Squash actually performed pretty well. It had 20 pop-ups and six of them were in the ballpark.
Overall, the results were stunning. It gave us a glimpse of how good automated fact-checking can be.
We’ll have more to share once we’ve reviewed the results, so stay tuned.
As for FactStream, it now has lots of timely fact-checks from the State of the Union on the main home screen, which continues to function well. We will fix any problems we identify with the live event feature and plan to be back in action for real-time coverage for campaign events later this year.