data – Anna Wahrman

RIP, EveryBlock

February 8, 2013 Anna Wahrman Leave a comment

EveryBlock, a hyperlocal news start-up that used data to filter neighborhood news and spark discussion, has been shut down by its corporate overlord. Apparently, NBC News acquired it last year (which itself was news to me) but couldn’t find the business model to continue operating the site. That’s a common tale among hyperlocal news sites, but it still stings when one closes down.

It’s too bad — I think it had more going for it than many similarly themed sites — and its founder, Adrian Holovaty, seems shocked that the site has met its end. When he sold the site last year, he was proud of its success and confident in its future:

“EveryBlock users have used our service to accomplish amazing things in their neighborhoods: starting farmers markets, catching flashers, raising money for their community, finding/reporting lost pets…and generally getting to know their neighbors and forging community bonds. These days, something like this happens on the site nearly every day — which casual onlookers might not notice because of our long-tail, neighborhood-specific focus. EveryBlock has become a force for good, and it’s got a bright future.”

Sigh. I suppose it’s not particularly interesting that a start-up failed to locate a business strategy or that it didn’t “pivot” quickly enough to “disrupt” via its “MVP.” What is interesting about this case is that the site was a news-centric one that really challenged newsgathering tactics, asked questions about the use and display of public data and, in its small way, wrought lessons for the [cue horror-movie scream] Future of Journalism. It began, after all, as a recipient of a Knight Foundation grant.

Even more interesting is that it evolved so much over its short life (actually, wait, is six years long or short in technology?). When it began, it was just one news-tech guy’s realization that news should not be story-centric but instead should be gathered as structured data. He married the programmer’s philosophy of the separation of content and presentation with the journalist’s instincts for ever-better storytelling. Holovaty’s blog post from September 2006 is, in retrospect, both amusing and prescient. In it, he calls for parsing data and creating CMSes that support content types other than words, two notions that are laughably obvious six years later.

(On the flip side, also laughable are the mention of PDAs and the idea that tagging was “trendy.”)

Holovaty turned those 2006 idea germs into EveryBlock’s mapping and reporting functionality and, ultimately, he created a robust community around neighborhood news. The site put forth a notion of what the oft-dreaded Future of Journalism could be, or one version of it, anyway. It tried something new. It experimented. And the experiment did yield results; unfortunately, the conclusion was that this model might not be quite right.

In its sad and clearly hasty post today confirming the shutdown news, EveryBlock seems to acknowledge that it was a victim of the unforgiving pace of change in the online journalism industry:

“It’s no secret that the news industry is in the midst of a massive change. Within the world of neighborhood news there’s an exciting pace of innovation yet increasing challenges to building a profitable business. Though EveryBlock has been able to build an engaged community over the years, we’re faced with the decision to wrap things up.”

In short: “We tried. We’d like to keep trying, but trying doesn’t pay the bills.” And that’s too bad.

Journatic and the future of local news

July 5, 2012 Anna Wahrman Leave a comment

This week’s This American Life featured a segment on Journatic, a hyperlocal, scaled-content creator that’s apparently replacing local reporters in many markets. I’ve previously written about hyperlocal news and the value of using algorithms in news creation, so the story was of great interest to me.

My argument with hyperlocal is that no one has yet figured out how to do it right. It sounds to me like Journatic is finding some success, but it’s also failing in important ways. My defense of algorithms is mostly to do with the company Narrative Science, which as I said is “not a threat, it’s a tool, and it fills a need.” That need is basically the scut work of news reporting, and although the folks there are working on this very issue, for now, “It’s a tool that does a programmatic task, but not a contextual one, as well as a human.”

Journatic aims to solve the hyperlocal problem with the algorithmic solution. The company scrapes databases of all kinds, then uses that data to “report” on local bowling scores, trash pickup times, where the cheapest gas is, and who has died recently. The company does this by using algorithms to mine and sort public information, and there’s nothing necessarily wrong with that.

When it launched, Journatic-populated site BlockShopper was basically a real-estate listings site based on publicly available data. Using public records, it would “report,” for example, that “123 Main St. is in foreclosure.” But since then, the algorithms and tools have gotten smarter. Soon it was able to say a home was in foreclosure “by the bank” and also add that it “is up for auction on March 31.” The site is now so smart that it actually feels almost invasive. To wit:

The real estate information contained in the article is publicly available, from the names of the people involved in the transaction to the price paid to the location details. The fascinating thing, and what pushes it into a brave new frontier of journalism and privacy invasion, though, is that the information on the professions of the involved is also publicly available (probably via LinkedIn). Arguably, all the article is doing is presenting public data in a new format. The difference is access and availability. In the pre-Internet days, there was no way to know public information except to go to the city records office and look, and there was really no way to know about peoples’ professions except to know them or ask them. These tasks required interested and motivated parties (such as journalists), because actually going places and talking to people requires on-the-ground reporting (not to mention complicit consent). This is not the sort of work Journatic traffics in. That’s not a criticism, necessarily, just a fact: There used to be barriers to the information; now there aren’t; Journatic uses this lack of barriers plus its algorithms to surface the data.

Journatic aims to solve the hyperlocal problem with the algorithmic solution.

At first, the company didn’t do any (or much) writing or analysis. According to This American Life and its whistle-blower, though, the company now pays non-native-English-speakers in the Philippines between $.35 and $.40 a story to try to add a bit of context to the data. Thirty-five to forty cents! However shady this is, though, it is not necessarily unethical. It’s capitalistic, and it’s pretty shameful, and it feels wrong somehow, but it’s not unethical journalistically.

Where it does get unethical is when readers are misled, and that has apparently occurred. They force these writers in the Philippines to use fake bylines like “Amy Anderson,” “Jimmy Finkel” and any number of fake bylines with the last name “Andrews,” in order to Americanize them and dupe readers, according to the show. This is flat-out wrong, and I think Journatic knew it — they apparently reversed their stance on this after the story aired.

But ethics aside, and journalism in broader context here, Journatic’s founder, Brian Timpone, claims that the “single reporter model” doesn’t work anymore. The Chicago Tribune, one of Journatic’s customers, says that it’s gotten three times more content for a lot less money. These are serious issues for the future of the profession (along with the opportunity for privacy invasion and privacy mishandling that all this unfiltered data presents). It’s no doubt true that the Trib paid less money for more content versus hiring local reporters. But what is the quality of the work? I think we all know the answer. Shouldn’t that be a bigger factor than it is? If you’re just turning out junk, your brand gets diluted, and your readers soon abandon you altogether.

It’s easy to criticize, but it seems to me that Timpone is trying, as we all are, to devise a way forward. That’s admirable, in its way. It’s a little scary, and the desire for progress sometimes makes us color outside of the lines, and when that happens, places like This American Life need to be there as a regulator, as has just happened. We’re all still muddling our way through the ever-changing new online media landscape, and we will test theories and make mistakes and learn lessons, and with any luck we will end up with a better product, one that serves readers first, last and always. I hope someone is able to someday crack the code of good news done quickly at good quality for a good wage. Until then, we must keep trying.

Algorithms as a tool to better journalism

April 27, 2012 Anna Wahrman Leave a comment

Wired‘s recent story about Narrative Science seems to have put some journalists into a bit of a tizz. The article is a must-read for journalists and coders — really interesting tidbits about what’s going on in this field now, and what might come to pass in the future.

I’m actually very excited about the possibilities of Narrative Science, an artificial intelligence product that transforms data (currently primarily from the sports and finance world) into stories. This is the exact kind of thing we’re after when we encourage J-Schools to put software engineering into journalism curricula so we can teach young journalists valuable new skills so they, in turn, can not end up helpless on the sidelines, as many of us current journos have been during the technology advances of the last decade.

The method does not determine the value

Narrative Science is not a threat, it’s a tool, and it fills a need. Instead of some capable writer poring over boring financial statements and trying to add sizzle in reporting on them, a machine reads the data and spits out two grafs. Two serviceable but really snoozy grafs, which probably would have happened if written by a human, too.

Here’s what’s intriguing, though: Narrative Science is working on ways to be not-snoozy, and in so doing they’re calling journalists on our BS, in a way. What I mean is this: Journalists have formulas. We do, and they’re taught in schools and learned on the job. “Reverse pyramid.” “Nut graf.” “Lede.” “Attribution.” These are plug-and-play tactics most of the time. Sure, these elements vary from story to story, and that is the fun part of what we do. We add details and context. We observe and report. But at core, we tell different stories using some slightly different combinations of these tactics and tools.

Arguably, feature stories have slightly more variety, but I’d also point out that (sadly) many features are also just puzzle pieces, if not downright parodies of themselves. For example, every feature on every female celebrity ever starts this way:

“[Lady celeb] walks into [L.A.’s or New York’s] [restaurant or cafe in trendy neighborhood]
looking gorgeous in [brand] jeans and no makeup.”

Whether the editors or writers are making the words hacky, hacky they are — and boring, just like the pieces Narrative Science is creating with its algorithmic journalism. Fascinatingly, according to Wired, the company actually has “meta-writers” whose job it is to help the computers add context:

“[Meta-writers are] trained journalists who have built a set of templates. They work with the engineers to coach the computers to identify various ‘angles’ from the data. Who won the game? Was it a come-from-behind victory or a blowout? Did one player have a fantastic day at the plate? The algorithm considers context and information from other databases as well: Did a losing streak end?”

But to answer the question posed in the headline of the piece, “Can an Algorithm Write a Better News Story Than a Human Reporter?” for now the answer is no. And journalists vs. algorithms is a faulty comparison.

Writers and editors add value using tools

Narrative Science, thanks to algorithms created by human engineers and journalists, is now at the level of being able to programmatically spit out phrases like “whacking home runs.” But it can’t gauge a crowd’s restlessness or excitement. It can’t interview a superfan after the game, sense that he’s fed up with the team and write a mood piece. It can’t connect on a human level to a victim of a crime, or spend days following a subject then put together disparate threads of the subject’s life into a coherent portrait.

Which is why it’s not a real threat just yet. The way I see it:

Narrative Science : journalists : : spell-check : copy editors

It’s a tool that does a programmatic task, but not a contextual one, as well as a human. Does spell-check tell you you have the wrong “hear/here”? No. Does it correct you when you’ve spelled “embarrassing” incorrectly because it is drawing from an enormous database of correctly spelled words? Sure, easy enough. Can it check a fact’s accuracy against a thousand links on the Internet? Probably. But can it call a source and make sure she wasn’t misquoted, then correct the quote before publication? Not likely.

Context is everything, and it’s ours to use. But we journalists have to use it. Yes, we have formulas. We write ledes, and we edit the story so the most important information is up front. But we have to step up our game. We have to go to the match, or the crime scene, or the meeting, or the fashion show, or the foreign city, or the war, and add context for readers. We shouldn’t hack our way through the really interesting stuff — we shouldn’t be allowed to. Let’s let bottom-scrapers scrape the bottom for us. Let’s not waste human effort on shitty content farms that pay $2 (!) an article. Let’s leave that for robots and invest elsewhere: in hiring more and better writers and editors to make connections, describe the atmosphere, make sense of things, tease out themes and (cue dramatic music) better humanity. Let’s invest in creating data and algorithms that we can program to help us help ourselves.