Media

Twitter announces firehose partnership with DataSift

firehoseTwitter and social media data company Mediasift today that they’re partnering to sell access to Twitter’s “firehose” of data. Mediasift is only the second company to sign such a deal with Twitter — the other is Gnip.

I interviewed Twitter’s platform lead Ryan Sarver this afternoon to talk about the deal and how it fits into Twitter’s broader platform strategy.

Last month, Sarver published a post in Twitter’s developer forum explaining that it’s a bad idea for developers to build apps that “mimic or reproduce the mainstream Twitter consumer client experience.” Instead, he wrote, they should focus on one of five areas –publisher tools, curation, realtime data, customer service, and value-added content. He said Twitter “wants the bar to be really low” for developers to build those services, which is why the company partnered with both Mediasift and Gnip. Many developers want access to Twitter’s data, but Twitter doesn’t want to spend a lot of time negotiating with all of them, or building the necessary data tools.

Instead, Sarver said, Twitter wants to work with a very small group of trusted partners who sell access to the data on Twitter’s behalf. Each of those partners should serve a different need. Gnip, Sarver said, focuses on “large bulk data”, while Mediasift’s DataSift service helps customers filter data from Twitter to find the specific information that’s relevant to them. (DataSift customers include companies like Klout, which identifies influential Twitter users on various subject. Mediasift also operates the Tweetmeme service for consumers.) And yes, Twitter wants to work with other data partners, as long as they are “adding their own variety” to the ecosystem.

“It’s not zero sum,” Sarver added. “There are so many different ways to analyze and quantify what’s going on.”

He also clarified that while companies like to talk about providing access to the full Twitter firehose, they’re not actually showing that firehose to their customers. Instead, Gnip and DataSift provide slices of data from Twitter — for example, Sarver said, a DataSift customer can’t ask to see every tweet ever, but it could request every single tweet that mentions “Coca Cola”.

[photo via Flickr/Perry McKenna]