Is Google working on a way to search Twitter and other microblogs?

June 13, 2009 | Dean Takahashi | Comments | |

Google is definitely collecting data on the web every second these days so the question is when, not if, Google will come out with its own microblogging search engine to compete with Twitter. Google is now preparing to launch a way to index and rank content from services like Twitter, according to the blog Google Operating System.

Indexing Twitter information is harder than it may seem. Until now, services have needed permission from Twitter to access its entire content, and there’s very little chance Twitter would ever give Google that access. So until now the assumption is that Google would literally have to “follow” every single user on Twitter, and then do continuous searches in order to index all their tweets — a ridiculously cumbersome process.

Twitter itself has recently started its own search, thanks to its acquisition of Summize.

So what’s the value of being able to search through Twitter? Well, you can find out what the most popular trends are at any given moment. There are so many people using Twitter now that it is instructive to see the words that are most often being tossed about in tweets, the 140-word posts that users make on Twitter, in real time.

Google, meanwhile, also has a limited way of tracking what people are interested in any given time, because it can track what people are searching for: See Google Trends for this. But if you search Twitter in real time, you arguably get more accurate and rich information about what people are talking about to the very second. This may seem extreme, but the day is coming when split-second information on search trends is useful — whether its for tracking the spread of sicknesses or trading on the stock market.

Twitter’s own search engine doesn’t search through anything except Twitter, and it only sorts results by date, not relevancy. Google could come into this picture and bring to bear its much more robust search and indexing technology. Let’s hope that the rumored Google microblogging search engine is really in the works.

[logo: Electricpig]

Next Story: Welcome to VentureBeat’s new design
Previous Story: Report: Marc Andreessen and Ben Horowitz raise $300M venture fund

Bookmark and Share

Tags: ,

Photo of Dean Takahashi

About the Author, Dean Takahashi

Dean is lead writer for GamesBeat at VentureBeat. He covers video games, security, chips and a variety of other subjects. Dean previously worked at the San Jose Mercury News, the Wall Street Journal, the Red Herring, the Los Angeles Times, the Orange County Register and the Dallas Times Herald. He is the author of two books, Opening the Xbox and the Xbox 360 Uncloaked. Follow him on Twitter at @deantak, and follow VentureBeat on Twitter at @venturebeat.

  • i think no
  • Andrew
    Not to put a damper on the notion, but Google's infrastructure technology is *theoretically* incapable of supporting real-time search. It is not even controversial, even among the other companies that use similar technology. Unless they have thrown everything out and started from scratch, I find it highly unlikely that any real-time search of microblogging is more than thin and superficial.

    That said, I may be reading too much into this. Yeah, it might be possible for Google to have a search service that works adequately in a very limited scope with respect to microblogging. Nonetheless, the general problem of real-time search of various data streams is theoretically equivalent to another longstanding unsolved problem in computer science, indexing true spatial data, which also remains manifestly unsolved. I don't think many people fully grok the theoretical context of these kinds of applications nor the implications of solving the underlying technical problems.

    To do this, Google would be required to discard their entire technology base as a theoretical prerequisite. I do not think many people would make that assertion.
  • D
    Google already shows a lot of recent results for topics in the news using regular Google search. Google wouldn't have to throw out what it has to add on a section for tweets, just like it adds in a section for news results.
  • Peter Antypas
    Andrew is right. Several years ago, eBay asked Google to provide them with search, and Google failed. The reason was (as Andrew pointed) that Google's algorithm is optimized for tons of slowly changing content. They simply can't handle massive real-time streams. And, no, there's nothing spectacular about indexing News. There's actually very few news streams out there (Reuters, AP, etc). Compare that with potentially millions of tiny documents (tweets) per hour and you have enough to keep CS geeks busy for years.
  • It would be interesting having adsense/adwords on twitter.
  • I agree with Howard H. Why not put some adsense on my twitter page, and give me a cut? I would post night and day, if I could make some money off of it.

    Chris
    http://worstiphoneapps.blogspot.com
  • Hi Dean, Thanks for the post. Very useful, I referenced this article in my own post today:
    http://apoint2share.com/?p=43