Robert Scoble, the tech blogger, has drawn a lot of attention with a talk about why Google is beginning to fail as a search engine, and how upstarts may eat its lunch within a few years.
It is provocatively titled Why Mahalo, TechMeme, and Facebook are going to kick Google’s butt in four years, and here’s first video of a series (we’ve embedded it below, after the jump).
His argument is solid for the most part. Google’s search results have deteriorated because expert marketers are increasingly gaming the popular search engine. They pay for links to make their web sites more significant in Google’s eyes, and there’s no way for Google’s algorithm to determine whether a link is paid or not.
A bunch of start-ups are using human editing filters to do a better job of sorting through what Web pages are significant and which ones are not. The best ones (TechMeme, Facebook) are aggregating thousands, even millions of people whose choices can help point to relevant information. For example, if real people at Facebook talking about me tend to link to my site, VentureBeat, well, that site will show up first when you search for my name. At Google, it doesn’t. The idea is that if you can peer into, and access all the real links made by people within Facebook, and combine it with other trust-oriented sites, you’ll have a better engine.
The start-ups haven’t yet fully harnessed the power afforded them by the people that use their services, in part because they didn’t start by wanting to solve the problem of search. They’ve happened on to it. Each of them bring different advantages and disadvantages to the game. There may be more promising that the ones that the three Scoble mentions, including Wikia’s secretive project that is underway.
Scoble says Google’s algorithm is “stuck in cement,” so elaborate and so entrenched that the big company won’t be able to adapt as quickly as small companies with less to lose using a radically different approach. Google couldn’t simply buy a competitor to help it adapt, this argument goes. It would find it too complicated to incorporate the human-filtered approach into its algorithm. Google doesn’t understand social technology, Scoble continues. Google even owns social network, Orkut, but hasn’t been able to leverage lessons learned from that experience to make Google better, he argues.
The main problem with Scoble’s argument is that the upstart companies will have trouble scaling. It is difficult — perhaps impossible — to organize humans to reliably track millions of changes that are happening all over the place ever second on the web. That’s why the upstarts may be more successful organizing efficient results for only the most popular topics, which is what Mahalo is doing. The other challenge for the upstarts is that search marketers will increasingly look for ways to came their systems too, setting up spam link-laden groups at Facebook, for example.