Google already knows its search sucks (and is working to fix it)

It’s a popular notion these days Google has lost its “mojo” due to failed products like Google Wave, Google Buzz, and Google TV. But Google’s core business — Web search — has come under fire recently for being the ultimate in failed tech products.

I can only ask: What took so long? I first blogged about Google’s increasingly terrible search results in October 2007. If you search for any topic that is monetizable, such as “iPod Connectivity” or “Futon Filling”, you will see pages and pages of search results selling products and very few that actually answer your query. In contrast, if you search for something that isn’t monetizable, say “bridge construction,” it is like going 10 years back into a search time machine.

Search has been increasingly gamed by link and content farms year by year, and users have been frogs slowly getting boiled in water without realizing it. (Bing has similarly bad results, a testament to Microsoft’s quest to copy everything Google.)

But here’s what these late-blooming critics miss: Yes, Google’s search results do indeed suck. But Google’s fixing it.

The much acclaimed PageRank algorithm, which ranks search results based on the highest number of inbound links, has failed since it’s easy for marketers to overwhelm the number of organic links with a bunch of astroturfed links. Case in point: The Google.com page that describes PageRank is #4 in the Google search results for the term PageRank, below two vendors that are selling search engine marketing.

Facebook, which can rank content based on the number of Likes from actual people rather than the number of inbound links from various websites, can now provide more relevant hits, and in realtime since it does not have to crawl the web. A Like is registered immediately. No wonder Facebook scares Google.

But the secret to Google’s success was actually not PageRank, although it makes for a good foundation myth. The now-forgotten AltaVista, buried within Yahoo and due to be shut down, actually returned great results by employing the exact opposite of PageRank, and returned pages that were hubs and had links to related content.

Google’s secret was that it could scale infinitely on low-cost hardware and was able to keep up with the Internet’s exponential growth, while its competitors such as AltaVista were running on expensive, big machines running processors like the DEC Alpha. When the size of the Web doubled, Google could cheaply keep up on commodity PC hardware, and AltaVista was left behind. Cheap and expandable computing, not ranking Web pages, is what Google does best. Combine that with an ever-expanding data set, based on people’s clicks, and you have a virtuous circle that keeps on spinning.

The folks at Google have not been asleep at the wheel. They are well aware that their search results were being increasingly gamed by search marketers and that this was not a battle they were going to win. The answer has been to dump the famous blue links on which Google built its business.

Over the past couple of years, Google has progressively added vertical search results above its regular results. When you search for the weather, businesses, stock quotes, popular videos, music, addresses, airplane flight status, and more, the search results of what you are looking for are  presented immediately. The vast majority of users are no longer clicking through pages of Google results: They are instantly getting an answer to their question:

Google weather search results

Google is in the unique position of being able to learn from billions and billions of queries what is relevant and what can be verticalized into immediate results. Google’s search value proposition has now transitioned to immediately answering your question, with the option of sifting through additional results. And that’s through a combination of computing power and accumulated data that competitors just can’t match.

For those of us who have watched this transition closely and attentively over the past few years, it has been an amazing feat that should be commended. So while I am the first to make fun of Google’s various product failures, Google search is no longer one of them.

  • http://www.realestateactive.com/ Michael Real

    We indeed see Google’s search results sucks especially for those competitive keywords since internet marketers had flooded it with their makeshift websites but for the knowledgeable, we can twist the search query to get the most relevant results.

  • http://www.magnity.com David Shantz -Magnity

    In my opinion Google's problems run deeper. They are stuck in the classic leader's dillemma – and having to reinvent major franchises is never easy – but really, Google's applications and interfaces rarely change much after launch. They are engineering focussed and almost never innovate in terms of User Experience. UI/UX may seem trivial, and design relatively unimportant – except when you stop to consider that the differences between MySpace and Facebook are mostly differences that you can see. There really isn't a massive difference in technology between the two social networks.In search, it also seems to come down to a lack of context. The company seem to be enslaved to a broken model to a degree that has not changed since 1999. That's a dinosaur.

  • http://www.youtube.com/dfmediainc DavidAFenton

    Search was just a fundraiser for the businesses Google really wants to be in.

  • http://www.google.com/profiles/adam.koncz lyesmith

    Google search results still the best. Yes its quality has declined, but it is not true that Facebook results are better or even close. Like-s do not represent importance nor relevance they represent popularity.

  • http://www.taranfx.com Taranfx

    If Google sucks, Bing sucks more and Facebook is nowhere. Stop complaining for a thing which is still the best. Everyone has challenges.

  • http://pulse.yahoo.com/_X25OEJP5MGT6YJEEVVNZOURI7M Tucros

    Assuming facebook built a search engine based on people's “Likes”, wouldnt it be just as easy for scammers to game that system? IMO, it will always be a cat and mouse chase because whatever new clever method they figure out for ranking pages, people will reverse engineer that and exploit it.

  • http://pulse.yahoo.com/_XHHPJJSDJASCXKKT5AS5CPVGGA Jerry Deboever

    It makes me laugh how people thinks typing one word on google search will give them the right answer right away. Nowadays, there are millions of pages and you might need to be more specific in your research to find what you are looking for. I also love the people giving crap to google but still uses it on a daily basis.

  • http://pulse.yahoo.com/_XHHPJJSDJASCXKKT5AS5CPVGGA Jerry Deboever

    It makes me laugh how people thinks typing one word on google search will give them the right answer right away. Nowadays, there are millions of pages and you might need to be more specific in your research to find what you are looking for. I also love the people giving crap to google but still uses it on a daily basis.

  • http://twitter.com/gigigan gigigan

    My opinion is that's the social value is a big thing, and google have to do something in this way. They NEED to implement a social layer into the search engine to value links on an human base (+ the algorithm) otherwise they can be kicked by Facebook, not for search itself but for the way people get news about some topics. I do wrote something on my blog about that but it is in French, I may translate it soon.

  • http://articulationstudy.posterous.com/ david karapetyan

    You lost me at facebook can provide more relevant results. I guess if you consider the opinions of prepubescent teenagers “relevant” information then yes facebook can indeed provide better results.

  • http://www.facebook.com/profile.php?id=504633504 Dan Mitchell

    David, the average Facebook user is a middle-aged adult, and “Likes” are registered mainly on sites geared to adults — major publications like the NYT and the WSJ. Am I missing something? Are you sure you know what you're talking about?

  • http://articulationstudy.posterous.com/ david karapetyan

    Really? I guess I'm hanging out in the wrong corner because my news stream is full of lolcats and other nonsense. I tried hanging out in the more “mature” section once but there I was inundated with either tea party nonsense on the right or the equivalent of it on the left. What I wonder is how you're getting useful information out of facebook? You should write a book about it because it'd definitely be a hit with whoever wrote this article.

  • http://www.facebook.com/profile.php?id=504633504 Dan Mitchell

    My facebook news stream is full of stuff from friends, nearly all of them middle aged adults. In the mornings, it's mainly full of news — the Economist, Atlantic, NYT, etc. etc. I have some immature friends — I've hidden them. I'm not sure what you're referring to with “corner” and “sections.” How does tea party stuff show up unless you've chosen to see it?

  • xmichaelx

    Wow. If your friends are children or Teabaggers, you get lolcats and hysteria. You're not “hanging out on the wrong corner”, you just have no friends.But I also can't imagine getting useful info from Facebook, since it would be trivial to game the Facebook system just by creating an army of fake profiles.

  • http://twitter.com/cselland Chris Selland

    It's really quite simple – if something better comes along, customers will migrate to it. That hasn't happened yet.

  • xmichaelx

    I spoke with some Google folks about 5 years ago about their increasingly dismal search results, and asked them why they didn't just include simple Dublin Core metadata in their searches, maybe with a 30-character limit, so that old-school keyword stuffing wouldn't be possible. This would easily allow a modicum of faceted searching and could instantly allow accurate date, author, and data type searching, without the overhead of full RDF.It immediately became clear that, although these guys are wizards at code and crawling and harvesting, they had no concept of *organizing* information. For 10 years they've tried to make keyword searching the be all, end all. It's the data equivalent of shoving a square peg in a round hole.

  • http://articulationstudy.posterous.com/ david karapetyan

    I meant “friends” and “groups” since that's how your news stream is populated. Those sites mentioned are already well organized with the most popular items within easy reach so I'm not really sure what value your friends add to the articles by “liking” them and pushing them onto your news stream. Next time your read one of those articles do me a favor and check out their front page to see if the article is within your field of vision. Mainstream outlets like the ones you describe already have their hand on the pulse of what people will read and what they won't so you are not going to find anything there you wouldn't hear during lunch just chatting with your friends. Since the article is about search and relevance so far no one has made a convincing case about how social approval means more relevant in terms of information content and pertinence. Facebook just turns everything into a fashion show. The news stream is just a never ending catwalk. Actual search is a very hard problem and sprinkling “like” all over the algorithms that currently scour the web is not going to make it better.

  • http://articulationstudy.posterous.com/ david karapetyan

    I was merely making a point about the value of social approval when it comes to obtaining information. My friends don't know any better than I do so what they “like” or don't like is not really going to blow my mind or add any value to what I'm searching for. What people want is curated content vetted by experts and most people on facebook are far from experts on anything. They are susceptible to shady marketing tactics just like everyone else. That's why a question mark appears over my head every time someone says social is better for everything because I simply can't fathom what these people are thinking.

  • http://twitter.com/nephari Ellie

    The majority of Google searches don't serve a blended result, so I don't see how this is having a major impact on mitigating the pitfalls of pagerank.Furthermore, having one answer from a blended result while conducting a research task is next to useless. Incorporating blended results into the SRP doesn't change the fact that the organic results are being overrun by mostly irrelevant sites.Blended results may help for some types of queries, but certainly not all, and I don't think they are anywhere close to being a solution for the problems Google is facing.

  • http://www.facebook.com/profile.php?id=504633504 Dan Mitchell

    Most of what you're saying here doesn't seem to make any sense, though it appears you've moved away from arguing that Facebook is filled with nothing but dumb teenagers posting lolcats.Just about every day I read things I've seen only because friends on Facebook (in my “corner,” which I built myself) linked to or liked. Since I'm a journalist, many of those friends are “expert” vetters, but not all of them are. They're mostly pretty smart, though. I also read through twitter and RSS feeds and simple Web surfing. Reality belies your argument – at least the parts of your argument that make sense and can therefore be belied.

  • http://twitter.com/gorash aaaaaaaaa

    Typical Apple user… they always vaguely complain something about “User Experience”. Whatever THAT means.

  • http://articulationstudy.posterous.com/ david karapetyan

    I see the hyperbole was a bit much for you. Let me dumb it down. Your profession has been sucking it up for a while now and maybe it's because “professionals” like you get their information from an echo chamber called facebook. Not really sure what reality you are referring to since facebook has more inane conversation and links than the worst of internet's forums but fortunately the forum posts don't have a “like” button so you can still have some hope left after reading a forum post but when some retarded remark has 10,000 likes then maintaining sanity and hope for the future of humanity gets a little harder. Facebook doesn't make things better because it encourages one off witty remarks with practically 0 informational content and link sharing that is more a popularity contest than anything else. When your search results are informed by the opinions of the masses and Lady Gaga jumps to the top for practically any query you can think of with the words “pop culture” in it then we'll see how much you like social search. Good luck with the journalism thing you'll need it.

  • narg

    Not sure what this guy is searching for but Google still remains the only search engine I use due to it working 100% of the time I need it to. I guess this guy doesn't realize that searching for Paris Hilton videos won't return anything worthwhile?

  • http://twitter.com/completeview inlevel

    I agree that Google improve search. I think we should remember that in recent years commercial companies had a choice: to produce valuable (but very expensive) content and do not be visible in Google search or invest some money for SEO agencies and get visibility with content generated by algorithms. This process has started and I will be difficult to stop it. Now biz knows that Google likes trash SEO specialist are feeding it and that it is matter of time and money to get visibility (not value of the content). We will see.

  • narg

    I agree. Google isn't broke. If anything the web is broken.

blog comments powered by Disqus