Google forges ahead with next generation of image search

Search giant Google wants to push forward with a new, improved form of image search that’s based on visual characteristics, not just text. Last week, two of the company’s scientists presented a paper outlining a system they’re calling VisualRank. They’ve told the New York Times it could be as important for image search as the now-legendary PageRank paper was for normal text search.

Currently, image search engines (notably Google Image Search) locate and rank pictures based on the text describing them on a web page. This can lead to some very hit-or-miss search results — typing in “mcdonalds”, for example, gives you a wide range of images, many of them with little visual relevance to the McDonald’s fast food chain. (Irrelevant, that is, unless it turns out you actually are looking for photos of fat giraffes.)

The new system proposed in the Google paper ranks images based not on text, but on the common “visual themes” found in each search result. In the McDonald’s example, the VisualRank system would see that the company’s famous golden arches are a common visual theme, and prioritize pictures that feature the arches prominently. Tests of this new system returned 83 percent fewer irrelevant search results than Google Image Search, according to the VisualRank paper. (In the “search graph” below, the two large images in the center are ranked the highest because they feature the common visual themes in a search for “mona lisa” most prominently.)

If Google can deliver on its promises, VisualRank could bring a sea change to applications like product search, where customers are often want to find and compare specific products, but may not know the exact words to describe them. There are already product comparison startups, such as Like.com, that use visual search. But Google’s plan to reveint the image ranking process is much more ambitious. In fact, Munjal Shah, chief executive of Riya (which owns Like.com), told the Times that Google’s goals are “largely impossible.”

Whether that’s true is a debate better left to computer scientists. But wagering against Google’s search technology has been a sucker’s bet in the past.