“If everybody is special, then no one is.”
Such is the problem companies face when crowdsourced criticism tends to be — er — not so critical.
Yelp, a community recommendations site, is the first that comes to mind facing this problem. The average rating for its more than 7 million reviews is 3.8 stars (out of five). Co-founder Jeremy Stoppelman says that’s a little better than other user-generated sites like YouTube, whose users say its videos are worth an average 4.3 stars.
Still, it means that any user looking for a place to have dinner inevitably comes to a list of all four-star restaurants (like the one to the left). There are so many reviews that it’s virtually impossible to have a perfect record, yet people are reluctant to hand out one- or two-star ratings. And when you’re looking for a fast solution on where to eat, a list full of four-star restaurants makes for an overwhelming “paradox of choice” situation. (Ironically, the site feels less useful to me now than it did two years ago when there were enough reviews that I could discover great finds, but not be daunted by all the four-star choices.)
So how does Yelp deal with this? The company says it’s not straying from the five-star rating system anytime soon, not even for some sort of percentile system where you can see how restaurants compare to each other. Instead, the company says its ranking system already factors in the number of reviews, whether they come from experienced Yelpers or first-time reviewers, and whether those reviews were voted helpful.
Virtually identical ratings mean people have to dive into reviews to understand what’s different, said Vince Sollitto, who heads communications for the San Francisco-based company. (Yes, it takes more time, but I bet it’s better off for Yelp’s advertisers.) The company’s also pushing more social features, nudging users to find and follow like-minded reviewers. Rather than relying on a generic one- to five-star system, Yelp says connecting people with similar tastes may be the best way of discovering the next gem.
VB's research team is studying web-personalization... Chime in here, and we’ll share the results.