In Thursday’s Google earnings call, CEO Larry Page told the world that the company’s fledgling social network, Google+ has reached 90 million registered users. He went on to say that, “Over 60 percent of Google+ users use Google products on a daily basis. Over 80 percent of Google+ users use Google products every week.”
I’m not impressed by the numbers, and I’m not impressed by what Page was trying to do with them.
Counting registered users instead of daily active users tells us nothing about the popularity of the service. Think of the millions of people who’ve registered for Google+ but never use it. Second, given the huge popularity of Google search, Gmail, and YouTube, it’s actually surprising that so few people who have registered for Google+ are using those more popular services on a daily basis — only 60 percent. After all, remember that a lot of Google+ users accidentally became Google+ users only because they were already attached to another Google service.
But what concerns me most is that Google is touting these meaningless statistics in the hopes that journalists will misunderstand them and report that Google+ is seeing rapid growth. The bottom line is, those 60 percents, 80 percents and 90 million registered users are just there to mask the fact that Google doesn’t want to tell us how many people are actually using Google+.
It’s intellectually dishonest. And as a public company, it raises questions of Google’s intent — the market is watching Google’s moves in social and needs to see traction. I expect better from Google.
This isn’t the first time Google has tried to mislead with statistics about Google+. In July, Page claimed that the service had 10 million users who shared 1 billion items a day.
That sounds incredibly impressive. But let’s do the math. That would mean that the average user was sharing 100 items a day. Robert Scoble was flooding my feed before I blocked him, but I don’t think even he was sharing 100 items a day. (I have since unblocked him.)
So how did we get to that number? Well, it turns out Google was counting every potential recipient of that message. A single message from Scoble today would count 240,000 times toward that number. That’s preposterous.
Google is by no means alone in how it plays with numbers. This deception happens nearly every day and is especially rampant in Silicon Valley where new business models are created and standard metrics aren’t always available. It also reflects the optimistic nature of the Valley. We want to see exponential growth. We see hockey sticks everywhere. Even worse, these statistics get thrown around in the echo chamber and presented as fact. And as they get reblogged and retweeted, they lose the disclaimers that made them technically true in the first place.
Every time I see a statistic, I try to figure out how much it was tortured. I want to know what it really means as opposed to what the person who is telling me the stat wants me to think it means.
I recently had a conversation with a PR person about a chart his company had put out and my coverage of it. He pointed out that I called them on the correct interpretation of the chart. His defense was that if you read the precise definition that they gave with the chart, it was accurate. They didn’t lie. It wasn’t their fault that most journalists interpreted it wrong. He’s partly right — journalists need to do a better job of understanding statistics. But the chart was clearly designed to generate the kind of coverage it generated.
Here are other common ways people deceive us with statistics:
Percentage growth. Triple and quadruple digit growth rates are also something companies like to tout. It amazes me that investors never seem to learn the lesson that you can’t have 4,000% growth forever. At the extreme high end, you’re bound by the population of the Earth. If, however, your startup has discovered life on Mars, I’m very interested in talking to you.
“Registered” users. Yes, let’s count the 14 million people who tried your product and decided they don’t ever want to see it again.
Percentages go both ways. A press release or analysis will often include the most favorable impression of a percentage. In a debate on Groupon, a “Pro” is that “20 percent of deal users became repeat buyers.” Is the glass 20 percent full or 80 percent empty? I think the fact that it’s 80 percent empty is more important.
Context matters. Big numbers are often reported without context. Foursquare announced that Starwood Hotels has awarded 10 million Starpoints through its check-in promotion. But is that a big deal? I have more than 400,000 points in my account. (I travel more than most people, but even that is small compared to last year’s top earner, who earned 19.7 million points in just the year.)
In Q4, Apple reported that iPad sales for the quarter came in at $6.9 billion. Google’s revenues for the quarter were $10.6 billion. With just one product, launched less than 2 years ago, Apple did 2/3 the revenue of all of Google. That’s a BFD!
Infographics are often dressed up lies. There’s a reason for the current infographic craze: unlike text, it’s not easy for journalists to break them apart and reformat them. You either include them or you don’t. The numbers in this infographic from Crowd Seats all seemed off to me. They seemed way too high for a company I’ve barely heard of. Turns out they were: despite the Crowd Seats logo plastered all over the infographic, the deals they cite were Groupons. (See my comment on that post for other issues with the numbers.)
An industry that regularly complains about poor math education in this country should do better when it comes to the statistics it uses. Using deceiving numbers leads to a race to the bottom where companies use crappier and crappier numbers to look more and more impressive. Companies that want to be forthright can’t — otherwise they look weak and unsuccessful. That’s bad for the startup ecosystem. It also keeps companies focused on vanity metrics to impress the press. That’s largely irrelevant.
Startups should spend their time building great products that people want to use, not just sign up for.
Rocky Agrawal is an analyst focused on the intersection of local, social, and mobile. He is a principal analyst at reDesign mobile. Previously, he launched local and mobile products for Microsoft and AOL. He blogs at http://blog.agrawals.org and tweets at @rakeshlobster.
Salesman photo via ShutterStock
VB's research team is studying web-personalization... Chime in here, and we’ll share the results.