VentureBeat

[Editor's Note: Numerous local entrepreneurs and investors were quick to point out the problems with measuring internet traffic in China, we discovered during a VentureBeat trip to the country in May. To help explain the issues, we asked Victor Koo (pictured), the founder of leading online vide-sharing site Youku, to contribute his thoughts in the column below. See this recent guest column by the folks at Web2Asia for more details on the Chinese internet market.]

China now boasts the largest Internet market in the world, with over 250 million users. The industry has seen impressive growth across 10 years, and is in many respects as advanced as in any other country. But when it comes to Internet measurement standards, China is arguably stuck in the Dark Ages.

Just one indicator of the gravity of this problem is the fact that some industry pundits, advertising clients and venture capitalists still make decisions based on Alexa statistics. It’s an act of desperation: There just aren’t enough third-party traffic ranking alternatives in China. They rely on Alexa in full knowledge that there’s literally an industry here that thrives on gaming Alexa rankings. Do a search on Baidu or Google in Chinese and you can easily find a long list of such companies, all promising to raise your rankings to Top 100 or Top 500 for a reasonable price. One simply cannot rely on Alexa to get realistic website measurement or comparison in China.

This lack of transparency isn’t because there aren’t companies competing in Internet measurement in China. International companies such as Nielsen/Netratings have offered site census measurement for websites that can afford their services. But they can’t very well publish industry rankings to date, because not everyone in the Internet industry participates. Netratings conducted a site census on my company, Youku, last December, for example, but our competitors in the online video space have either not used their services or have not published their results. That makes it difficult to benchmark across the industry.

Besides site census, large-scale panel-based industry surveys are probably the best alternative. However, existing surveys by Shanghai-based iResearch—one of the leading Internet research companies in China—or by the U.S.-based firm comScore don’t measure traffic from Internet cafes, even though they are a significant part of Internet access in China: some 30 to 40 percent of users routinely access the Web via Internet cafés. On the bright side, I understand that iResearch has been testing their Internet cafe sample recently to provide a more complete picture, and the iResearch sample is already over 100,000 and growing. The Internet Society of China/Data Center of the Chinese Internet is another large sample-based study conducted regularly, so you do see organizations trying to improve the metrics situation in China.

Of course, a lack of understanding of what are the most important measures also plagues the industry. As more and more websites use Ajax technology and more new applications such as online video become increasingly mainstream in the Chinese market, the relevance of measures such as page views have become outdated. Leading Chinese research organizations such as iResearch are highlighting user time spent and unique visitors as the better benchmarks, but it takes a while for the media and advertising community to catch up to this thinking.

This lack of consensus both among publishers and media buyers has created a situation that’s favorable to the old picture-and-text portals who have dominated the Chinese Internet scene since its early days, namely portals Sina and my alma mater, Sohu. Lacking reliable cross-industry traffic data, media buyers tend to reflexively buy on the big, name brand portals that everyone knows, or on the handful of larger vertical portals focused on industries like automotive, real estate, and IT. The way that advertising is sold on the portals and on most of the verticals is still very Dark Ages: on “cost per time” basis, rather than on CPM. The shift away from it has been very slow.

There’s hope, though, that the Internet video industry will help to overcome that inertia and speed things up. We’re working with Internet Society of China on measurement standards, and starting to work with China Advertising Association to establish advertising standards.

Unlike text-and-picture sites, which have to fight one another for the small slice of pie that brands allocate to online advertising, Internet video sites are positioned to begin tapping into the far larger broadcast budgets. But before that can really start happening, media planners and buyers need reliable, transparent data from trustworthy third-party metrics providers gathered in a methodologically sound manner. The online video industry needs to offer advertisers and agencies standards — like ad units that don’t vary wildly from site to site — to enable scalability in video media buying. The two market leaders in China have between them an 80% market share already, so the market’s no longer fragmented, but standards are still missing. Meanwhile, the feedback we’ve gotten from media agencies and advertisers is good so far: They’re eager to move beyond the Dark-Age “cost per time” basis that saps them of their ability to really assess effectiveness. It’s up to us to work together to create an atmosphere of comfort and trust in video advertising. It’s in everyone’s interest.

[Editor's note: This is an Op-Ed piece by Rebecca A. Fannin, author of "Silicon Dragon-How China is Winning the Tech Race," and the international editor at the Asian Venture Capital Journal. She is the former international news editor at Red Herring. Her work also has been published in Worth, Inc. and Fast Company.]

It’s sad but true. Silicon Valley has its blinders on when it comes to China tech entrepreneurship. How do I know this? I’m here in the Valley this summer after several months in Asia’s major business capitals, where I’ve been preaching the theme of an emerging Silicon Dragon in China. From Hong Kong to Singapore to Tokyo to Beijing and Sydney, they get it. But here in the Valley, there’s push back to the idea that China poses a threat to the Valley’s continued tech dominance.

It took me a while to realize why. The Valley really is threatened and, rather than confront the challenge, prefers to remain in a state of denial. It’s easier in the short term.

The questions I’m asked here in the Valley mostly center on Chinese government politics, Internet censorship, counterfeiting, and pollution. Sure, these are big issues, but there’s very little curiosity about what new technologies are being developed in China. It’s just a given that there really aren’t any, so why bother asking?

Yet China has the fastest growing venture capital market in the world and the highest growth of new patent applications globally, too. China –- the nation that gave us the abacus, silks, paper and gunpowder — ranks eighth in the world for new patents, and is headed for third place internationally -– right behind the U.S. and Japan — for the most new patents.

The innovations are coming in mobile communications and the internet, where China has the largest markets worldwide, but also in cleantech, biotech and basic R&D. Dozens of home-grown Chinese companies have gone public in the U.S. as well, and with large market caps.

On Friday, the New York Times took note and wrote about the trend.

The Bay Area is the optimistic capital of the world, but sometimes we have to face harsh realities. Jerry Yang of Yahoo is. He told me he thinks it will only be one decade before China begins to rival the West for tech leadership. Leading venture investors such as David Chao of DCM, who has a rare east-west perspective, believes we’ve already seen several Steve Jobs-like characters in China. Joe Chen, who runs the Facebook of China -– Xiaonei.com –- is one he names. Fan Zhang of Sequoia Capital China observes that Chinese entrepreneurs are very quickly moving from copycats of successful U.S. technologies to inventors. Jim Breyer of Accel says the Chinese entrepreneurs make their U.S. counterparts look like they need to be on steroids to compete.

These thought leaders understand that the next, new thing might NOT come from the Valley. Yet, the Valley prefers to largely ignore this important trend. At two major technology conferences here in the Valley in late July, I observed that neither one had more than a bare mention of what’s going on across the Pacific. But there was a lot of talk about Twitter and the iPhone. This is even more surprising when you consider what a creative force so many China-born engineers have had in helping to develop our leading tech brands.

You have to be a futurist to believe that China could be developing from a low-cost manufacturer to a high-tech inventor. In fact, the most responsive audience I’ve had so far was in Columbus, Ohio, where I spoke to a group of futurists! As I head for Beijing and the Olympics this week, I can’t help but think this fast emerging tech trend in China is a wake-up call for the Valley.

[Editor's Note: Silicon Valley venture firms and technology companies have been active in China for years, but with mixed results. Domestic search company Baidu has beat out Google for much of the market, local social networks like tencent.com and Xiaonei appear to have the upper hand over international competitors like MySpace and Facebook. Similar trends can be seen across most segments of online Chinese markets. Where does that leave e-commerce in the country? George Godula, Markus Fuhrmann and Michael Hohenwarter, from a China based market expansion service provider called  Web2Asia, have been studying the question. This is the first in a three-post series from them on the state of the internet market in China.]

Disclosure: Web2Asia. recently paid for VentureBeat to visit China and meet some of these companies first-hand.

China has the largest internet population in the world — its 230 million users nearly equals the size of the entire U.S. population. If last year is any indication, the number of Chinese users will continue to rise quickly: In 2007, the number of users increased by 73 million to 210 million, representing a growth rate of 53.3 percent, according to the China Internet Network Information Center (CNNIC).

Driven by breakneck economic development, web access is quickly coming to more and more of the country’s 1.3 billion people — and traffic to e-commerce sites is booming (even if transactions aren’t — more on that shortly).

But don’t let those numbers fool you. Even though internet use is spreading quickly, the current penetration rate of 16 percent is still 3.1 percentage points lower than the world average of 19.1 percent and far below advanced internet nations such as Norway (88 percent), the U.S. (71.7 percent), South Korea (71.2 percent), and Germany (64.6 percent). In metropolises like Beijing and Shanghai, however, the penetration rate reaches almost 50 percent.

The tremendous growth in user numbers cited by the CNNIC is also largely due to many new connections in rural areas. In 2007 the growth rate in rural areas even exceeded the 100 percent mark, reaching 127.7 percent. By the end of 2007 rural internet users amounted to 52.65 million, and out of all new users last year, 40 percent, or 29.17 million, are from rural areas, according to the CNNIC.

Due to the spread of powerful broadband connections, a surge of various internet applications can be observed. At the end of 2007, 77.6 percent of Chinese users had access to the internet via broadband connections. 50.4 million users have also chosen to access the internet via mobile phones apart from other internet access methods. The majority of internet users, 67.3 percent, access the internet from home. However, internet cafes still get a lot of business and are important supplementary access spots, too — 33.9 percent of all users use these facilities to go online. Only 24.3 percent of users access the internet at work. Internet cafes are important places to promote online games, videos and movies, as the crowd there is mostly young and educated.

The internet is used differently in China than in the U.S. and Europe. The following applications are rated according to their popularity in descending order based on a January report by the CNNIC

1. Music
2. Instant Messaging
3. Video
4. News
5. Search Engines
6. Online Games
7. E-mail

The popularity of music, videos and games demonstrates the entertainment function of the internet in China. Online music and instant messaging are especially interesting to people with a lower educational background, according to the CNNIC, which could explain the huge increase in the number of users in rural areas.

Chinese internet users are also mainly young people. 18-24 year olds, especially students, represent 31.8 percent (PDF) of total user numbers. In contrast to the overall population, the internet user community has a rather strong educational background. However, in recent years we’ve seen an expansion into groups with a lower educational background. Since 1999 the share of users with a background of higher education dropped from 86 percent to 36.2 percent (PDF). This is likely correlated with the spread of the internet into rural areas where the educational background of the population is still very low.

As we indicated above, though, growing internet traffic in China hasn’t resulted in growing e-commerce. A brand new CNNIC report issued at the end of June, which covers 19 economically developed cities and provides the most authoritative figures on e-commerce in recent years, shows that only 25 percent of Chinese internet users shop online. In comparison, up to 80 percent of U.S. and European internet users purchase goods over the internet.

Chinese online shoppers are characterized by a strong educational background — the higher the education level, the higher the online shopping rate. 56.5 percent of all internet users with higher education use the internet to make purchases online. In addition, over 80 percent of online shoppers reside in urban areas and mostly work in joint ventures or foreign-invested enterprises.

The city with the highest online shopping rate is Shanghai with 45.2 percent, followed by Beijing with 38.9 percent and Guangzhou with 31.9 percent. The other cities under survey reached an online shopping rate of 21.6 percent. In economically less developed areas, the online shopping rate is rather low, since online payment is not very popular and significant problems remain with logistics in these areas.

Another interesting observation is that the longer users have had access to the internet, the higher their online shopping rate is. For example, 42.4 percent of all users who had access to the internet before 1999 shop online, while the rate among users who discovered the online world after 2007 reaches only 5.7 percent. This is the main reason why the online shopping rate in tier-one cities is significantly higher than in the rest of China.

The numbers, of course, are going up. According to the annual China Internet Research Center (iResearch) report, last year 55 million Chinese internet users shopped online for a total of $8.25 billion (including consumer-to-consumer and business-to-consumer transaction volume, excluding online travel bookings). This represents an increase of $4.3 billion over 2006, and the report projected that online spending will reach $58.5 billion in 2011.

In the first quarter of 2008, China’s online shopping transactions amounted to $3.46 billion, which means a 151.8 percent year-over-year increase. The number of online shoppers is expected to hit 75 million at the end of the year.

Yet the new CNNIC report shows the level of online spending remains humble and amounted to only around 0.6 percent of total retail spending in 2007. The average spending per online consumer was about $150 last year. In the first half of 2008, total online shopping expenditures amounted to $2.36 billion in the 19 cities under survey. Online spending of users from Shanghai has reached $160 in the first six months of 2008 and marks the highest level of online spending in China. The income of online shoppers in general is higher than the average income of regular internet users. Internet users with a monthly income above RMB 2,000 amounted to 26 percent, whereas the same is true for more than 50 percent of online shoppers.

Overall, the most popular online goods are clothes and accessories, followed by books and audio/video products, and cosmetics and jewelry. Different user groups buy different products. Female shoppers focus on cosmetics and jewelry (73.1 percent vs. 26.9 percent of men), whereas male online shoppers go for digital products (73 percent vs. 27 percent of women), all kinds of recharge cards (77.7 percent vs. 22.3 percent of women) and computers and accessories (81.6 percent vs. 18.4 percent of women).

Online shoppers have accepted online shopping as a regular internet application. However, major obstacles limiting growth in Chinese e-commerce remain, including a lack of trust in online payment systems, a fear of credit card and online sales fraud, and a fear of website ID information theft. Also a factor is a widespread problem of counterfeit goods. Nevertheless, the rapid growth of internet penetration and e-commerce in China is proof that online behavior can change quickly.

79.7 percent of online shoppers in the major 19 Chinese cities did not encounter any unpleasant experiences, according to the latest CNNIC report. Users who already encountered a negative experience say that it was related to one of two main problems — quality and logistics. Users complain that the delivered products sometimes do not match the picture in the product description, are fake or are damaged on delivery. Concerning logistics, shoppers complain mainly about overly long delivery times. A strengthened logistics management system could increase customer satisfaction substantially.

Online shoppers in the major urban areas also largely adopted electronic means of payment when purchasing goods online. 71.3 percent of shoppers use electronic payment forms when purchasing goods online. Cash on delivery is still very popular with 35.7 percent of users claiming that they make use of this payment form. Some 14.7 percent of online shoppers made use of postal money orders. Among the means of electronic payment 76.2 percent of online shoppers use Alipay, a third-party payment solution (”China’s PayPal“). 32.5 percent state they pay via e-bank accounts, and 11.6 percent claim to pay by credit card. Alipay’s western counterpart, PayPal, is only used by 0.8 percent of online shoppers who pay electronically. Credit card use is still very underdeveloped in China, and by the end of 2007 only 70 million credit cards had been issued, according to the People’s Bank of China. Similarly, western payment services such as PayPal are far less popular than local provider Alipay. The most striking difference in payments, however, is the high use of cash on delivery for e-commerce payments in China — labor is still so inexpensive that many strong local providers can afford huge workforces for delivery and payment collection.

George Godula is managing director of MH | direkt Asia Co. and founder of Web2Asia. Markus Fuhrmann is cofounder of Web2Asia Michael Hohenwarter is research analyst at Web2Asia. MH | direkt is an international direct marketing and fulfillment provider that established Web2Asia in 2008 to support Western internet and e-commerce companies in their expansion into China, Korea and Japan.

[photo credit: minimotosgo.com]

This is an contributor column by Anand Rajaraman, a co-founder of Kosmix, and founding partner of Cambrian Ventures. Cambrian Ventures in an investor in Webaroo, the company behind SMS GupShup, which he writes about below. He explores the technology and business of search, social media, and advertising on his blog Datawocky.

I’m a big fan of Twitter. I’ve been using the service only for a short time, and that’s been enough to make me an addict. Yet amazing though Twitter’s growth is, it pales in comparison to the adoption rate for another microblogging service: India’s SMS GupShup. Twitter has over 1 million users, by TechCrunch’s estimates (Twitter declined to give out absolute numbers). SMS GupShup has amassed over 7 million in the shorter time since its launch. Earlier this week on my blog, I explained how SMS GupShup has managed to scale to higher usage without missing a beat thanks to the way they’ve architected the service. But the bigger question is, why is SMS GupShup bigger and growing faster?

Think about the biggest stories we’ve heard about Twitter in the past couple of months:

• Student twitters his way out of Egyptian jail.
• Twitter breaks the news about the China earthquake way ahead of the mainstream press.
• Twitter helps cyclone Nargis victims in Burma.
  

Notice a pattern here? Many of these stories happened in developing (third world) countries, not in the US. (There have been interesting US stories too, of course, like twittering California wildfires.) This leads me to my hypothesis: Microblogging is a nice-to-have in developed economies, like the US. It’s a must-have in developing economies like India, China, and Egypt.

In essence, microblogging is semi-synchronous publish-subscribe messaging. It’s publish-subscribe because it decouples senders and their reader(s), who can choose which senders to follow at any point in time. It is semi-synchronous because readers can choose either to follow it synchronously (via various desktop tools, or their mobiles), or read it later. In the Western world, the penetration of PCs is almost universal, so we have other PC-dependent messaging options such as blogging (asynchronous publish-subscribe); email (asynchronous point-to-point); instant messaging (synchronous point-to-point). Yes, none of them offers quite what Twitter does, but the majority of people in the majority of situations can make do with the conventional options.

Contrast this with the situation in third-world nations: PC penetration is incredibly low, but mobile penetration is incredibly high. For example, India has about 40 million PCs but 10 times as many cell phones. This makes short text messages sent via SMS the main written communication mechanism. Blogging, email, and IM are just not options, so microblogging becomes the main form of publishing, communication, and self-expression.

The rapid growth of SMS GupShup certainly testifies to the latent need for the service in India. The company’s employees created the first set of groups and invited their friends, who were young and tech-savvy. Usage took off rapidly from there. The first set of groups centered on humor (including some quintessentially Indian joke varieties that don’t translate well into English), technology, horoscopes, weather, and health. The service is very popular among college students, stockbrokers, clubs, and some large employers with distributed teams. User numbers have skyrocketed from just over 1 million in January to 7 million in June. Today over 10 million messages are sent every day over SMS GupShup. The largest group is the Sikh Network, whose members include 140,000 followers of the Sikh religion. Each day the coordinator of the group posts a quotation from the Guru Granth Sahib, the holy book of the Sikhs.

I’ll close with a vignette that illustrates the magnitude of the need. One day the GupShup spam control team noticed several messages that looked like gobbledygook to them. So they sent these suspected spammers account termination notices. They didn’t expect the response: messages not just from those senders but from many others, pleading with them not to terminate the accounts. It turns out the messages were in a language called Hmar, only spoken by some 65,000 tribal people living in the hilly regions of India’s northeast. There are now several Hmar groups on SMS GupShup; the tribal group sees this as a major communication channel. Being too small to attract mainstream media, the group also sees SMS GupShup as their main form of media and a way to save their language and culture from extinction as they assimilate into the Indian mainstream.

[Editor's note: We've already had one contributor from the mobile industry describe the opportunities that the new iPhone' built-in GPS service could provide to third-party mobile software developers. But GPS technology is still a work in progress. In the article below, Polaris Wireless Chief Executive Manlio Allegra, pictured, discusses issues with using GPS in dense, urban environments -- and solutions to the problem.]

With Apple’s announcement last week that the new 3G iPhone will be GPS-enabled, the media’s attention is once again on wireless location technology and the drive to improve performance for location-based services (LBS) applications. Once these capabilities are truly commercial, wireless carriers will be able to deliver timely, location-specific content to mobile users from real-time traffic reports and inventory tracking solutions to people finder and other concierge services.

However, GPS is not the technology that can get us there—at least not alone. Sure, GPS works wonders for most open sky applications like traffic navigation, but the technology fails to deliver high-accuracy locations in environments where end-users make most of the wireless calls—such as dense urban environments and indoors—due to obstructions from large buildings and walls. For this reason, the future of LBS cannot rely solely on GPS. Because the vast majority of today’s wireless usage is indoor and urban, GPS delivers a 50 percent solution (at best!), and here’s why.

1. Missing the L in LBS

Consistently pinpointing accurate location is essential for LBS, making sure consumers get relevant content in real-time. Most LBS applications that rely on GPS, Assisted-GPS (A-GPS) and their fall-back solution, Enhanced Cell-ID (E-CID), fail to reliably deliver high accuracy. These solutions can only produce urban and indoor accuracies within hundreds of meters, making the technologies undependable for delivering location-specific content city block to city block. Alternate solutions can determine location with a much higher degree of accuracy in challenging environments.

According to most research, more than 50 percent of cell phone calls are made in urban areas (cities) or indoors (office buildings, schools, malls, convention centers). Since the majority of wireless usage is concentrated in these dense settings, it is essential that wireless carriers use a location technology that is optimized for these environments. Unfortunately, GPS and A-GPS have serious limitations in these dense environments, because the satellite signals are too weak and corrupted by reflections from nearby buildings and walls. Location errors in these environments are more than 250 meters for many of the calls—errors that are larger than two city blocks.

2. Unacceptable Time to First Fix

Time is an essential ingredient in successfully providing LBS to consumers. Positioning latency—or Time to First Fix (TTFF)—determines how fast a service or content can be delivered to a mobile user. Time is of the essence because most users are typically on the move and need accurate, real-time and location-specific information. However, GPS can take up to a full minute or longer to make a first fix—an unacceptable delay—while non-GPS technologies can often determine a location in seconds. Customer satisfaction and the proliferation of LBS applications necessitates that information is provided in seconds. Consumers simply aren’t willing to wait minutes.

3. Throwing Satellites at the Problem

These severe GPS limitations in urban and indoor settings are not going to be solved in the foreseeable future. GPS solution providers are trying to address these problems by adding new satellite constellations (Galileo) to their receivers, hoping that density will yield better coverage. Unfortunately, they are finding that adding satellites only leads to more complexity and higher cost in the handset, without adding much indoor performance.

Satellites simply cannot provide the level of accuracy needed in those dense urban and indoor environments—no matter how many birds are flying overhead and how much power is added to the satellite signals. The fundamental problem is that GPS satellites fly at over 20,000 km from the ground, making the signals exceedingly weak. Satellites also have limited power budgets due to their solar power supplies. Even when the weak signals can be received, they are corrupted by reflections from nearby buildings and walls. And since GPS signals can never effectively penetrate deep inside buildings another solution is needed.

A More Reliable, Accurate Wireless Location Solution

Given these inherent drawbacks of GPS, Apple decided to enhance its GPS capabilities with a wireless location solution from Skyhook Wireless based on WiFi Access Points (AP). However, location technologies based on WiFi continue to lack complete coverage in dense environments, failing to cover ALL buildings in a wireless network—a capability that will not be added in the foreseeable future. How many times have you tried to log onto the Internet via a public WiFi network and couldn’t receive a strong signal? How many times have you gone into a building and couldn’t even get a WiFi signal? The coverage simply isn’t there, and that is an unreliability that LBS (or Public Safety) applications cannot afford.

In addition, the AP database on which the Skyhook technology relies is highly variable, as consumers and IT departments change AP locations all the time while lacking a reliable mechanism to keep track of the changes. The root of the problem is that WiFi is an unlicensed band that is unplanned, unmanaged and dynamic—essentially an ad hoc network. It is nearly impossible to produce a reliable and accurate location in a timely manner from the chaos—an absolute necessity for the proliferation of LBS.

Fortunately, there are carrier-grade solutions out there that are much better suited to LBS applications, including solutions that take a hybrid approach by leveraging multiple wireless location technologies to pinpoint highly accurate positions. These technologies range from hardware-based Uplink Time Difference of Arrival (U-TDOA) to Wireless Location Signatures (WLS). Polaris Wireless, Andrew Corp. and TruePosition are the leaders in this hybrid approach, building a combination of technologies that synergistically work together to accurately pinpoint the location of handsets. However, there are clear advantages between these competing technologies.

Andrew Corp.’s Geometrix Mobile Location Center (MLC) system uses a combination of U-TDOA, A-GPS and E-CID to determine handset location. TruePosition also incorporates multiple location technologies for its Position Determining Equipment (PDE) solutions, including a similar combination of U-TDOA, A-GPS and E-CID. These approaches with U-TDOA are inherently complex and costly to deploy and manage, with new radio hardware required at the base stations.

Taking what we have found to be a simpler and more cost-effective hybrid approach, Polaris Wireless has developed a reliable network-based solution based on Radio Frequency (RF) Pattern-Matching and A-GPS to determine handset location to a higher level of accuracy. The Polaris WLS Technology is a software-only, network-based solution that provides a much higher level of accuracy in dense and indoor environments than solutions that rely on other wireless location technologies or exclusively on A-GPS. Polaris is more reliable, commercially available, much easier to deploy and does not require new hardware or widespread buy-in from customers, making the initial deployments more scalable, cost-efficient and powerful. The technology has been successfully deployed in the US for public safety applications (E911) as well as for LBS applications.

This Polaris WLS technology is based on the principle that every location has a unique radio frequency (RF) signature. Like a fingerprint’s pattern of lines and swirls, a location can be identified by a unique set of values including measurements of neighboring cell signal strengths, time delay and other network parameters. Polaris capitalizes on this substantial suite of existing network measurement information, combining it with A-GPS to determine accurate locations to within 50 meters.

While GPS-enabled iPhones are certainly a step in the right direction, wireless carriers need to get past the hype surrounding A-GPS and look to more accurate and cost-efficient hybrid approaches that provides a rich user experience. Only then will the industry finally be able to truly capitalize on the power of location enabled applications.

Manlio Allegra is CEO of Polaris Wireless, a leading wireless location company that provides network-based software products for accurately determining the location of mobile phones.

When I started my first mobile internet company nine years ago, my partner and I were most excited by the potential for location-based applications. We started with maps and local information, but our vision was much broader: A location-aware phone connected to the internet could effectively extend your senses, making you aware of everything that’s going on in the world around you. Our greatest fear was that we would be too late to market.

Nine years later, we are still talking about the potential for location-based applications. What happened? The answer has a lot to do with the structure of the wireless industry. But the industry is changing fast, and I believe all that potential might finally be realized.

Since the days of WAP, wireless carriers have lived in fear of being reduced to dumb pipes like their crazy old wireline uncles. What’s saved them so far is a difference in the economics.

PCs were always expensive relative to the cost of dial-up internet access, but phones have always been cheap compared to the cost of wireless service. Carriers can afford to give them away if customers sign a long-term contract; this in turn gives them complete control over the handset. Before the mobile internet it didn’t really matter, and carriers in other countries saved money by emphasizing prepaid service and letting the customer buy their own phone. But US carriers focused relentlessly on the subsidy model, which matched the buy-now-pay-later mentality of American consumers, and today 95 percent of cell phones in the US are given away or sold by a carrier. (Try to imagine what the web would be like if you had to buy your PC from AOL.)

The good news is that they accept responsibility if there’s something wrong with the device. The bad news is that they lock it down to maximize their revenues over the life of the contract. If there’s a native IM client, messages are treated like texts, not data. You can’t easily upload your photos to Flickr because they want you to pay a toll for picture messaging. You can’t back up your address book using any service but theirs. You can’t change the default browser, install Skype, or load your own music in some cases. Every event is a billable event. But the most tightly guarded APIs of all are for location.

Why? In public, the discussion centers on privacy and safety. When it comes to sharing your physical location, these are serious issues. But there are lots of ways to manage the risks: fine-grained privacy settings, everything private by default, application-level permissioning, strong parental controls. Yahoo’s FireEagle is one great example. No, the real issue is who gets paid for what.

Eric Carr from location-based service company Loopt recently wrote a great primer on location technologies for TechCrunch. The key thing to understand is that some of these technologies require no help from the carrier’s network, while others do. The solutions that leverage the carrier’s network have been on the market for ten years, widely deployed for five, and ought to be better, faster, and cheaper than solutions that require the handset to do all the work (autonomous GPS) or that depend on patchy WiFi coverage.

Carriers saw that they controlled the distribution of handsets, they controlled what got onto a handset, and they controlled access to location data even if a customer brought her own phone to the network — unless she paid more than $500 for a phone with autonomous GPS and WiFi. But they overplayed their hand. If they liked your idea for an application, they wanted you to launch it under their brand, support every phone in their lineup regardless of volume, and settle for a minority revenue share that you had to negotiate for your specific app, knowing that without a brand of your own you could be displaced by another vendor at any time.

Most startups never got a meeting. Most of the rest walked away from that deal. Some took the money, wrote lousy applications that performed badly, and walked away leaving carrier executives thinking that maybe there was ‘no there, there’ in location. And a handful have tried to make it work: Televigation, Wavemarket, Loopt, etc. It’s hard. Take Loopt: great team, great app, but they’ve been around for three years and raised almost $20 million and still most people can’t get Loopt on their phones.

Once again, the Valley is excited about location-based applications, this time because of the iPhone. Has anything really changed? Yes.

Apple tried and failed to break the subsidy model, leading one analyst to describe the iPhone 3G as a step backwards for open access. He’s wrong. The subsidy itself doesn’t matter. What matters is how much freedom developers have to build applications for the phone and to market those applications directly to consumers, and by that measure the new iPhone is a big step forward for the US market. Apple and AT&T have at last decided that the way to make money in mobile is to make the pie as large as possible while guaranteeing yourself a slice. For location apps in particular, the iPhone offers both A-GPS and triangulation based on both cell tower ID and WiFi MAC addresses. But developers don’t need to worry about the details — location is just another API call.

Consumers are much more likely to use applications on the iPhone than on any other device, Apple’s new store will probably be the most effective distribution channel for mobile apps ever launched, and Apple wants a flat 30 percent, regardless of which APIs you use and how often. Since Apple no longer gets a share of the monthly bill from AT&T, it has every incentive to help developers succeed. I’m betting that most of the restrictions that remain — such as no turn-by-turn directions apps — will be gone within a year.

Apple deserves credit for creating the device, but this new model was forced on it and on AT&T by consumers and third-party developers. Once normal people finally realized that the internet on your phone might actually be a good idea, they started complaining about the restrictions. Too bad it had to take nine years.

Jason Devitt (pictured above) was co-founder of Vindigo, one of the first companies in the US to develop mobile location-based applications for consumers, including MapQuest for mobile phones. Jason’s new company Skydeck (http://skydeck.com) helps consumers manage their cell phones. Skydeck is in private beta, but readers can join straight away by mentioning VentureBeat when they sign up here: http://skydeck.com/apply.

[Editor's note: Skydeck is one of the startups nominated for our MobileBeat 2008 conference, and you can vote for all the nominees here: http://venturebeat.com/mobilebeat-2008/companies/]

[Editor's note: This is an Op-Ed piece by Mike Hirshland, a General Partner with Polaris Venture Partners in Boston.]

The All Things Digital Conference — aka “D6″ –- last week in Carlsbad, CA was true to form. It’s one of the better events of the year. Host columnists Kara Swisher and Walt Mossberg were characteristically snarky and grumpy (respectively), and both the speaker and attendee list were platinum level.

While the real value was, as always, in the hallways, the conference sessions themselves had points both high and low. Rupert Murdoch took the gold medal as the best speaker, with Barry Diller a not too distant silver. Unlike some of the other rising and falling internet stars and corporate types, both Murdoch and Diller are old-line media moguls who built real, sustained business empires from scratch, and it shows. No BS, no talking points from PR handlers. Not afraid to say what they think, they have balls, they have gravitas, and it shows. Jerry Yang and Time Warner CEO Jeff Bewkes were totally lame and tied for last place. Facebook kid-CEO Mark Zuckerberg was kind of endearing in his youth and earnestness. Gates and Ballmer were, well, pretty standard Gates and Ballmer.

All the fun really was the schmooze-fest in the hallways. What makes D one of the very few “must attend” events for me is that it has a great mix of people: early stage entrepreneur/angel types like Loic LeMeur (Seesmic), Mark Pincus (Tribe, Zynga), Mike Jones (Userplane), Peter Pham (Photobucket, Billshrink) and Gregg Spiridellis (Jibjab); new and old media dealmakers like Wade Davis (Viacom), Mike Marquez, (CBS) Peter Levinsohn (Fox), Steve Wadsworth (Disney) and Megan Smith (Google), bloggers Rafat Ali, Om Malik, Mike Arrington and Matt Marshall, VCs like David Sze and David Hornik. For a geographically challenged deal guy like me, the value in having all these under one roof for a couple days is huge. (I tried selling one of these guys my portfolio for $5 billion, but he didn’t think it was quite the bargain I did. Maybe next year.)

While I neither met the next great startup to fund nor heard any brilliant insight that I will take away with me, there was at least one discussion that really has me thinking. Swisher made the standard quip that all of the currently popular Facebook apps seem so childish and trivial; where are the grown-up apps that have real value and become for Facebook what Word, Excel and Powerpoint became for Windows? While the true answer probably is that those apps don’t yet exist (I tend to agree that today’s hit FB apps are pretty trivial), Zuckerberg’s response included the germ of what I think is a pretty profound insight: we tend to think of killer apps as single monolithic applications that gazillions of people use, like Excel, email or search. On a socially networked web, though, we are also going to see lots of smaller mini-apps that standing alone have less usage and less utility, but in the aggregate have tons of usage and utility.

I don’t think the idea is fully formed yet, but I do think Zuckerberg is onto something.

And last, but certainly not least, I should ‘fess up that one of the other draws is that the conference is at the swank and sunny Four Seasons resort in Carlsbad, CA and perennially hands out kickass schwag. Who says work can’t be fun too? For more impressions on the conference, check out Scott Duke Harris’s story in the San Jose Mercury News.

Mike Hirshland (pictured) blogs as VCMike.

I have a very counterintuitive — and perhaps unfashionable — approach to innovation. But it works.

Instead of focusing on hot new waves of technology that may or may not flood the market with success, I concentrate on the oceans of opportunity that currently exist in staid and mature IT. There are five reasons why this Retro Innovation is the most lucrative innovation:

1. Old markets are huge. They’re also established and well understood by customers. So, if you can insert a better mousetrap that addresses the nagging customer pains that still exist in these well-developed arenas, you’ve got immediate gold. The market doesn’t have to grow from $20 million to $1 billion over time for your product to achieve scale, traction and acceptance; and customers don’t have to discover how and why your product matters and fits into their lives.

2. Venture capitalists and inventors are fixated on what’s new, next and on the horizon – the semantic Web, grid networking or social networking, for example. There isn’t a lot of interest in the tools and applications of the past, especially given the dominance of the market leaders in these categories, so innovation in this area faces less competition.

3. Old markets are often populated with executives and entrepreneurs who are victims of their own bias and stasis. So where you are competing, it’s with the “establishment,” which usually means dogmatic developers who have been left behind as the aggressive and truly creative players seek the next big thing. Clayton Christiansen tells us in “The Innovator’s Dilemma” that established products are inertially bound from obsoleting themselves.

4. Old markets actually lend themselves to seismic change if the right technology solutions are brought to bear. Conventional wisdom says that big markets with “good enough” technology are sometimes not worth pursuing because the customer’s pain and priorities simply aren’t high enough on the list for them to care about your better mouse trap. But I believe you can be a game-changer in these situations by offering a new technology paradigm. This doesn’t mean leveraging state-of-the-art technology to improve on the existing product; it means combining many fresh technologies to create a unique product that addresses old needs in an innovative way. For example, take what Pampers did to the cloth diaper in the 1960s, or what Post-It Notes did to the notepad in the 1980s, or what iPod did to MP3 Player in the 2000s.

Consider the surprising success of Zimbra, which offers an open-source email platform product, for instance. We all know that the email product market has been won and done many times over with the heaviest of heavyweights — Microsoft, IBM, Yahoo, Google and every telco and cable company on the planet — throwing their considerable weight around. And yet, Zimbra provided email in a different and essential enough way that one of the heavies, Yahoo, found the need to pay $350 million to bring the upstart into the fold, despite already having an email service. Zimbra’s killer twist was that it was open source and white label-able for large companies that wanted to provide email services to their clients. As a result, it enabled Yahoo to add business-class email to its consumer email services.

5. In old-market innovation there’s far less visibility, fewer spotlights, and much more room to truly experiment with the best solution before launching. Everybody is watching the social networking market like a hawk today, for example, but Zimbra almost snuck up on people because email is considered such a mature application. The bottom line is that you can quietly and efficiently learn a lot about what users like and don’t like in an old technology market, and then design your product accordingly. You can also test your solution more openly (with some disguise, of course) in these well-established and somewhat neglected arenas.

Innovation — as we all know — requires inspiration, creativity, discipline and luck. And to be successful at it, you have to pick your opportunities carefully. My view is that if you take your inspiration and creativity and pour them into meticulously selected markets that have established track records over time, you’ll eliminate the need for luck. Retro is definitely the way to go!

[Photo above: Flickr/strange wax]

Brent Frei is founder of Bellevue, Wash.-based Smartsheet.com, a privately-held Software as a Service (SaaS) provider.

[Note from Eric Eldon: I wrote a rather positive article this week on the proposed $10 billion California High-Speed Rail bond measure. If approved by the state's voters this November, the bond will lead to high-speed trains stretching from Sacramento and the Bay Area all the way down to San Diego. Martin Engel, a transportation commissioner for the City of Menlo Park, Calif. (a city that the train would run through), thinks its a terrible idea because it's so expensive and because it's solving the wrong problems. Here's his response.]

If I may, I would like to respond, more or less point by point to your article, Eric. Actually, the train ticket price that is most frequently quoted is $55 one way for the SF to LA route. That will be in 2030, according to the CHSRA [Ed. My source on fares is here]. What do you suppose that $55 will have risen to in twenty years, when these trains are finally running?

Train ticket sales today, for regular as well as high-speed trains worldwide, begin at twice the $55 amount for similar distances. Even not-so-zippy Acela (the Washington D.C. to Boston commuter train) is more costly. Many bloggers who’ve visited Europe or Japan will convey that information; that is, it’s expensive to ride those trains. And you doubtless know, all passenger rail systems are massively subsidized; so their expectations of profits are highly hypothetical.

You talk about the “value” of this project. A better way would be to talk about the “cost/benefit” ratio. You have to acknowledge that the costs will be staggering, regardless of where the money comes from. There are many examples of under-pricing (lowballing) infrastructure projects (Bay Bridge; Boston Big Dig; the Eurotunnel, for example). A Danish researcher, Bent Flyvbjerg, has published findings about this practice. My point is this: Is there a cost point for this project that makes it no longer worthwhile? Say, like the Iraq war. And, if so, what would that be?

If you become a student of this practice, you will discover that you cannot disagree with a cost projection of $100 billion (not the currently projected $42 billion). It will be all borrowed money, both from public state and federal treasuries and private ones. These debts require not only interest payments, but also a return on principle, like a mortgage. When you say that it will not all come from California, you are probably right. However, it is California that will have to repay these loans. Some say that it will help our economy. What if many of the investors are from overseas? Is being in hock to overseas investors such a good thing for the state’s economy?

Given the fare cost and the limited connections, the likelihood of lower income workers using this train as a mode of commuting is actually rather slim. Those people would prefer slower and lower cost modes of transit. Let us be clear here, whatever this train is, it is not mass transit. In large high-density areas such as the LA basin, it could serve commuters, but that is not the rhetoric or the intention of the developers.

Yes, they claim that it will create 450,000 new jobs and will require 300 thousand man-years to construct. The fact is, there is no way of predicting this. Half a million jobs will be created in the next fifty years — or not — based on large, macro-economic forces, the nature of the global and national economy, the well-being of California’s industrial and agricultural capacity and similar basic sea-changes. Were such job growth to take place, it would be presumptuous to give credit for such expansion to the creation of a north-south railroad system.

Speed of travel, the CHSRA claims, will be faster by train than plane. No matter what the obstacles are, that’s nonsense. Indeed, airports are becoming more efficient at moving people through the terminal obstacle course. At the same time, it is naive to believe that this train, and others, will not require far greater security measures than are now in place. We have been blessed by not having a post 9/11 terror attack. I suggest that with the next one, there will be a major clampdown, trains included.

Back to the $42 billion cost. Construction, which won’t begin for another several years, will take 10 years, more or less, to complete. Do you want to predict the construction costs 10 to 15 years out? You already know about sinking dollars, rising inflation world wide, fierce competition for construction materials, decline in credit availability, etc. The $42 billion is a number like the $55 train ticket. Even if it were true today, which it isn’t, these numbers will skyrocket. I say again, $100 billion total costs are not unreasonable.

A word about environmental benefits of the train. I suggest that they will not be anywhere near as great as claimed. Comparing “the train of the future” with the automobiles of the present is unfair. Planes and cars in the future will be more ecologically friendly. Electricity, on the other hand, may become dirtier (fossil-fuel powered plants) as our demands increase exponentially. This is a domain where it is almost too easy to overstate benefits in politically correct language.

Eric, you mention the increase in riders on other rail carriers. The CHRSA frequently predicts 117,000,000 passengers annually. They even paid a consulting firm to come up with those numbers. 117 million people is over a third of the entire U.S. population. How rational is that?

There are many more issues that need to be seen through lenses other than those provided by the train developers. Amazingly, very few people have been willing to seek out alternative perspectives or to challenge the rail authority’s assumptions.

Martin Engel was born in Germany, received an M.A. from Harvard, and a PhD from Syracuse. He served in the US Air Force. He has held faculty positions at Wayne State University, Carnegie-Mellon University and most recently at Stanford University. He was also a senior program officer at the US Department of Education, and a program director at Apple.

[Train route map via CHSRA]

[Editor's note: This is an OpEd piece by David Gal, an assistant professor of management at Northwestern University]

Last year, Facebook chief executive Mark Zuckerberg suggested that a PC operating system was the inspiration for Facebook’s new “Platform.” With Platform, anyone could write applications for Facebook. Facebook’s in-house applications would get no special treatment, he declared.

The analogy to an operating system is appealing. For many years Microsoft’s Windows operating system has benefited from the large number of applications written by outside developers. People buy Windows, not necessarily because it is the best operating system, but because it has the most applications. Like Microsoft, Facebook does not have a monopoly on great ideas nor unlimited bandwidth, and a platform ostensibly allows Facebook to leverage the talents of the entire developer community to its benefit.

However, Facebook’s most important strategic asset is not its community of developers but its network of users. Does Platform build or leverage this strategic asset? I believe Platform was intended to leverage the network, but, in fact, it squanders it, by fragmenting the network across a large number of applications in each application category. Enter a search on Facebook for any category of application, from dating to chess, and you are faced with a large number of choices.

But aren’t more applications, as the Microsoft analogy suggests, a good thing? Abstracting lessons through analogy can yield unexpected insights, but analogies can be stretched too far. A large number of applications is a benefit to Microsoft Windows, because being able to run the largest number of applications is an important source of value for users of an operating system. However, the most important source of value for users of a networked application is the number of relevant users using the same application. When multiple applications compete for users in a network, then fragmentation results, and an important source of value is frittered away.

Won’t Darwinian selection ensure that the best application in each category will prevail? Maybe, but probably not. Darwinian selection does not work so well in a networked world. Multiple competing applications may coexist in a category, leading to diminished overall adoption of the category. Moreover, those applications that come to dominate a category will not necessarily be the best or the best-maintained; rather, they will likely be the first to attract a large number of users in a particular category. This is because in a networked application, having a large number of users may outweigh any functional benefits that a competing application can provide.

Summary

To conclude, Facebook’s focus should be on building, maintaining, and leveraging its core strategic asset, its network, and Platform is not well-suited to these goals.

First, platform does little to build the network, because while a large number of applications is an important reason for people to buy Windows, a large number of relevant users is the most important reason for people to join Facebook. In fact, rather than attracting users to Facebook, most Facebook applications acquire users through Facebook (and then get to split revenues with Facebook).

Second, useful applications are important to maintain the network, but the most useful applications are those that have the largest number of relevant users, that are well-maintained, and that do not require users to share their personal data with third-party developers. Platform applications are unlikely to meet these criteria.

Lastly, leveraging the network implies using that asset to pursue applications from which Facebook can derive revenue, such as dating, jobs, fantasy sports, event-ticketing, auctions, etc. By relegating these applications to Platform and focusing on advertising revenue through impressions, the user-base is fractured, and the value of the network is squandered.