Carriers can't handle ballooning smartphone traffic alone

May 3, 2010 | Ed Robinson

Ed Robinson is the CEO of Aptimize, a company that produces software to accelerate websites.

It’s a familiar feeling for many of us. Your excitement over your new iPhone rapidly turns to anger as you battle to connect to the internet over the AT&T network. It’s like buying a shiny new sports car and getting stuck in rush hour traffic when you pull off the lot — you’re left feeling disappointed and frustrated.

AT&T’s uncapped data plans sound like a great idea. After all, everyone loves an all-you-can-eat buffet. But in this case there are too many patrons and not enough food. AT&T’s data traffic has increased by 5000% over the past three years.

And if you haven’t heard – the wait for the iPad 3G is over, so the AT&T network is in for a world of hurt.

How can any carrier hope to keep up with this increase in demand?

It’s not the infrastructure, stupid

AT&T can invest as heavily as they want in infrastructure — but they’re never going to outpace Apple’s ability to sell iPhones, pure and simple. It’s not ultimately an infrastructure challenge –the infrastructure battle has already been lost; it was yesterday’s problem. Today’s problem is a much more sophisticated one with non-trivial scaling challenges. It’s definitely not something you can just throw more horsepower at — we’ve seen AT&T try that already, and they keep coming up short.

As a result of these shortcomings, and as human beings have always done, we have adapted our behaviors to fit the situation – we avoid relying fully on using our iPhones for real-time data connections, especially at conferences and sporting events where we’re competing with every other iPhone. Everyone has a backup plan, a set of last-resorts that they keep in mind in case AT&T goes down or underperforms (Verizon MiFi, anyone?).

Is this just short-sightedness by AT&T? Or should Apple allow people to use other carriers?

At the heart of the issue is a very obvious problem of logistics. Like having too many cars on the freeway, AT&T’s network is saturated with iPhone traffic, and every new iPhone contributes to the problem.

If Apple breaks its exclusive carrier arrangement with AT&T, this will immediately provide some relief as the load is spread amongst other networks, but that’s only a temporary solution. For a more permanent fix we need to look at broader trends in the market.

There’s a (bloated) app for that

Most people would agree that the iPhone is more like a desktop computer packed into a convenient form-factor than a traditional “smart phone”. The iPhone encourages us to do more, explore more, to build our digital lives around its simple, convenient interface.

But the iPhone’s features are ultimately the heart of the problem. Whereas older smartphones had very simple browsers that limited what you could do, the iPhone has a full-featured Safari browser that uses considerable bandwidth. On top of the full browser is the easy-to-use YouTube app for watching videos — which wastes bandwidth like a Hummer wastes gas.

The iPhone is a network killer not because there are so many of them in use, but because its constituent apps are extremely inefficient in their use of the network.

Blind leading the blind

AT&T is just the beginning of a potentially larger and more impactful long-term trend.

Instead of learning from the iPhone’s inefficient use of the network, other device makers and carriers are copying Apple. Microsoft’s Windows 7 Phone releases later this year with a richer interface, including a full browser and video support. In an attempt to keep up, Nokia is also adding more multimedia features to its line-up. You can expect to see a barrage of new devices in the near future, each with more ability to eat bandwidth than the last.

But none of the new efforts are future-proofed against eventual (indeed unavoidable) network outages due to app-level inefficiencies.

And back to the present moment, cellular networks are already struggling to keep up. In the last six months, the network outages that affected AT&T also hit Verizon in the USA, and O2 in the UK.

We are rapidly buying new, more capable phones, and the networks are offering cheaper and cheaper data plans (even as their own ability to meet demand for data is collapsing). The all-you-can eat buffet is now open 24 hours, and soon there will be nothing left to eat.

Just what the doctor ordered

Think about how urban planners go about fixing rush-hour traffic problems. Yes, they build more roads and make existing roads bigger, but they also look at what is actually traveling on the roads to determine where the real problem is.

They target single-occupancy vehicles as the main culprit because they are the least efficient use of the infrastructure. They encourage people to carpool or take public transportation rather than putting another car on the road with 4+ empty seats. It’s a great analogue for our current situation with cellular networks.

Yes, we need to build more network capacity (roads) and open the iPhone to other carriers to increase bandwidth availability and spread the load across many carriers. There’s no doubt these moves would help the problem. But to be more impactful, we also need to look at the hungry new breed of phones and applications to figure out how they can use bandwidth more efficiently. This is the real critical success factor and the only way to achieve a long-term solution that breaks us free from the “just throw more infrastructure at it” mentality.

Here’s an amazing fact to illustrate this point:

Most webpages are designed for a browser on a desktop computer with a screen size of 1024 x 768 pixels. The iPhone’s screen size is less than a quarter of this, yet it still downloads the entire page, then scales it to fit the screen. By merely doing this scaling on the server-side the data traffic could be reduced by more than half.

Similarly, most people visit the same sites over and over and over again. If the networks use a combination of far-future-expires caching, Gzip compression, image resampling and file merging on the server, the data traffic for most websites could be reduced 80-90% for mobile devices.

Similar techniques exist for videos — one of the causes of today’s data bloat on the networks. To get videos to stream in real-time, the servers and browsers use CODECs (COde – DECode) to thunk the full screen 29 FPS video into a smaller size that can be sent packet-by-packet to the mobile device. Some of the newer CODECs use very sophisticated techniques to detect changes from frame-to-frame and only transmit changes rather than each entire frame. Newer CODECs can also adjust quality — so, for example, if the network is busy, the video quality (and data) is reduced to ensure it streams without halting. Video processing has traditionally taken a lot of processing, but now we’re at a stage where the technology is reaching maturity and servers have enough power to stream more videos with less data.

We’re also at a point where the networks can intelligently partition bandwidth so everyone is ensured of fast access, even if the technology super adopter next door is attempting to watch eight videos at once.

The situation is, in fact, similar to the global energy crisis. We can’t just drill for more oil or build more networks – we need to become bandwidth-efficient. Building more networks isn’t the complete answer. We need to learn how to make better use of their existing capacity.

Innovations occur in the tech sector much faster than pipes can be dug into the ground. This is nothing new in fact. If you had your choice – would you rather have a software-based solution in 6 months, or a new set of double-wide pipes in 5 years? Dealing with this problem on servers and within the apps themselves is as big a part of the real-world solution as adding capacity to networks.

And if you’re not interested in dealing with the problem at the server level or the application level, I’m sure the carriers have a metered data package they’d like to sell to you.

Don’t miss MobileBeat 2010, VentureBeat’s conference on the future of mobile. The theme: “The year of the superphone and who will profit.” Now expanded to two days, MobileBeat 2010 will take place on July 12-13 at The Palace Hotel in San Francisco. Early-bird pricing is available until May 15. For complete conference details, or to apply for the MobileBeat Startup Competition, click here.