July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

Welcome back and Happy 2013. May it be a successful one for all our startup entrepreneurs.

We track over 40 sites in our Startup Shootout Monthly Index – collaboration sites, travel,
mobile and social gaming, social retail, social networking and digital entertainment – so we see
performance across startups in a variety of industry verticals. As we get to grips with January
and think of plans for the year ahead, we know this is a peak time for travel reservations. Cruise
anyone? Planning that family trip for Spring break? Let’s take a look at our travel category this
month and see how the companies we track are performing.

We focus on tracking startups, so the big names like Travelocity and Priceline aren’t part of
the mix. The index includes some sites that are designed to enhance the travel experience through
reviews or online community forums, as well as online travel agencies that let customers
search for online bookings of airfare, hotels, and rental cars. Two startups aiming to get your
travel dollars are JetSetter and Kayak. These sites compete against the big names in the
industry, but just as importantly, they compete against each other for customers willing to
try a new OTA site.

Optimizing the Online Travel Agency Experience

Interestingly, neither JetSetter nor Kayak is optimized for the tablet experience. Using these
sites on a device like the Apple iPad, customers will see the same web site as they get on a
desktop computer system. On the other hand, both JetSetter and Kayak provide a mobile-
optimized site specifically for smartphone users. Many of the travel sites in the Startup
Shootout Index do not have mobile-optimized sites designed to meet the screen challenges and
network limitations encountered by users of smartphones. So, right off the bat, these two sites
stand out.

JetSetter performs slightly better than Kayak on the desktop environment while Kayak does
better on mobile smartphone devices such as the Apple iPhone 5. Though JetSetter is slower
than Kayak by only a few seconds, this equals a 27% slower experience for their customers on
smartphones. What can we learn from these two sites? Why is one faster than the other?

Kayak has far fewer new HTTP requests – about 14 compared to the 33 on JetSetter. Because
mobile 3G networks have high latency, each HTTP request can introduce delay to the user
perception of the page load. Keynote recommends that companies focused on mobile-
optimized sites should aim for page designs with 10 or fewer new HTTP requests. Kayak has
optimized its site to use CSS sprites, for example, loading multiple small icons in a single
image instead of loading several small image files. Keeping the element count low is a great
competitive advantage for Kayak on mobile.

In addition to fewer new HTTP requests, the Kayak mobile home page also has many fewer
bytes of data on the page. Both JetSetter and Kayak have more than 100 KBytes of page
content, but the JetSetter site exceeds 800 Kbytes. JetSetter also has a very heavy single
JavaScript file – one file that is over 120 Kbytes. It is typically taking three to five seconds to load
this one file. Here at Keynote we recommend 50 Kbytes to 100 Kbytes (or less) to avoid latency
issues.

Besides being slow, this one file creates another major performance problem on the page.
Mobile browsers can “block” when loading JavaScript files. In order to make sure that scripts
files load in a predictable order, browsers often stop loading anything else in parallel while a
script file is loading. Properly implemented, JavaScript does not need to block during downloads.
It can be set up to allow simultaneous downloads. Unlike JetSetter, the JavaScript on the Kayak
site is not blocking. This is one of the big reasons for the 27% speed Kayak has on smartphones
over JetSetter.

On the tablet, these two online travel agency sites are performing very slowly. Over 3G mobile
network connections, Kayak takes just over 24 seconds on average to load. JetSetter takes
just under 30 seconds, making it approximately 21% slower on tablet than Kayak. Neither site
is making a real effort to improve the user experience for tablet visitors on mobile network
connections (or other slower network connections).

Conclusion

Overall, we’re surprised by how many startups we track – especially in a category like
travel – are not yet offering smartphone-optimized sites. We realize this development isn’t
cheap, but for startups trying to capture the interest of consumers willing to try something
new, the smartphone browser is a place where your site can stand out. In this respect, Kayak
and JetSetter set themselves apart from their startup competition with sites optimized for the
smartphone browser. But as we look more closely, Kayak’s site better adheres to Keynote’s best
practices for page weight, element count, and Javascript implementation. Once again, it just
goes to show the potential room for improvement when delivering sites across desktop, tablet
and smartphone environments.

Ken Harker is the mobile evangelist at Keynote Systems.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

We will hazard a guess that over the holidays, a lot of people will receive brand new smartphones and tablets. And while they are relaxing during the vacation, chances are they will check out some of the latest digital entertainment sites.

At Keynote, we love how digital entertainment sites are bringing a wide range of music, video and podcasts to the device of your choice and that they are adding social media capabilities to recommend more content that you might like and connect you with like-minded people.

We track a number of companies in the digital entertainment category of the Keynote Startup Shootout Monthly Index and thought this would be an opportune time to see how they are performing. Let’s see how successful these startups are at delivering the best practices across three screens – desktops, smartphones and tablets.

We test our startups using IE9 on a desktop machine, and across the Apple iPhone 4 and the Apple iPad 2 connected to live mobile networks.

First, It’s All About the Page Loads

Keynote recommends the industry best practice for page loads on a smartphone be under five seconds. Among the startups in the Keynote Startup Shootout index, only Spotify and rdio manage to deliver that. Project Playlist took a whopping 36.4 seconds on average in November!

Even on the desktop, where a response time of two to three seconds is the norm, the Spotify home page takes a sluggish 9.62 seconds on average. These entertainment sites clearly have a lot of work to do in order to improve their mobile performance and strengthen their competitive position in the market.

We decided to take a closer look at SoundCloud, as it offers some great examples of what to do and what not to do for the smartphone environment. Overall, we found that SoundCloud has a site optimized for a smartphone screen (good), but not optimized for the mobile 3G networks (bad). Let’s take a look at what some of these examples look like.

One of the ways that the SoundCloud smartphone site follows good practices is loading image elements using data URIs (Uniform Resource Identifier). Data URIs let the website include images as encoded text strings directly in the HTML for the page, avoiding extra HTTP requests for the image content. This enables the display of multiple images from one download. It’s a very nice move – a much better use of bandwidth, removing the need for about 10 external requests on the mobile home page.

Another way SoundCloud reduces the number of network requests on the mobile home page is with a technique called CSS sprites. CSS sprites combine several small images into one large image and rely on CSS rules to display portions of that larger image where they are needed on the page. A little more complicated than Data URIs, CSS sprites have some advantages for images that need to be cached and re-used on other pages on the site. These are all good examples of optimization.

 Second, It’s All About the Page Loads

Now let’s look at some areas in need of improvement on the SoundCloud smartphone site. A key issue is redirection from the initial page request to the base page HTML call. The first redirection is to an (unnecessary) secure domain and then once it recognizes the visitor as a mobile user, on to a non-secure mobile domain. These are the type of steps that can add a full second to the delay that users experience before the page begins to load.

Another interesting item which affects response time is analytics tags, such as Google Analytics. At SoundCloud we see that analytics tags are being loaded before some critical page content is requested. Not only does this add to the delay, but also adds to the risk of causing a problem. If a third party service has any performance or availability issues, it could actually block the page from loading. Better to load these third party services after the critical page content has loaded.

It also seems SoundCloud has made a conscious decision to display a large amount of content on the mobile home page – a total of 25 thumbnail images are displayed in a list. The amount of dynamic content on that opening page slows response time down with 25 round trips required over the low-speed, high-latency 3G network. This would be acceptable for site visitors on WiFi connections, but on 3G mobile network connections it can have a big impact. Although SoundCloud has optimized the non-dynamic image content on the page for mobile performance, the decision to send so much dynamic image content on top of it results in slow page loads for the 3G mobile site visitor.

By contrast, we see that competitor Spotify has very limited dynamic content on its home page– and hence its nifty 4.67 second smartphone response time, or better still rdio at 3.9 seconds.

In Conclusion

Some of these startups will no doubt get a visit over the holidays, so let’s hope they can deliver a satisfying, initial customer experience – across multiple devices – before the consumer moves on to faster or easier to view sites. Obviously our advice is to do testing scenarios on a variety of mobile devices and across different provider networks, and then monitor end user experience for better decision making, competitiveness and growth.

See you back in 2013!

Ken Harker is the mobile evangelist at Keynote Systems.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

For online businesses, executing a mobile strategy is still an act of experimentation. Unlike the desktop web, the ingredients that make a recipe for mobile success are being tweaked by companies as large and small alike.

When it comes to mobile strategies, this month’s Startup Shootout Collaboration index provides a wide range of examples. Some companies do not provide mobile-optimized (“m.”) sites for their services on mobile devices. One company even goes so far as to refuse an iPhone user to browse its website. However, it is notable that the three leaders in performance across three screens (Dropbox, Box, YouSendIt), are also the three leaders in the collaboration space itself. Dropbox and Box.net appear to be the most mobile-friendly. Coincidence? Let’s see.

When looking at the index, we noticed that startups leverage mobile in ways that directly benefit the interest of their user strategy. Since collaboration tools are generally used as productivity applications, enabling a positive user experience across 3 screens (desktop, smartphone and tablet) can be critical, especially for enterprise use that may include remote workers.

Here’s how the hottest startups in collaboration are managing the mobile aspect of their online experience:

Dropbox

The clear market share leader in the collaboration category keeps all their sites simple and clean. From the mobile browser, visitors are directly pointed towards the login and then to download their files. Dropbox has pruned all unnecessary objects and keeps the interface fast and simple across all 3 screens. Dropbox maintains the fastest average speed across the desktop, smartphone and tablet screens at 9.84 seconds.

Box.net

As another top contender, Box.net also does a great job when optimizing for its mobile users. It’s clear that Box.net wants mobile users to be able to access files and collaborate with others not only while at the desktop but on-the-go. Box.net asks mobile users if they would like to download their iOS app, but also gives them the option to continue to a mobile website. As a result, Box.net performs well across each of the 3 screens, with only slightly slower performance than Dropbox.

YouSendIt

YouSendIt does not offer their users a mobile optimized site. Instead, they prompt the user to download their iOS app (like Box.net) or continue on the full (desktop) website. While this is not optimal, it does give both new users and existing customers reasonable options for either learning about the service or interacting with it directly.

The other collaboration companies haven’t quite figured out the three screen divide as well as the top competitors. One example is Springpad whose mobile interface leaves the user experience with much to be desired and is frankly a bit confusing.

Springpad.com

It looks like Springpad has developed a site for its iPhone users and tries to push those users seeking www.springpad.com directly to Springpad’s iOS application using the “springpad://” command (as seen below).

If you have installed Springpad app on your iPhone, it will automatically open up the application. But this mobile strategy doesn’t work for new visitors to Springpad, since shortly after the site times out. It seems that Springpad’s mobile strategy is to push users to leverage its application rather than use the service through mobile web. A poor assumption for Springpad is to assume that when users attempt to access the site through Safari, they want to use the app. This approach denies mobile users access to a mobile web experience they may desire.

Directly opening a mobile app when browsing a web page sometimes makes sense in the context of certain tasks. But it only makes sense when you first check whether the native app is installed and then try to open it. As a result, Springpad’s performance in the index suffered greatly.

SugarSync, Huddle

SugarSync offers a downloadable app for their service on mobile, but smartphone and tablet users browsing their website receive the same experience as a desktop user which results in long average response times of more than 46 seconds on iPhone and 47 seconds on iPad.

Visitors to the Huddle website also receive the same experience across devices, but with better performance. Huddle’s service is available via downloadable app for iOS (iPhone and iPad), Android and Blackberry.

Overall Thoughts

Collaboration startups clearly want their customers to use an on-device app when on a smartphone or tablet, but taking a one-size-fits-all approach is bad for users and performance.

For the full data from this month’s Startup Shootout, see below.

Aaron Rudger is the senior manager of web & mobile performance and Venkatesh Giri is the mobile evangelist at Keynote Systems.