July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously. VentureBeat is Keynote’s exclusive media partner, so we’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

This month’s Keynote Startup Shootout Index continues to demonstrate that early-stage companies struggle to understand or address the challenges of delivering acceptable Web performance across three screens—desktop, tablet and smartphone. Across the Index, social networking companies offer the most robust performances across all three screens while social retailers lag behind.

With Tax Day just behind us, we thought we’d also look at performance across three screens for early-stage online tax and personal finance companies.

Keynote set up some custom testing for Freshbooks, Outright, and Mint.com specifically to see how these sites performed in the run-up to April 15. We took home page measurement averages from April 1-16 and instead of measuring the smartphone and tablet experience over a 3G wireless connection (as we do for the regular the Startup Index), we assumed that most users would be looking up their tax-related financials over a high speed connection like Wi-Fi.

Here’s what we saw.

Freshbooks

On the desktop, Freshbooks clocked in at a respectable 1.85 seconds and 99.82% availability. On the iPhone it was 2.61 seconds, and 99.90% availability, and on the iPad 3.78 seconds and 99.93% availability. Not a bad overall performance at all, especially on the iPhone. Freshbooks actually redirects iPhone users to a unique smartphone-optimized site, which helps deliver great performance without compromising on content.

Outright

Outright does an acceptable job across all three screens. On the desktop, it’s running at 2.45 seconds and 99.94% availability. It’s a little slower on the iPhone and iPad at 4.57 seconds/99.73% availability and 4.59 seconds/99.76% availability respectively. This is despite the fact that Outright has implemented a responsive Web design for both the smartphone and tablet, as well as a “splash” or interstitial page encouraging first-time iPhone/iPad visitors to download their on-device app.

Mint

Mint offers a decent desktop performance – loading in 4.51 seconds with 99.78% availability—but still above the recommended threshold of three seconds. Performance suffers though on the smartphone and tablet, dropping to almost twice the time to a lengthy 9.91 seconds/99.69% availability and 9.89 seconds/99.59% availability respectively. Keynote research studies have found that most mobile Web users expect a mobile site to download in less than four seconds.

Conclusion

The challenges of optimizing your site for the three screens always poses a prioritization problem, but without such tailoring you can see vastly different levels of performance. Overall it looks as though Freshbooks appears to be paying the most attention to these differences, and is getting the best performance as a result.

As we compare these personal finance sites to the whole Keynote Startup Shootout Index, we see they fare better than social retail sites. Credit should be given to the personal finance sites for doing a better overall job of three-screen optimization.

To view the full range of Keynote Indices, visit here.

Keynote tests the sites in the index hourly and around the clock from four locations over the three largest U.S. wireless networks, emulating the browsers of three different devices. Data is collected from San Francisco and New York and then aggregated to provide an overall monthly average in terms of both performance and availability.

Photo Credit: Tom Cheredar/Warner Brothers 

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

It’s a common challenge we see frequently with customers. In past articles, we’ve looked at some standard best practices Keynote recommends for delivering the best graphical experience without killing performance. This month, we’ll dig a little deeper into some of these techniques, specifically for the mobile environment, and look at how our startups on the Keynote Startup Shootout Index are handling these challenges.

Keeping the Connection Clear

Behind every web experience is a complex process of establishing network connections and making requests for specific content. There can be a lot of communication overhead needed to get the content from a web server to the web browser. Today, the standard is to make multiple HTTP requests over a single web server connection. Known as “persistent connections” or “HTTP Keep-Alive”, this is now a standard supported in all current browsers, so there is really no reason not to use it.

Even the fastest sites on the web can overlook optimizations like persistent connections. One site on the Startup Shootout Index that is not using HTTP Keep-Alive is Rovio. Rovio is already the fastest site in the Mobile and Social Gaming category of our index, but their user experience could be further improved by turning on persistent content connections.

Optimum Ways for Managing Images

Images, we know, can be the prime suspects in slowing mobile performance. How many images you place on a page and how you manage them are critical to the user experience. Keynote has identified four key areas for optimizing the use of images for maximum mobile performance.

1. Serve images at the same resolution that they are actually displayed on screen. 

For many site owners, creative content often starts out in enormous high resolution, which makes them a big problem for smaller mobile screens. The 3G network then strains to transmit them. Many sites struggle with this problem. The smartphone-optimized site for social retailer Gilt has a small number of images that are being served at higher resolutions over the network and then are scaled to a smaller size for display on the device. Although in the case of the Gilt home page this is only impacting a small number of images, the site is transmitting over 65% more bytes of data for those images than are necessary. Serving the images at the proper scale would not only reduce the number of bytes that need to be sent over the network, but would also enable the browser to display them on the screen more quickly.

2. Use compression tools

Web designers have many image formats to choose from. PNG and GIF image formats are “lossless” in that they encode exactly the content of the original image. The JPEG image format, on the other hand is a “lossy” format, where there is a tradeoff between higher levels of compression versus high levels of image quality. JPEGs are best suited for photographic images, where some compression is less noticeable. GIF and PNG files are better suited for image content with sharp edges or straight lines, such as buttons or images with text.

It is important to choose an appropriate level of compression with JPEGs, but many web designers could also optimize their images stored in the “lossless” image formats. Lossless compression techniques can optimize images stored in GIF or PNG formats. They accomplish this by stripping out unneeded meta-data like comments or by optimizing color palettes to remove data about colors that are never used in the image. There are several excellent (and often free) lossless compression tools available. For PNG images, tools such as OptiPNG and PNGOUT can even work as filters in a production deployment process to make sure that every image is compressed automatically during the release process.

3. Reduce the number of small images requests.

It still surprises us how many web sites – approximately three quarters of the sites we look at, and especially mobile sites – could significantly enhance performance if they just looked at ways to reduce the number of image requests on each page. This can be done with a technique called CSS (Cascading Style Sheets) sprites. Doing this takes time and effort but there are a number of tools that can help. CSS files are used by the browser to decide everything from what fonts are going to be used and background colors or images to where to display content on the screen. CSS rules can be written that pull out a small region of a larger image file, for example, to display at a certain spot on a web page. This means that each small image you want to place on the page does not need to be stored in its own file. A collection of smaller images can instead be placed into a single image file, and clever CSS rule-writing can make sure that the image content is displayed exactly where you want it on the page.

By replacing many small image files with a single larger image file, you may not be sending any fewer bytes over the network, but you can reduce the network overhead of making many separate file requests. This is especially helpful on mobile network connections where the network latency is much worse than the throughput. CSS spriting is something that every mobile site designer should at least consider in the site design process, as it can greatly improve performance.

4. Look at alternatives to CSS Sprite.

An alternative to the CCS Sprite is the Data URIs (uniform resource indicator). For images being used only once, it’s a quick solution. To use a data URI, you convert an image to a string of text using a tool that does Base64 encoding. What looks like a string of garbage characters can be decoded in the browser and used as the original image data. By inserting the Base64-encoded image data directly into the HTML for the page, small images can be included without the need to make separate network requests.

There is one major caveat to using data URIs, though. Images encoded as data URIS do not get cached separately from the HTML file they are included in. So, if you have a collection of images that are used on multiple pages on your site, you will be best served with using CSS sprites. For a single image that is unique to one page on your site, you might consider using a data URI to optimize the performance.

So a common theme emerges – avoid round trips to the server. If you are unable to keep the number of images on a page to a small number, then this is the area that will bring down performance.

Conclusion

Given the unique challenges of the mobile environment, here’s what we would suggest as the main three best practices to stay close to:

1. Avoid redirects wherever possible. This is time critical to the user experience.

2. Keep the number of DNS (domain name system) lookups on a page as small as possible as every new domain lookup adds time to loading. If you do need to have domain lookups, choose them carefully. We recommend limiting them to two domains. Be particularly aware of the third party domains, such as Google Analytics and Adobe to name just two common ones. Also make sure they are loading at the end to avoid delays which you, the site, operator, have no control over. Again this is critical for mobile performance – it is less impactful on desktops.

3. Keep the number of overall requests to ten or less. For example, one JavaScript file, one CSS etc. – anything to avoid the number of round trips on a 3G network.

Keynote tests the sites in the index hourly and around the clock from four locations over the three largest U.S. wireless networks, emulating the browsers of three different devices. Data is collected from San Francisco and New York and then aggregated to provide an overall monthly average in terms of both performance and availability.

Photo Credit: Shutterstock

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously. VentureBeat is Keynote’s exclusive media partner, so we’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

The Keynote Startup Shootout Monthly Index measures a range of startups’ Web performance across three screens – desktop, smartphone and tablet. We frequently see big differences in these performance numbers, for the same company, across these three screens.

Why such discrepancies? The challenge for startups is understandable – how do you prioritize where to put scarce development resources when you have three screens to deliver on. Perhaps your target market also has a clear preference for a particular device that makes it worth focusing on that at the potential expense of other screens?

Take for example Box and Dropbox. On the desktop, they load at an acceptable (but not great) 3.17 seconds (Dropbox) and 3.44 seconds (Box.net). On the iPhone, however, Dropbox falls to 10.35 seconds and Box.net to 15.32. On the tablet, Box.net maintains the 15 second load time but Dropbox drops even further to 21.81 seconds. In contrast, Rovio Entertainment (creators of the Angry Birds games) shows a cumbersome 14.75 seconds to load on the desktop but delivers impressive 5.98 seconds and 6.28 seconds on the iPhone and tablet respectively. Rovio’s successes have largely been on the mobile screens, so perhaps it is not a surprise that they’ve done more to optimize the mobile experience.

Given that dilemma, we decided to take a look at some approaches for making the most of optimizing a site across the three screens even when resources are tight.

Content Trade-offs

On the desktop, many Web sites want to deliver a rich content experience for the user –graphics, video, and multiple ways to sign up or interact. This is especially true for sites in the gaming and entertainment industries. But rich content is usually where the first problems of performance start. How fast does that initial page render happen while all that rich content is loading? When does the user begin to be engaged with the page?

For years, Web sites have used monitoring services that provide an external perspective of performance by measuring complete page load times. Testing systems located in datacenters all over the world measure the amount of time it takes to download content from a web site. This continuously robotic sampling of a web site’s responsiveness can tell web site owners a lot about how well their site is delivering content, but just looking at end-to-end page load timings doesn’t represent how well a user’s web browser is actually assembling that content or how soon the user can begin to interact with the page.

New industry standard timers supported by modern browsers can reveal performance more holistically—based on users’ true experience during the page load. The World Wide Web Consortium (W3C) and major browser organizations use the Navigation Timing standard for measuring speed from the browser client. “The browser itself saves timestamps from various events in the process of navigating to a page, including timestamps for the starting and ending of phases,” said Internet Explorer Program Manager Jatinder Mann, who is in the W3C working group.

Now implemented in the current versions of Internet Explorer, Chrome, and Firefox, the Navigation Timing standard helps measure:

• The first point at which the user sees something other than a blank screen (“Time to First Paint”)

• When a page can be fully clicked, swiped and scrolled (“Time to Interactive Page”)

• The total time a page takes to render completely (“Total User Experience Time”)

These are important events in a page’s lifecycle and make the difference between a user seeing something and seeing nothing. As an example, imagine you visit two sites, each with a page load time of five seconds. If site A starts delivering content in half a second and site B doesn’t start giving you any content at all until four seconds, your perception of site A will be much more positive even though both sites have the same overall 5 second user experience time.

For intentionally content-heavy sites such as those in the digital entertainment or gaming industries, developers need to ensure the initial render time is as fast as possible. User satisfaction will be improved if at least some key elements of each new page are rendered as quickly as possible in order to maintain the user’s interest.

And how exactly can you do that?

Site designers have a lot of control in how fast their pages can achieve the Time to First Paint, and adhering to industry best practices is an important starting point. For most modern browsers, the initial render of a page will not happen until all the Cascading Style Sheet (CSS) content that the browser is expecting on the page has been loaded. CSS files are used by the browser to decide everything from what fonts are going to be used and background colors or images to where to display content on the screen. If browsers begin rendering the page before all CSS content is downloaded, there is a risk that a late-arriving CSS file could result in dramatic changes to the page layout – and very disruptive to the user experience of the page. Browsers are therefore conservative and choose to wait to render the page until all CSS has downloaded.

Going back to our startups’ sites, Rovio could be doing a better job of managing the CSS on its site. The final CCS files critical to the initial render of the page are not loading until the 38th network request on the page. Images and other content on the page are loading over the network, but are not being displayed on the screen right away because the browser is waiting for those final CSS files to arrive. Site visitors going to http://www.rovio.com on a desktop browser are waiting an average of 3.4 seconds to see something other than a blank screen.

So how can you avoid this problem? The best practice recommendation is to load all CSS as soon as possible after the base HTML document for the web page. It sounds simple, but merely changing the order in which assets are referenced in the HTML document can dramatically change the user experience of the page load. Give the browser an opportunity to render the page as quickly as possible. Site owners should also track initial render performance as a key user experience metric. All of the major browsers (IE, Chrome, Firefox, and Safari) now have built-in tools that help developers measure the Time to First Paint during the development process. Once the changes are rolled into production, continue to monitor the initial render experience with a program of ongoing site monitoring.

We know it’s a tough call prioritizing screen developments but with some best practices, we can minimize the delays and keep users engaged.

Photo Credit: Irish Typepad/Flickr

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

Welcome back and Happy 2013. May it be a successful one for all our startup entrepreneurs.

We track over 40 sites in our Startup Shootout Monthly Index – collaboration sites, travel,
mobile and social gaming, social retail, social networking and digital entertainment – so we see
performance across startups in a variety of industry verticals. As we get to grips with January
and think of plans for the year ahead, we know this is a peak time for travel reservations. Cruise
anyone? Planning that family trip for Spring break? Let’s take a look at our travel category this
month and see how the companies we track are performing.

We focus on tracking startups, so the big names like Travelocity and Priceline aren’t part of
the mix. The index includes some sites that are designed to enhance the travel experience through
reviews or online community forums, as well as online travel agencies that let customers
search for online bookings of airfare, hotels, and rental cars. Two startups aiming to get your
travel dollars are JetSetter and Kayak. These sites compete against the big names in the
industry, but just as importantly, they compete against each other for customers willing to
try a new OTA site.

Optimizing the Online Travel Agency Experience

Interestingly, neither JetSetter nor Kayak is optimized for the tablet experience. Using these
sites on a device like the Apple iPad, customers will see the same web site as they get on a
desktop computer system. On the other hand, both JetSetter and Kayak provide a mobile-
optimized site specifically for smartphone users. Many of the travel sites in the Startup
Shootout Index do not have mobile-optimized sites designed to meet the screen challenges and
network limitations encountered by users of smartphones. So, right off the bat, these two sites
stand out.

JetSetter performs slightly better than Kayak on the desktop environment while Kayak does
better on mobile smartphone devices such as the Apple iPhone 5. Though JetSetter is slower
than Kayak by only a few seconds, this equals a 27% slower experience for their customers on
smartphones. What can we learn from these two sites? Why is one faster than the other?

Kayak has far fewer new HTTP requests – about 14 compared to the 33 on JetSetter. Because
mobile 3G networks have high latency, each HTTP request can introduce delay to the user
perception of the page load. Keynote recommends that companies focused on mobile-
optimized sites should aim for page designs with 10 or fewer new HTTP requests. Kayak has
optimized its site to use CSS sprites, for example, loading multiple small icons in a single
image instead of loading several small image files. Keeping the element count low is a great
competitive advantage for Kayak on mobile.

In addition to fewer new HTTP requests, the Kayak mobile home page also has many fewer
bytes of data on the page. Both JetSetter and Kayak have more than 100 KBytes of page
content, but the JetSetter site exceeds 800 Kbytes. JetSetter also has a very heavy single
JavaScript file – one file that is over 120 Kbytes. It is typically taking three to five seconds to load
this one file. Here at Keynote we recommend 50 Kbytes to 100 Kbytes (or less) to avoid latency
issues.

Besides being slow, this one file creates another major performance problem on the page.
Mobile browsers can “block” when loading JavaScript files. In order to make sure that scripts
files load in a predictable order, browsers often stop loading anything else in parallel while a
script file is loading. Properly implemented, JavaScript does not need to block during downloads.
It can be set up to allow simultaneous downloads. Unlike JetSetter, the JavaScript on the Kayak
site is not blocking. This is one of the big reasons for the 27% speed Kayak has on smartphones
over JetSetter.

On the tablet, these two online travel agency sites are performing very slowly. Over 3G mobile
network connections, Kayak takes just over 24 seconds on average to load. JetSetter takes
just under 30 seconds, making it approximately 21% slower on tablet than Kayak. Neither site
is making a real effort to improve the user experience for tablet visitors on mobile network
connections (or other slower network connections).

Conclusion

Overall, we’re surprised by how many startups we track – especially in a category like
travel – are not yet offering smartphone-optimized sites. We realize this development isn’t
cheap, but for startups trying to capture the interest of consumers willing to try something
new, the smartphone browser is a place where your site can stand out. In this respect, Kayak
and JetSetter set themselves apart from their startup competition with sites optimized for the
smartphone browser. But as we look more closely, Kayak’s site better adheres to Keynote’s best
practices for page weight, element count, and Javascript implementation. Once again, it just
goes to show the potential room for improvement when delivering sites across desktop, tablet
and smartphone environments.

Ken Harker is the mobile evangelist at Keynote Systems.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

We will hazard a guess that over the holidays, a lot of people will receive brand new smartphones and tablets. And while they are relaxing during the vacation, chances are they will check out some of the latest digital entertainment sites.

At Keynote, we love how digital entertainment sites are bringing a wide range of music, video and podcasts to the device of your choice and that they are adding social media capabilities to recommend more content that you might like and connect you with like-minded people.

We track a number of companies in the digital entertainment category of the Keynote Startup Shootout Monthly Index and thought this would be an opportune time to see how they are performing. Let’s see how successful these startups are at delivering the best practices across three screens – desktops, smartphones and tablets.

We test our startups using IE9 on a desktop machine, and across the Apple iPhone 4 and the Apple iPad 2 connected to live mobile networks.

First, It’s All About the Page Loads

Keynote recommends the industry best practice for page loads on a smartphone be under five seconds. Among the startups in the Keynote Startup Shootout index, only Spotify and rdio manage to deliver that. Project Playlist took a whopping 36.4 seconds on average in November!

Even on the desktop, where a response time of two to three seconds is the norm, the Spotify home page takes a sluggish 9.62 seconds on average. These entertainment sites clearly have a lot of work to do in order to improve their mobile performance and strengthen their competitive position in the market.

We decided to take a closer look at SoundCloud, as it offers some great examples of what to do and what not to do for the smartphone environment. Overall, we found that SoundCloud has a site optimized for a smartphone screen (good), but not optimized for the mobile 3G networks (bad). Let’s take a look at what some of these examples look like.

One of the ways that the SoundCloud smartphone site follows good practices is loading image elements using data URIs (Uniform Resource Identifier). Data URIs let the website include images as encoded text strings directly in the HTML for the page, avoiding extra HTTP requests for the image content. This enables the display of multiple images from one download. It’s a very nice move – a much better use of bandwidth, removing the need for about 10 external requests on the mobile home page.

Another way SoundCloud reduces the number of network requests on the mobile home page is with a technique called CSS sprites. CSS sprites combine several small images into one large image and rely on CSS rules to display portions of that larger image where they are needed on the page. A little more complicated than Data URIs, CSS sprites have some advantages for images that need to be cached and re-used on other pages on the site. These are all good examples of optimization.

 Second, It’s All About the Page Loads

Now let’s look at some areas in need of improvement on the SoundCloud smartphone site. A key issue is redirection from the initial page request to the base page HTML call. The first redirection is to an (unnecessary) secure domain and then once it recognizes the visitor as a mobile user, on to a non-secure mobile domain. These are the type of steps that can add a full second to the delay that users experience before the page begins to load.

Another interesting item which affects response time is analytics tags, such as Google Analytics. At SoundCloud we see that analytics tags are being loaded before some critical page content is requested. Not only does this add to the delay, but also adds to the risk of causing a problem. If a third party service has any performance or availability issues, it could actually block the page from loading. Better to load these third party services after the critical page content has loaded.

It also seems SoundCloud has made a conscious decision to display a large amount of content on the mobile home page – a total of 25 thumbnail images are displayed in a list. The amount of dynamic content on that opening page slows response time down with 25 round trips required over the low-speed, high-latency 3G network. This would be acceptable for site visitors on WiFi connections, but on 3G mobile network connections it can have a big impact. Although SoundCloud has optimized the non-dynamic image content on the page for mobile performance, the decision to send so much dynamic image content on top of it results in slow page loads for the 3G mobile site visitor.

By contrast, we see that competitor Spotify has very limited dynamic content on its home page– and hence its nifty 4.67 second smartphone response time, or better still rdio at 3.9 seconds.

In Conclusion

Some of these startups will no doubt get a visit over the holidays, so let’s hope they can deliver a satisfying, initial customer experience – across multiple devices – before the consumer moves on to faster or easier to view sites. Obviously our advice is to do testing scenarios on a variety of mobile devices and across different provider networks, and then monitor end user experience for better decision making, competitiveness and growth.

See you back in 2013!

Ken Harker is the mobile evangelist at Keynote Systems.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

With the holidays upon us, we thought we’d take a look at how ready our index startups are for two great holiday traditions – shopping and travel. Between Black Friday, Cyber Monday, holiday vacations, and winter storms, these sites are about to get a workout.

As part of the Keynote Startup Shootout Monthly Index we track some of the top travel and online retail sites, across desktop, smartphone and tablet devices. We’ve discussed before the impact of this “three screen” world, especially in adapting to mobile. Let’s see if our index startups are ready for the holiday challenge.

The Virtual Shopping Experience

Creating positive online shopping experiences used to be so straightforward: have the right products at the right price, a smooth checkout process, good overall site performance, and the sales will come rolling in. Mobile sites are becoming increasingly popular for shopping and it is important for retailers have a strong mobile presence. But mobile sites present a whole different set of challenges to ensure they look good and function efficiently. The shopper, of course, just expects a consistent level of performance, regardless of device.

Fab.com offers a native app for the mobile visitor, but it is buried at the bottom of the site. Even if a user does manage to find it, it has taken them a while to do so, which slows down the shopping process. Key takeaway? Put that app right up top if you want to get it used.

None of the retail sites offer tablet-specific pages, but some are designed for better performance.  The two fastest home pages for tablet users, RueLaLa and Ideeli, load about three times faster than the others on the retail index. Keeping pages lightweight and less complex, whether the page is specifically optimized for tablet users on not, is key to earning their loyalty rather than driving them to frustration.

Beware of the online retailer that fails to provide a mobile or tablet-specific experience. Otherwise, they are loading incredibly large (desktop) pages across a usually much weaker network and screen experience.

Travel-log

As with shoppers, travelers tend to be in transit if they are accessing these sites from their mobile device. If they are on the road, they may have varying degrees of network support – sitting in an airport or station, driving, on a train, or finding themselves in rural locations with limited access.

Again, we see a number of the travel sites delivering a full desktop page when accessed via a tablet. Companies sometimes assume that tablet users have stronger wi-fi connections than mobile users, but this is not always the case.

Tripology, interestingly, has neither a mobile-optimized site nor app, never mind being optimized for tablets.

AirBnB has an iPhone-optimized site, but the loading time is slow. It loads an excessive element count – which means too many http requests. We can offer a couple of best practices here.

• Keep to ten or less HTTP requests. These each involve round trips from the browser to the server which add to a site’s latency. On the AirBnB site, we are currently seeing 47 http requests. That will definitely slow you down.
•  Redirections can be painful in the mobile world, and again we’re talking about slowing down the service dramatically. With AirBnB, there are two redirections from the original URL – first from a non-secure to a secure to a secure mobile site. Ironically its front page appears to be a secure site, which is unusual for a home page. Secure pages are usually reserved for transaction areas and credit card/billing-type pages.
• We recommend at most two or three domains for mobile or tablet apps due to the network issues they can have. We saw 12 different domains on AirBnB’s home page, each of which can add a quarter of a second to loading time.

Conclusion

We continue to see tablet-specific sites fall somewhere in between the full desktop experience and the mobile experience. A startup might have the desktop and the mobile covered, but not the tablet. It makes a big difference when it comes to the customer experience.

The holidays will keep pushing the web performance of these retail and travel sites, plus we’ll assume many people will receive new devices which will only increase the demand. Let’s hope 2013 brings “three screen success” to all out Startup Shootout companies.

Aaron Rudger is the senior manager of web & mobile performance and Ken Harker is the mobile evangelist at Keynote Systems.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. We’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

For online businesses, executing a mobile strategy is still an act of experimentation. Unlike the desktop web, the ingredients that make a recipe for mobile success are being tweaked by companies as large and small alike.

When it comes to mobile strategies, this month’s Startup Shootout Collaboration index provides a wide range of examples. Some companies do not provide mobile-optimized (“m.”) sites for their services on mobile devices. One company even goes so far as to refuse an iPhone user to browse its website. However, it is notable that the three leaders in performance across three screens (Dropbox, Box, YouSendIt), are also the three leaders in the collaboration space itself. Dropbox and Box.net appear to be the most mobile-friendly. Coincidence? Let’s see.

When looking at the index, we noticed that startups leverage mobile in ways that directly benefit the interest of their user strategy. Since collaboration tools are generally used as productivity applications, enabling a positive user experience across 3 screens (desktop, smartphone and tablet) can be critical, especially for enterprise use that may include remote workers.

Here’s how the hottest startups in collaboration are managing the mobile aspect of their online experience:

Dropbox

The clear market share leader in the collaboration category keeps all their sites simple and clean. From the mobile browser, visitors are directly pointed towards the login and then to download their files. Dropbox has pruned all unnecessary objects and keeps the interface fast and simple across all 3 screens. Dropbox maintains the fastest average speed across the desktop, smartphone and tablet screens at 9.84 seconds.

Box.net

As another top contender, Box.net also does a great job when optimizing for its mobile users. It’s clear that Box.net wants mobile users to be able to access files and collaborate with others not only while at the desktop but on-the-go. Box.net asks mobile users if they would like to download their iOS app, but also gives them the option to continue to a mobile website. As a result, Box.net performs well across each of the 3 screens, with only slightly slower performance than Dropbox.

YouSendIt

YouSendIt does not offer their users a mobile optimized site. Instead, they prompt the user to download their iOS app (like Box.net) or continue on the full (desktop) website. While this is not optimal, it does give both new users and existing customers reasonable options for either learning about the service or interacting with it directly.

The other collaboration companies haven’t quite figured out the three screen divide as well as the top competitors. One example is Springpad whose mobile interface leaves the user experience with much to be desired and is frankly a bit confusing.

Springpad.com

It looks like Springpad has developed a site for its iPhone users and tries to push those users seeking www.springpad.com directly to Springpad’s iOS application using the “springpad://” command (as seen below).

If you have installed Springpad app on your iPhone, it will automatically open up the application. But this mobile strategy doesn’t work for new visitors to Springpad, since shortly after the site times out. It seems that Springpad’s mobile strategy is to push users to leverage its application rather than use the service through mobile web. A poor assumption for Springpad is to assume that when users attempt to access the site through Safari, they want to use the app. This approach denies mobile users access to a mobile web experience they may desire.

Directly opening a mobile app when browsing a web page sometimes makes sense in the context of certain tasks. But it only makes sense when you first check whether the native app is installed and then try to open it. As a result, Springpad’s performance in the index suffered greatly.

SugarSync, Huddle

SugarSync offers a downloadable app for their service on mobile, but smartphone and tablet users browsing their website receive the same experience as a desktop user which results in long average response times of more than 46 seconds on iPhone and 47 seconds on iPad.

Visitors to the Huddle website also receive the same experience across devices, but with better performance. Huddle’s service is available via downloadable app for iOS (iPhone and iPad), Android and Blackberry.

Overall Thoughts

Collaboration startups clearly want their customers to use an on-device app when on a smartphone or tablet, but taking a one-size-fits-all approach is bad for users and performance.

For the full data from this month’s Startup Shootout, see below.

Aaron Rudger is the senior manager of web & mobile performance and Venkatesh Giri is the mobile evangelist at Keynote Systems.

A recent Keynote study of the preferences and expectations of smartphone and tablet users shows that a majority of website visitors want a fast download from the desktop (sub-3 seconds) and tablets (sub-3 seconds) and that their expectations for smartphones isn’t far behind (4 seconds or less). (See above table.) What’s more, the study shows that smartphone and tablet users are actively going to mobile websites in large numbers for shopping. Survey respondents reported that slow loading web pages was the number one source of frustration.

Unfortunately, when it comes to Social Retail startups on the smartphone or tablet, there are probably a lot of frustrated visitors. Social Retail sites delivered an average load time of 2.16 seconds, which is in line with user expectations. However, from an iPhone they’d need to wait an average of 12.77 seconds to get to the home page and worse yet, it would take 26.47 seconds from an iPad over a 3G connection.

When we compare the Social Retail experience on a smartphone to the average (7.28 seconds) of more established brands for the same week in our retail smartphone index, we see that the new kids on the block are nearly twice as slow. The good news for social shopper is that the trend over the past few months seems to be showing improvement. But what’s troublesome is that Social Retail websites are now loading 10 times slower than what users want.

So where are Social Retailers missing the boat when it comes to delivering a good experience across all three screens?  Many of these sites are making the same common mistakes that would lead to a frustrated user. Let’s take a look at some examples:

Gilt

Meeting expectations from the desktop, they aren’t faring as well on the iPhone.  The size of their mobile site is rather large (554 KB) and using 9 domains, delivering 41 images and 55 elements can be contributing to slow speeds.  Also, following mobile best practices of combining Cascading Style Sheets and JavaScript, and keeping JavaScript at the bottom could help them improve their 15.56 response time.  But on the iPad, the speeds are much slower.  Here they’re using 19 domains and not following many of the same best practices.  Compounding this they’re delivering 2,232 KB of content over the mobile network.

Living Social

Living Social is doing better than the average with respect to speed (download rate) for the iPhone user delivering a clean, simple website designed in accordance to many mobile best practices.  However with 300 KB of content it is on the heavy side.   So why such is the site so slow for the iPad user?  Firstly, they’re delivering nearly 2 MB of content with over 150 elements.   They aren’t combining Cascading Style Sheets or JavaScript, have oversized images and aren’t following other W3C best practices.  Worse yet, they make the iPad user go through a 3 step registration process that isn’t required of smartphone users.  Websites are important for all retailers because they’re far more discoverable than apps which require a download from an app store.  Three additional steps needed to get to a home page increases the likelihood of user abandonment.

Living Social was not the only Social Retailer requiring upfront registration, and collecting information is very important in Social. But if fake information will get you to the destination, companies should weigh what’s gained in collecting junk data against the potential loss of real prospects.

RueLaLa

RueLaLa seems to be the one Social Retail site with good response times from the desktop and iPhone, and better than average speeds for tablet users.  On the smartphone they’ve kept their site down to 80 KB and limited to 12 objects, in line with the size of top performing mobile sites.  The tablet is where they stand to improve.  Reducing the site from the current 600KB size and following design best practices can speed up their 14.23 second download.

RueLaLa also presents smartphone and tablet users a splash screen to promote their downloadable app.  A splash screen is like an interstitial ad, but users must click through to remove it from the page.  The splash screen adds weight to the page but more importantly, it can be a distraction for a potential visitor coming to a site for the first time.  For Social Retailers trying to be discovered, carefully consider if a splash screen is the right strategy. You may increase app downloads, but if it’s at the sake of having your products discovered, and purchased, the cost may be too great.   Also, remember that smartphone and tablet users may actually prefer the web over apps.  Trying to push them down a different path could end up in a less than satisfying experience.

For the full data from this month’s Startup Shootout, see below. As always, we are including VentureBeat’s load times too.

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously. VentureBeat is Keynote’s exclusive media partner, so we’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

It looks like attention to mobile devices is becoming increasingly apparent across the Keynote Startup Shootout. That’s no surprise: social mobile gaming is growing, and with it, the need for speed.

A recent research report by Parks Associates on “Trends in Digital Gaming” shows that smartphone and tablet adoption has grown along with the percentage of U.S. gamers who regularly download games or gaming apps to their mobile devices. Game downloaders grew to 18 percent in 2011 from 7 percent in 2008, the firm found. Among tablet owners, 71 percent of adults and 79 percent of teens play games on the device at least one hour per month.

These growth trends verify the importance for game publishers to optimize further and stay competitive on the mobile platform, for both smartphones and tablets. Some of our leading startups in the Shootout have taken this into consideration and we have seen changes in this month’s Social Gaming Startup Shootout.

Rovio

Over the past couple of months, Rovio has been able to greatly improve its response time on smartphones and tablets. In April each took an average of 24 seconds to download. The company has subsequently redirected iPhone and iPad users to a much lighter, mobile-optimized site. The mobile-optimized site loaded in roughly 7 seconds in the July index, making it the fastest social gaming site on the iPad and second fastest on the iPhone. That’s great, but its average response time on the desktop continues to be in excess of 13 seconds — very poor.

By not maintaining a persistent connection, the Rovio website makes a new connection request for each element downloaded, adding up to 150 milliseconds per request. Download around 63 objects and it starts to add up, resulting in poor load time.  Not maintaining a persistent connection is a major violation of performance best practices and one that’s only further exacerbated by one’s distance from the content server.

Papaya Mobile

Another social gaming site is also faster on the smartphone than the desktop but for a conscious and strategic reason. Papayamobile.com has a 3.47 second response time on the desktop — average for the category but a bit on the slow side. However, on the iPhone it’s a blazing-fast 2.58 seconds. The reason it’s so fast is that it has required a login as a first step to access the site. Being rather light, the login screen loads quite quickly. But it gives the user no other avenue for interacting with the site.

CrowdStar

CrowdStar arguably had the best overall performance for social gaming websites. Its desktop time was better than average, and the smartphone response time of 6.97 seconds is remarkably good considering it’s delivering the entire site to iPhone users. (Hint: optimize the site for even faster speeds.) The response time for iPad users is better than average and may end up being tolerable when people are connected over Wi-Fi instead of the 3G network.

Tim Murphy is senior manager for mobile quality at Keynote Systems.

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously. VentureBeat is Keynote’s exclusive media partner, so we’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

As the summer travel season heats up, more travelers are searching for special deals and last-minute bargains. It seems that travel startup sites are responding with improved performance and availability. In June’s shootout, all but one travel site was faster than 4 seconds on the desktop, as shown in blue in the chart at the top of this post.

Mobissimo averaged 4.93 seconds, more than 25 percent slower than the other sites. The Mobissimo site is one of the heavier sites in the Travel shootout index, requiring over 100 new HTTP requests for the desktop-optimized version of the page to load. Also, nearly half of the requested elements are JavaScript files. Together, those are two clear reasons for the delay in speed. One consequence of the page design is that even the initial page render (when a site visitor first sees visual changes to the screen) typically takes more than 3 seconds.

Kayak, on the other hand, was the fastest site in the Travel shootout this month. Its desktop-optimized home page loads only 6 JavaScript files, and the time to initial render was faster than 1 second.

Kayak was also a standout for its mobile-optimized sites as well this month. Together with TripAdvisor, it was one of only 2 sites to serve the mobile-optimized home page in 6 seconds or less to smartphone users (the red bars in the above chart). The overall availability was lower on the mobile sites this month, but both TripAdvisor and Kayak maintained excellent availability.

Kayak’s smartphone-optimized site loads just ten elements, but could potentially be optimized even further. Four of the individual images used on the Kayak mobile home page are small PNG image files that could be combined into a single file using CSS sprites. Doing so would reduce the number of round trips between the phone and the web server and could lead to even faster page load times. Another opportunity for Kayak to improve its home page performance is in optimal use of redirections. Currently, if a site visitor types “kayak.com” into the mobile browser, the site will first send a redirection to “www.kayak.com” and then a second redirection from “www.kayak.com” to “m.kayak.com.” Eliminating this middle step could reduce overall page load times by 1 to 1.5 seconds on average.

In tablet-optimized offerings (the orange bars), the Travel industry also improved in June, on average loading page 1 second faster this month. Tablet users on 3G mobile data connections can expect very slow page load times, however. Only three of the seven sites in the Travel shootout were faster than 20 seconds on average.

Tripology and Kayak were the leaders in tablet website performance for this category. Tripology does well despite serving essentially the same content to tablet users as it serves to desktop users. Kayak does as well, but at the price of trimming off many of the links at the bottom of the page, for tablet users. By not optimizing content for the tablet user with the 3G mobile network in mind, pages that take less than 2 seconds on average to load over a high-speed desktop network connection can take 15 to 16 seconds to load over 3G mobile connections. Most sites in the startup shootout have yet to optimize their sites specifically for tablet users, and it really shows.

It is clear that the travel industry is quickly catching on the optimizing idea, especially for their peak season. However, travel sites overall have yet to catch on to the 3-screen optimization strategy to get the best from their desktop, smartphone and tablet users. That’s going to be important, as web access diversifies even further and users become increasingly mobile in the travel season.

See below for a summary of the June startup shootout across all categories, or click over to Keynote for the full Startup Shootout data set.

Ken Harker is a mobile performance specialist at Keynote Systems.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously. VentureBeat is Keynote’s exclusive media partner, so we’ll be bringing you a fresh set of data from Keynote every month. See below for the full data set, click here to see a larger summary graph, and check out previous Startup Shootout results.

Papaya Mobile is a new name in the mobile and social gaming index and their response time results tell an interesting story. A desktop response time of 5.34 seconds is slow by industry standards. However, they’ve managed to match that speed on the iPad 2 (5.41 seconds) and have a very fast download on the iPhone 4 (2.11 seconds). Digging deeper, we see that their desktop site is quite heavy at approximately 1.7MB (the index average is 1MB), while their iPhone-optimized site is very light at 15KB (the index average is 500KB). Most interesting is that they are sending iPad users on one carrier (we’re measuring over AT&T and Verizon) to the iPhone site, while the others get the full, heavy site over a 3G connection. The average score of the combined download speeds reflects neither of the two experiences (fast on one carrier, slow on the other) and leaves us to wonder which site (the “light” version or the full one) Papaya intends on delivering to iPad users.

Two new names in the social retail index are Rue La La and Ideeli. Rue La La has a blazing-fast desktop speed at .93 seconds and a relatively fast iPhone 4 download as well, but sending five times more content to the iPad is leading to a fairly slow, 14.5-second download. A side-by-side comparison with Ideeli shows still a rather quick desktop experience (1.59 seconds) but Ideeli is slow on both the iPhone and iPad while sending different content to each device. While it’s clear that companies need a more specific strategies for the difference of delivery over three screens, it still doesn’t ensure a quick download unless performance goals are part of the plan.

Pinterest was added to the Social Networking group and they’ve single handedly brought down the collective scores of the category in availability. On the iPhone, using one of the two carrier networks we tested, Pinterest’s site was only available half of the time. Why the abysmally low availability? Pinterest often sends 5MB of data to iPhone 4 users on each network.  Keynote has also noticed that some issues with redirection to m.pinterest.com are also included in this factor.

Digital Entertainment sites continue to have the most consistent performers across all three screens with Spotify, Rdio and Turntable.fm setting the pace for the entire index.

Nisheeth Mohan is a senior product manager for mobile at Keynote Systems.

Here’s the scenario: You’ve arrived at the airport only to discover that the airline canceled your flight and — oops — forgot to notify you.

But when you pull up a travel site on your phone to book a new reservation, it takes ages for the site to load.

That experience is all too common, according to Keynote Systems, whose Startup Shootout Index profiles the speed and responsiveness of web sites for various startups. This month, Keynote’s index found that in the travel category, there’s often a big discrepancy between how fast a site is on the desktop and how fast it is on an iPhone or iPad.

For instance, Kayak and TripAdvisor both have very snappy desktop and iPhone websites, loading their respective home pages in just a few seconds. But their tablet sites lag way behind, with 17- and 25-second response times, respectively.

“For a traveler on the go, 17 and 25 seconds can seem like an eternity and lead to a lot of frustration,” said Tim Murphy, a marketing manager at Keynote Systems.

Keynote Systems‘ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously.

Check out the full complement of startup speed tests in six different categories below.

To do the test, Keynote used its network of PCs to simulate desktop web traffic to selected startup websites from a variety of locations around the U.S. Additionally, Keynote simulated traffic from iPhone 4 and iPad 2 users in four locations: Chicago, Dallas, New York, and San Francisco. Tests were done over a period of weeks and the results averaged into scores for each website. In addition, Keynote created an index score for each category, which represents the average of all the websites in that category.

VentureBeat is Keynote’s exclusive media partner, so we’ll be bringing you a fresh set of data from Keynote every month. And lest you think we don’t take this stuff seriously, we asked Keynote to add our own homepage to its tests, so we’re now listed in its desktop news site index as well as its smartphone news site index. As you can see, we have a lot of room for improvement ourselves.

Note that the test measures home page response times, not the time to complete a transaction or do other activities on these sites. It’s an important measure of responsiveness and has a significant impact on these startups’ brands. Keep that in mind the next time you’re trying to load a website on your iPhone.

For a startup, a lot is riding on the home page. You might be surprised, then, to learn that startups vary wildly in how quickly and reliably their website home pages load.

That’s especially true when you look at load times on mobile devices. An increasing number of people use smartphones and tablets to browse the web, and yet — for some reason — the mobile versions of many startups’ sites lag far behind their desktop equivalents.

Keynote Systems‘ new Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously.

If a home page doesn’t load quickly and reliably, it risks driving away valuable new customers. If you’re lucky enough to get someone visiting your site, you don’t want them disappearing before you’ve had a chance to make your pitch.

To do the test, Keynote used its network of PCs to simulate desktop web traffic to selected startup websites from a variety of locations around the U.S. Additionally, Keynote simulated traffic from iPhone 4 and iPad 2 users in four locations: Chicago, Dallas, New York, and San Francisco. (We’re using Apple’s mobile devices as proxies for all mobile gear, which is not a complete representation of the mobile universe, obviously, but it does represent a wide swath of mobile users.) Tests were done over a period of weeks and the results averaged into scores for each website. In addition, Keynote created an index score for each category, which represents the average of all the websites in that category.

Note that the test measures home page response times, not the time to complete a transaction or do other activities on these sites.

The 34 startup websites measured by Keynote are grouped into six categories: Collaboration (including sites like Box, Dropbox, and Huddle), Travel, Mobile and Social Gaming, Social Retail (Gilt Groupe, Fab.com, and the like), Social Networking, Digital Entertainment (Spotify, Rdio, and Turntable.fm). To view this month’s full data set, check out the spreadsheet embedded below.

VentureBeat is proud to be Keynote’s exclusive media partner for this index, which means we’ll be publishing the results every month. We’ll focus on a different segment of the index each month. And yes, we know that those who live in glass houses should not throw stones, which is why we’ve asked Keynote to measure VentureBeat’s website performance as well, and we’ll report that data as soon as we have it. We’ll keep lobbing those stones, but at least you’ll know that we’re smashing our own windows, too.

This month, we’ll zero in on digital entertainment startups, whose response times are shown in the graph above.

Spotify has been in the news a lot lately, thanks to its arrival in the U.S. and its recent partnership with Facebook. The company has done a good job of keeping its performance consistent between desktop, smartphone, and tablet versions of its site, as has Rdio.

However, other sites don’t show the same consistency. Facebook has the top-ranking response time for its desktop and smartphone websites, but its tablet site ranks fifth overall.

“We’re seeing that tablets are often a distant third when it comes to performance,” said Tim Murphy, senior mobile marketing manager at Keynote Systems. On Facebook’s performance, Murphy noted that the site’s iPad response time is nearly eight times slower than the desktop on the same 3G network. “The iPad is 2X slower than the iPhone largely because they’re sending about 265KB of data to the tablet user versus 142KB to the smartphone user,” Murphy added.

As for the other categories, Murphy noted that mobile and social gaming sites seem to lag behind the rest of the startup categories. But retail sites need to improve too. “Retailers need to pay more attention to the mobile web, to meet the needs of the growing number of mobile retail consumers,” Murphy said, citing recent data from eMarketer on the growing number of mobile consumers.

You can see the full set of this month’s Startup Shootout Index data in a Google spreadsheet, or view it in the embedded spreadsheet below.