Australia is slow, Singapore is fast, and Europe to Oregon is twice as fast as Europe to California.
Increasingly, we rely on “the cloud” to deliver web services. But the cloud is not one amorphous thing, and the cloud is not all equal. As every developer knows, what we generically call the cloud reduces down, ultimately, to physical servers and wires. And even within one company’s cloud — such as Amazon — there are vast differences in speed and optimization from region to region.
What that ultimately means is that developers who don’t pay attention to cloud regions, as server debugging company Takipi discovered, can actually cause their apps to run 10 or even 200 times slower than necessary.
And Amazon doesn’t disclose any of that data.
“We noticed that in many cases there’s 10X (or even more) difference in the performance of services due to AWS/S3 regions and external APIs,” Takipi co-founder Iris Shoor told me via email.
In fact, Amazon’s cloud performance could vary in speed from region to region by as much as 200 times.
For example, upload times vary significantly within each region. Uploading a file in Virginia to Amazon’s U.S. East cloud region, also in Virginia, takes three time longer than uploading a file in Singapore to AWS in Singapore. But it’s moving files between regions to serve differing international clients that will cause the greatest slowdowns: Shifting a file from Amazon’s São Paulo Region in Brazil to its Sydney, Australia region takes 200 times longer than shifting files between geographically close regions like Amazon’s U.S .West (Oregon) and U.S. West (California).
“It seems that the new region — Australia — is by far the slowest,” Shoor says. “Uploading files to Australia can cause 10X latency versus other regions.”
And that makes sense — Australia is an island continent served by a limited number of international connections, and the file has a lot longer to go. But it is something to keep in mind when building global services.
And some of the latency issues aren’t easily explainable due to geographical isolation — they’re more likely side effects of Amazon Web services system design. For example, there’s a big difference between uploading a file from Amazon’s S3 servers in Ireland to the U.S. East Coast and transferring the same file from the same location to the U.S. West Coast. The West Coast, Takipi says, will take twice the time, despite the fact that it’s only 25 percent further.
Even worse, Europe to California takes twice the time of Europe to Oregon:
“It’s possible to make amazing optimizations just by changing the region,” says Shoor. “For example, choosing Oregon over CA for a company that serves the European market will cut the upload time by half.”
One of the main problems when hunting down the cause of slow services, Takipi says, is API latency. And regionality has a huge impact there, even if you’re as geographically optimized as you can possibly be.
For example, Shoor says, even if your app is optimized for Australia and you’re using Amazon cloud services in Australia, Facebook’s API is 11 times slower down under, on average, than it is elsewhere.
To monitor those slowdowns, Takipi is currently building “Project Barkdog,” a service to inform developers of slow APIs globally. Barkdog will likely launch in a couple of weeks, and will inform developers via email or Twitter when APIs that they’re using are slowing down.
“It’s possible to speed up performance significantly,” Shoor says. “Amazon doesn’t share any of this data.”
Image credits: Takipi, Amazon iCloud by John Koetsier, Globe by WoodleyWonderWorks/Flickr