Microsoft confirms Azure storage issues around the world (updated)

Microsoft today disclosed that the storage tier from its Azure public cloud is facing issues in data center regions across the globe. Other services are also having issues as a result of the storage errors.

“Starting at 22:42 UTC on 15 Mar 2017, customers using Storage may receive failure notifications when performing service management operations – such as create, update, delete – for resources hosted in this region,” Microsoft said on its Azure status page. “Other services that leverage Storage may also be experiencing impact. Retries may be successful. In addition, a subset of customers in East US may be unable to access their Storage accounts. Engineers have identified a possible fix for the underlying cause, and are applying mitigations [sic]. The next update will be provided in 60 minutes, or as events warrant.”

Out of 28 data center regions, 26 were having storage issues, according to the status page.

Customers might also be facing issues when they try to create new virtual machines (VMs), Microsoft said on the status page.

The downtime comes a few weeks after Amazon Web Services (AWS) got hit with an S3 outage, which led to other services going down. The downtime affected Docker, Medium, Slack, and the U.S. Securities and Exchange Commission, among other customers. AWS said the issue resulted from human error.

An Azure outage in 2012 lasted more than half a day.

The outage is impacting Microsoft’s Visual Studio Team Services (VSTS), among other things.

Update at 5:30 p.m. Pacific: The status page now says services are operating normally. “Engineers have identified a software error as the potential root cause,” Microsoft said in an update. “Engineers have applied a patch and mitigated the issue.”

Update at 5:32 p.m. Pacific: Now it appears a single region — East US — is still having storage issues. “Starting at 15:19 UTC on 15 Mar 2017, a subset of customer using Storage in East US may experience difficulties accessing their Storage accounts in East US,” Microsoft said. “Customers in this region may also experience failures when trying to access a subset of their Virtual Machines. Engineers have identified a potential root cause and are actively investigating. The next update will be provided in 60 minutes, or as events warrant.”

Update at 5:48 p.m. Pacific: More services are now being impacted as a result of the East US storage issues. “Starting at 15:19 UTC on 15 Mar 2017, a subset of customer using Storage in East US may experience difficulties accessing their Storage accounts in East US,” Microsoft said in an update. “Due to a dependency on Storage, customers using the following services may experience failures provisioning or connecting to Azure Search, Azure Service Bus, Azure EventHub and Azure Stream Analytics in East US. The next update will be provided as events warrant.”

Update at 6:59 p.m. Pacific: “Engineers have identified the root cause as a power event affecting a single scale unit. Data center technicians are on site working to restore power to the scale unit,” Microsoft said.

Update at 10:01 p.m. Pacific: The end of the outage could be near. “Engineers have now begun the recovery of Azure services and customers should start seeing recovery,” Microsoft said.

Update at 11:27 p.m. Pacific: Microsoft says everything is now operating normally. All told, the issues lasted more than eight hours, according to the most recent status update.