Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Updated 1:16 p.m. Pacific

Amazon.com and you’re (not) done? Well, for a while today.

Amazon.com was not exactly down, but the site might as well have been. All product searches were coming up error 400, though as for 1:16 p.m. Pacific today, it appears as if the problem has been fixed.

Twitter is blowing up about the problem, which is more than a little embarrassing for a company that prides itself on providing uptime assurance in the cloud for thousands of web-based businesses:

wow, all my searches on Amazon are coming up with error code 400! wonder what’s going on there.

— Empty Mirror (@EmptyMirror) March 1, 2013

Realistically, however, this is not a cloud failure or a web serving error. It’s a programming or configuration error, a bad request being sent to the server. Amazon should have it fixed shortly.

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Amazon’s homepage went down for about 51 minutes this afternoon in an highly unusual outage. Visitors to the site received an ‘Http/1.1 Service Unavailable’ message, although Amazon Web Services was still up and running. This is the most significant outage Amazon has experienced in years. A group of hackers called the NaziGods are claiming responsibility on Twitter, however sources with knowledge of the event deny that it was related to any outside group.

Estimates put Amazon’s sales-per-minute at over $100,000.

We have reached out to the company for comment and are waiting for a response. In the meantime, you may continue buying panini-makers and Kindles in peace.

[Updated: 10:05 pm PST: The service is back up]

Microsoft’s Xbox Live online gaming service is having an outage this evening just as new downloadable content arrives for Call of Duty: Black Ops II.

Just 20 minutes ago, Microsoft posted the problem with signing into the service this evening. The company said, “You might have been kicked off of Xbox LIVE, and you might be having some problems logging back in. We are investigating this issue, and we’ll get this fixed ASAP so you can get back to your favorite games, apps, movies, and music. We will let you know how things are going in 30 minutes. We appreciate your patience.”

No telling how long this will last right now. But Call of Duty fans are likely to be frustrated as Activision Blizzard was set to launch its new Revolution map pack DLC. It’s not surprising to see an outage when a new Call of Duty map pack hits because the traffic can jam the service. But we don’t know the cause right now.

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

Tom Paseka works for CloudFlare, the content delivery network that handles more traffic than Amazon, Wikipedia, Twitter, Instagram, and Apple combined, delivering more than two billion pageviews per employee. The company knows a few things about the Internet.

What Paseka knew last night, apparently before any Google employees noticed, was that Google’s services appeared to be offline. Tracing the problem, he noticed an Indonesian Internet service provider in the path to Google — odd by any standard.

Particularly when CloudFlare is just a few miles from Google, not an ocean away.

It turns out, Paseka learned, that the Indonesian ISP Moratel was giving its users an incorrect route to Google. And because Moratel was trusted by other networks upstream, the incorrect route was propagating around the globe. As Paseka writes:

And, quickly, the bad routes spread. It is unlikely this was malicious, but rather a misconfiguration or an error evidencing some of the failings in the BGP (Border Gateway Protocol) Trust model.

The fix was simply notifying Moratel about the issue, which Paseka did. Three minutes later, the problem was fixed and Google’s services were back online. Of course … they had never gone down. But they had been inaccessible.

You may not have noticed unless you were in Hong Kong. Paseka estimated that the entire outage affected only about 3-5 percent of the Internet population.

No word on whether Google engineers sent their CloudFlare colleagues a box of donuts or a Google hoodie in thanks.

photo credit: GustavoG via photopin cc

Update: Twitter returned to fully-operational status at about 9:30am Pacific time, after about 45 minutes offline.

Twitter is down right now — hard down. The site is unresponsive, and DownForEveryoneOrJustMe says it’s down for you as well as Yours Truly. In addition, any web address shorteners based on Twitter are down as well, so any t.co links you click on will not work.

Twitter’s site status page confirms the issue, saying simply: “Users may be experiencing issues accessing Twitter. Our engineers are currently working to resolve the issue.”

Visiting the site a few minutes ago I received nothing, but now I’m seeing this:

Source: John Koetsier

Now that is down, and down hard. A little embarrassing?

I reached out to a press contact at Twitter, and she responded with the following confirmation:

Thanks for getting in touch. We’re aware of the issue and are looking into it. Will let you know when we have an update. And keep an eye on status.twitter.com for updates as well.

A few weeks ago, a number of us in the VentureBeat office were wondering what a Twitter outage costs the economy. After all, Twitter is a big part of our economic life: With over half a billion members, Twitter is huge and still growing.

News outlets, retailers, and companies of all kinds depend on the service to promote their businesses.

So we asked an analyst, Ray Wang at Constellation Research. According to Wang, the economy could be losing as much as $25 million per minute of Twitter outage.

Here are his assumptions:

1. Twitter has 500 million users
2. Let’s say 100 million are business users
3. Let’s use an average salary of 125k (blended rate) over 124,800 minutes
   (assuming a 40 hour work week/year)
4. Let’s say an outage means we lose about $1/person/minute
5. Let’s say 25% of that 100M are online during the work hour doing real work during an outage.

So according to the Twitter status update on the site issued … 40 minutes ago … this current outage may have cost the economy $1 billion already.

A couple of caveats:

This is a scribble-on-the-back-of-the-envelope calculation. And work that is not done this very minute on Twitter, or sales that don’t happen right now because you didn’t get that tweet about the big sale, may very well happen tomorrow. Or as soon as Twitter pops back up.

In addition, although I think the “blended rate” amount takes into consideration wages and benefits, most people are NOT making $125,000 per year. So that might need to be sliced in half, or more.

But massive communication utility outages certainly result in real economic loss — even if it’s not $25 million a minute.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

It’s been a rough night for T-Mobile customers, as the carrier confirmed that it experienced a nationwide data outage across its network. (Update: T-Mobile now says data service has been restored.)

We’ve been seeing reports from T-Mobile customers experiencing data issues tonight, though not all T-Mobile customers appear to be affected. T-Mobile’s Twitter support account confirmed that something is up and that the carrier is feverishly working to restore services. (I feel for the poor soul running that Twitter account tonight.)

For now, all T-Mobile customers can do is sit tight and pray that the carrier fixes things before the morning. Or you could join the complaint party over at T-Mobile’s support forums.

It’s been a while since we’ve seen an outage on T-Mobile’s network. Verizon’s LTE network, on the other hand, has had sporadic issues over the last year (that’s the price you pay for being first to LTE).

Design is determining the winners in everything mobile. The most successful players are focusing on one thing: How to make products, services, and devices as compelling and delightful as possible – visually, and experientially. MobileBeat 2012, July 10-11 in San Francisco , is assembling the most elite minds to debate how UI/UX is transforming every aspect of the mobile economy, and where the opportunities lie. Register here.

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

An outage of Amazon’s Elastic Compute Cloud in North Virginia has taken down Netflix, Pinterest, Instagram, and other services. According to numerous Twitter updates and our own checks, all three services are unavailable as of Friday evening at 9:10 p.m. PT.

Amazon’s service health dashboard indicates that there are power issues in its North Virginia data center, most likely caused by severe storms in the region.

Independent checks by VentureBeat of Netflix, Instagram, Pinterest, and Heroku show each site not operating. Netflix has acknowledged the issue on its customer support Twitter account:

Instagram’s support Twitter account has also noted ongoing issues:

Heroku’s status and incident dashboard says: “Our engineers continue to work to restore affected systems. Some production applications are unaffected but many applications are offline. API access is disabled while we restore service.”

As of 8:49 p.m. PT, Amazon claims that “power has been restored to the impacted Availability Zone and we are working to bring impacted instances and volumes back online.” If power has been restored to its North Virginia data center, we would expect that the affected services will be back online shortly. But until then, agitated users will just have to sit back and relax.

Update: As of 9:20 p.m. PT, Amazon said it is still working to fix the issue: “We are continuing to work to bring the instances and volumes back online. In addition, EC2 and EBS APIs are currently experiencing elevated error rates.”

Update 2: As of 9:54 p.m. PT, Amazon said: “EC2 and EBS APIs are once again operating normally. We are continuing to recover impacted instances and volumes.”

Update 3: Independent checks of Netflix and Pinterest show the two sites back and working normally in most regards. Most likely, service is being restored to all services as we update.

Update 4: At 4:42 a.m. PT, Amazon said: “We are continuing to work to recover the remaining EC2 instances, EBS volumes, and ELBs.” At this time, Instagram is still down, while Heroku said it is still recovering some features as of 7:10 a.m. PT.

Update 5: As of Saturday afternoon, all affected services including Instagram appear to be functioning normally again.

With the critical Amazon outage, which is the second this month, we wouldn’t be surprised if these popular services started looking at other options, including Rackspace, SoftLayer, Microsoft’s Azure, and Google’s just-introduced Compute Engine. Some of Amazon’s biggest EC2 outages occurred in April and August of last year.

Let us know in the comments if you are seeing any other services that are down in this massive outage.

Storm clouds photo: divingrocks/Flickr

Newvem, a startup focused on scanning and analyzing companies’ cloud services to optimize their use of Amazon cloud technology, recently did a study of its beta launch participants. The most shocking findings?

A full 40 percent of Amazon Web Services (AWS) users are not prepared for the next outage.

“At least 20 percent of users could easily have withstood the outage, had they implemented best practices,” Cameron Peron, Newvem’s vice president of marketing, told VentureBeat. “And only 60% of users back up all of their EBS volumes.” (An EBS volume is an “Elastic Block Store,” Amazon jargon for data storage.)

“Amazon doesn’t make any promises to back up data,” Peron said. “The real issue is that many users are under the impression that their data is backed up … but in fact it isn’t due to mismanaged infrastructure configuration.”

The problem is not unique to beginners — enterprise users of cloud services have issues as well.

“Light, medium, and heavy cloud users are running clouds where on average 40 percent of their cloud — data, applications, and infrastructure — is NOT backed up and exposed to outage meltdown,” Peron said.

Part of the problem seems to be a perception that when you put services on the cloud, everything is managed for you. Amazon or Microsoft or Rackspace has my back: I don’t need to worry.

“Bottom line: clouds go wild,” said Peron. “The last thing that developers/operations should do is assume that the cloud works on its own.”

Instead, cloud services need to be carefully monitored and managed — especially for what will happen in the event of a service disruption. Proper set-up with services in multiple availability zones can help protect against outages. Careful data and application backup can prevent re-start issues.

“Unhealthy instances” are unresponsive servers within your cloud

In spite of the issues, Peron still feels that Amazon has the best cloud technology:

“Despite all of the hype, AWS is the most reliable cloud out there – period. We’ve seen many of our beta participants leaving other clouds for AWS because it provides Class A infrastructure, elasticity, flexibility, and great business model.”

One option for businesses that absolutely must have complete service reliability is hosting with multiple cloud providers. That’s becoming more of a possibility now that OpenStack is making some headway. OpenStack is an initiative to standardize cloud configurations, making it easier for companies to host on multiple clouds with a minimum of configuration and customization.

Peron agrees that this is a good idea, but it comes with a cost.

“As a best practice in availability, especially to protect a cloud from an outage, a multisite deployment with multi region configuration in AWS and/or between clouds would be one of the basics.” Due to the expenses of running multiple clouds, however, he suggests that “it’s cheaper running a multisite deployment within AWS as opposed to two clouds.”

Newven is a 12-person company headquartered in Tel Aviv, Israel. It’s backed by a $4 million round of initial investment led by Greylock Partners, Index Ventures, and others.

Image credit: ShutterStock

Update: Spotify’s service has been restored for all users, according to the Spotify Status Twitter account. Original post below.

Many users of the popular music streaming service Spotify are unable to log in to their accounts this morning in a large outage, according to a Spotify Twitter account and many agitated users around the web.

“We can confirm that most users are unable to log in at this time,” the Spotify Status Twitter account wrote about an hour ago. “We are working on a fix, watch this space.”

Many users around Twitter complained (as they should) about the outage. One tongue-in-cheek response from Hannah Innit in London notes: “#spotify down. i have absolutely no idea how i’m going to get through the last 27 minutes of the week.”

The company followed the first tweet up with another post, saying: “Still working on login issues. Please note that this will affect our website login as well. Thank you for your patience!”

Spotify has since issued the following update: “Some users are able to log in, but we are still seeing errors. Sorry for the inconvenience, another update is coming soon.”

Is your service out? Let us know in the comments.

Developing post. Refresh for updates.

Photo illustration: Sean Ludwig/VentureBeat

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

After a huge service outage, Virgin Mobile is reporting that its network is up and running once again.

Around 18 hours ago, the mobile provider noticed the widespread loss of service and issued a statement saying, “We apologize, but we are currently experiencing a national data and text messaging outage. Our engineers are working diligently to resolve it.”

Customers in some areas reported that service was up and running again around five hours after that statement was issued. However, Virgin Mobile was only able to confirm that all service had been restored as of this morning.

“We are happy to report that the outage has been resolved,” wrote a company spokesperson on the Virgin Mobile Facebook page. “Again, we are sorry for any inconvenience.”

Consumers were also advised in that statement to try the old turn-it-off-and-then-back-on-again trick if they still didn’t have text and data service.

In somewhat related news, Virgin Mobile released the commercial below a week or two ago. In it, Sir Richard Branson claims that he’s gonna “service you — every last one of you” with unlimited data and texting. If you were affected by the service outage and feel the need for some catharsis, the ad spot also contains a Richard Branson lookalike slap reel, which you can replay as many times as it takes to get the rage out of your system.

Image courtesy of Creatista, Shutterstock

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Microsoft’s Azure cloud platform suffered a serious outage Tuesday evening and many customers are still without full access, according to the Azure Service Dashboard.

Azure is used by many companies to host and build web applications through Microsoft’s own data centers.

According to a circulated statement from Microsoft’s outside PR firm, Microsoft first noticed the issue Tuesday at 5:45 PM PST and did what it could to resolve the issues. “Windows Azure engineering teams developed, validated and deployed a fix that resolved the issue for the majority of our customers,” the statement reads. “Some customers in 3 sub regions – North Central US, South Central US and North Europe – remain affected. Engineering teams are actively working to resolve the issue as soon as possible.”

As of this writing, the Azure Service Dashboard indicates that much of the services provided by Azure have been restored. But the biggest glaring problem appears to be the company’s SQL Azure Data Sync feature, which is down in every region of the world it serves. This feature allows developers to sync and backup important database and coding.

Vineet Jain, CEO of Egnyte, a cloud storage provider for businesses, said in an e-mailed statement that the downtime highlighted a flaw with traditional cloud infrastructure. Egnyte offers a hybrid cloud solution that helps eliminate downtime. Jain writes:

The outage that occurred today with Microsoft’s Azure cloud platform can be crippling and cost millions if not billions of dollars, but more importantly highlights the down side of a pure cloud strategy in the enterprise. While we certainly don’t relish these moments, the downtime can be significantly mitigated if organizations were to adopt a hybrid cloud strategy. By maintaining a behind the firewall presence and syncing that to the public cloud, companies are creating an insurance policy just for these situations. At the same time they can keep downtime to a minimum and insure their employees are as productive as possible in an emergency situation like this. Hybrid cloud is the smartest path to a productive workforce for today’s enterprise.

Did your Azure service go out? How many features stopped working?

Storm clouds photo: divingrocks/Flickr

With so many games requiring online passes to activate multiplayer content, or even to unlock single player elements, this lack of store functionality is a big issue. If the problems with the store continue, this is likely to impact a lot of gamers over the weekend in the affected regions.

While the digital storefront is still usable, and items can be purchased, they can not be downloaded. When attempting to look at their download history, many users are simply receiving the error code 80023102, along with the message “No content was found”.

Community Moderator SuperFastZombie responded to the first reports of this issue on 3 Feb, saying, “This is an issue we are aware of following the PSN maintenance last night. We are currently working on a fix, there should be nothing you have to do your end so please bear with us and I will update this thread as soon as its sorted.”

Reports of this problem have come in from many countries across European, including the UK, France, Spain and The Netherlands. It is also affecting accounts held in Australia and New Zealand. The problem does not seem to be affecting all European Playstation Network accounts, with some users reporting that they can still download store content. The North American store appears to be working normally.

A second update from SuperFastZombie was provided a few hours ago, but it gives no clue as to when the situation is likely to be resolved:

“Update – As of this morning I can confirm that we are still working to resolve this issue.  For those of you worried about losing content I can assure you that this is not the case.  Your download lists and purchase history have not been affected, the problem is the ability to access them from your console.

Once this is fixed all your content will be available as before. I will keep you updated even if there is no update, if that makes sense?”

We will update this post if we hear any further news.

Update: UK Community Team member Yaster has now stated, “I’m being told that the cause of the problem has been identified, but a resolution has yet to become available. I’ll continue to push for updates.”

Update 2: Another member of the Playstation Community Team, RabidWalker, issued the following update at 10:40am GMT, on 6 Feb: “Currently work is being carried out to resolve the issue users are having, however we have not been given an exact time for this to be completed yet.”

Update 3: Sony Entertainment Network reports that this issue is now resolved. “Hi all. We know some of you may have experienced some issues with PS3 downloads in the last 24hours but this should all be working now.” Ignoring the fact that this has been a problem for over 72 hours, it is nice to finally see a resolution.

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Losing network access seems to be a common occurrence for Verizon Wireless customers these days. They woke up to yet another nationwide data outage this morning.

The outage, which affected the carrier’s 3G and 4G LTE data networks, drove subscribers to Verizon’s technical support board and support Twitter account. Verizon hasn’t yet divulged the cause of the outage, but now it seems service is slowly being restored, Engadget reports. (Above, Verizon’s LTE network coverage in the North East.)

Verizon suffered a similar outage for its LTE network earlier this month, as well as back in April. Network outages are a fact of life for practically every service, but for Verizon they could severely tarnish the reputation of its LTE network.

The carrier was the first in the country to roll out LTE, and it has by far the most coverage. But since it’s a fairly young network, Verizon needs to make sure that any problems are fixed quickly — otherwise consumers will wonder if LTE is worth the extra cost.

The 4G technology is already causing issues with some consumers due to its heavy battery usage. Verizon certainly doesn’t need more problems with its LTE experience.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Siri, Apple’s new voice assistant feature for the iPhone 4S, is apparently suffering from an outage, which has rendered it useless for many iPhone owners. The outage began at around 11 am PST today.

Siri isn’t just a simple piece of voice control software. To function properly, Siri must communicate with one of Apple’s servers for additional information. For example, when you ask Siri what the weather is like in Nashville, Tenn., it plays an elaborate game of telephone with a main database that has the information, and then updates you with the answer.

This is precisely the reason jailbreaking the iPhone 4 and the iPad to run Siri is so difficult.

We’re contacting Apple for additional information and will update you with what we find out. In the meantime, feel free to let us know if you’re experiencing the Siri outage by leaving a comment below.

Update: We called Apple’s customer service number to find out more information. Despite toggling off the Wi-Fi and restarting the phone as they suggested, I was unable to get Siri to connect (not that I’m surprised). The Apple Care supervisor I spoke with said it’s entirely possible that it’s a network outage.

Update 2:40pm Pacific: Siri is still offline, according to our tests, and numerous other users are also reporting outages. So far, Apple has not issued an official statement, and it has not responded to our requests for comment.

Update 4:04pm Pacific: Siri functionality has been restored for some iPhone 4S users — including myself. At this point we still don’t know what was causing the outage, or how widespread it was, and Apple still hasn’t responded to our requests for comment.

Siri did respond, however. I asked her, “Siri, where you been all day?” Her response? A number of day spas and hair salons (Screenshot).

Thanks to Armen over at iSmashphone for tipping us off, and to VentureBeat’s Heather Kelly for the suggestion about asking Siri where she’s been.

RIM first showed-off its new BBM Music app back in late August, but it is just now rolling out to all users. The service costs $5 a month and works differently than other music services.

Each paid user of BBM Music can pick 50 songs to post to their BlackBerry Messenger profile, and anyone who is friends with that user can then listen to the posted songs. So, the more friends you have on BBM Music, the more music you can listen to. Users are allowed to swap out 25 of the posted tracks each month.

The company officially describes the service as such:

BBM Music is a BBM-connected social music service that allows BBM and BlackBerry smartphone users to discover, play and grow their music collections together. In other words, BBM Music helps you connect with your friends around music on a whole new level.

The introduction of BBM Music isn’t particularly groundbreaking, but RIM is exploring new ways to connect with (and hopefully retain) owners of its smartphones. RIM faces stiff competition from Apple’s iPhone 4S and various Samsung and Motorola Android phones, and has gone through a rough patch the past few months. Not only did the company experience a disastrous worldwide outage recently, but its BlackBerry PlayBook tablet had a considerable OS update pushed back to February.

Personally, I think the BBM Music service is going to be a considerable flop because there are already better options for on-demand music streaming for BlackBerry devices. Spotify debuted its BlackBerry app just a few weeks ago, and Rdio and Rhapsody have high-quality apps that provide a better music-focused experience. Most importantly, they don’t limit you to just 50 tracks.

Watch the offical BBM Music intro video below for more details:

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

After the worst service outage in its history last week, Research in Motion said Monday that it will try to appease its BlackBerry users by giving them free apps worth more than $100.

The BlackBerry brand has struggled to keep pace in the past few years with the influx of Apple’s flashy iOS phones and the swelling number of powerful Google Android devices. Add in a considerable outage that left tens of millions of BlackBerry owners without access to e-mail, web and messaging abilities and the company may be looking at a mass exodus.

RIM co-CEO Mike Lazaridis apologized via video for the horrific service disruption that spanned the globe last week, saying “We’ve let many of you down.” Lazaridis and RIM followed up on those comments Monday by saying the company would give all of its users free apps and give its valuable enterprise customers a free month of high-level technical support.

RIM’s other CEO Jim Balsillie told the Associated Press that the company is taking the outage very seriously and stressed the company’s customers would be shown their appreciation by the free applications.

“This is something we would like to offer as our form of thanks,” Balsillie said. “It’s a $100 worth of premium apps. It’s a substantial offer to our 70 million users around the world.”

The applications RIM plans to give away will appear in BlackBerry App World during the coming weeks, and the offer stands until the end of the year. The free apps include The Sims 3, N.O.V.A., iSpeech Translator, Bejeweled and Texas Hold’em Poker 2.

RIM will be holding a BlackBerry developer’s conference tomorrow in San Francisco, where the company is expected to introduce a major software update to its flailing tablet, the BlackBerry PlayBook. On top of that, the company will likely show off a prototype or two of its next-generation BlackBerry device that will run the new QNX operating system.

Even with a free application giveaway, the company is up against a strong tide of change. If RIM doesn’t wow its developer base — usually the most ardent of fans and critics — it will be hard for the company to recover its stance as a reliable and steady mobile manufacturer.

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Poor Research in Motion. Just two days from Apple’s iPhone 4S launch, the BlackBerry maker is in the midst of a three-day long service outage that has now spread to North America.

The outage, which is the worst since 2009 for RIM, could cause many consumers and IT departments to rethink their mobile strategy. Clearly, relying on RIM’s proprietary systems has its drawbacks.

Thanks to the outage, tens of millions of BlackBerry users have been experiencing spotty email, Internet, and BlackBerry Messenger connectivity.

“The messaging and browsing delays being experienced by BlackBerry users in Europe, the Middle East, Africa, India, Brazil, Chile and Argentina were caused by a core switch failure within RIM’s infrastructure,” RIM said in a statement on Tuesday. “Although the system is designed to failover to a back-up switch, the failover did not function as previously tested. As a result, a large backlog of data was generated and we are now working to clear that backlog and restore normal service as quickly as possible.”

As Reuters points out, the outage could lead some IT outfits to consider letting employees use their personal devices, instead of buying and deploying RIM hardware and services. Many companies are now trying to integrate personally-owned iPhones and Android devices into their corporate networks. An outage like this could be just the push some progressive IT workers need to convince their companies to jump off the BlackBerry ship.

The Sony PlayStation Network (PSN) suffered an outage today and the online gaming service, which had more than 77 million registered users earlier this year, still isn’t working.

After Sony’s big PSN outage in April, it’s easy to assume the worst about a little downtime with the network. The six-week outage earlier this year came after hackers found a flaw in the network’s security. More than 10 million credit card numbers may have been stolen in that attack.

Now is not a good time for the outage, since it’s the middle of the fall video game selling season and Sony’s Resistance 3 shooter game is generating a lot of demand for multiplaying gaming sessions.

The service has been down for a number of hours. Sony recently distributed a v3.72 firmware update that went live on the 19th. Users have to download that to get online. But having downloaded that patch won’t help you get online during this outage.

Sony has pledged to do better on security, saying it would spend more on the network to ensure its safety.

Like Nintendo last night, Sony also cut its profit forecast. Sony said it lost 15.5 billion yen ($199 million), compared with a profit of 25.7 billion a year earlier. The announcement shows how quickly things can change for dominant companies in industries with rapid changes and unforeseeable events.

The profit outlook for the fiscal year ending March 31, 2012, was cut from 80 billion yen to 60 billion yen. Sales are now expected to be 7.2 trillion yen, instead of 7.5 trillion yen. Analysts expected a loss of 2.5 billion yen.

In the quarter, Sony’s movie division earned an operating profit of 132.1 billion yen, up 9 percent, thanks to films such as The Green Hornet, Battle: Los Angeles, and Just Go With It. But consumer products and services reported an 18 percent decline to 732.3 billion yen because of lower LCD TV and PC revenue. Music sales were down 0.6 percent.

The PlayStation business saw a slight profit, thanks to PlayStation 3 software sales. Sony sold 26.1 million games in the quarter, up 5.2 percent. But PS 3 sales fell by 25 percent, down to 1.8 million units sold, probably because of the six-week outage the company suffered after the PlayStation Network was hacked.

So while Sony’s president in charge of network entertainment Tim Schaaf, who oversees PSN, said the downtime was a “great experience,” it wasn’t such a good deal for the company’s bottom line.

The PS 2 sold 1.4 million units, down from a year ago, and the PlayStation Portable sales rose 50 percent to 1.8 million units.

One of the big products coming this fiscal year is the PlayStation Vita, a replacement for the PSP and Sony’s best hope for competing with the Nintendo 3DS and Apple’s mobile devices. Damage to Sony factories from the quake and tsunami amounted to 5.3 billion yen, most of it covered by insurance. But Sony said the recovery was progressing faster than expected. A strong yen also hurt Sony’s profits, as it made Japanese products more expensive overseas. Sony also said it will start selling its first tablet computer soon.

Sony said that user logins in North America on the PSN had been restored to the same level as before the attacks.

As for the game business, Billy Pidgeon, an analyst at M2 Research, said, “These numbers are very bad for Sony considering the competition and where we are in the cycle.”

PSP sales are up in Japan, but that is an anomaly, Pidgeon said.

“PS3 hardware numbers ought to be up, not down,” he said. “There is great software in the pipeline, but hardware price may need steep cuts or the new software won’t help. Xbox 360 is still selling strong and is capable of price elasticity.
Microsoft can cut prices and potentially murder Sony through the holiday into 2012.  In fact, Microsoft ought to use this opportunity to improve positions worldwide.

The management reshuffle is unsurprising, as Sony has faced many issues recently along with the attack on the PlayStation Network. Not only was it slapped with lawsuit associated with the theft of personal data during the PSN attack, it also had its Sony Pictures website hacked with personal data stolen there too.

Kazuo Hirai, president of Sony’s games unit since 2006, will be replaced by Andrew House, head of the European video game unit, on Sept. 1. Hirai will continue to oversee the gaming business as the unit’s chairman and retain his post as executive deputy president of the entire Sony group.

With the upcoming launch of the PlayStation Vita portable console and the push to make the PlayStation 3 more popular, House will have plenty to do. The Vita faces stiff competition with the Nintendo 3DS, the successor to the mammoth-selling Nintendo DS.

Are you looking forward to Sony’s upcoming offerings? Do you trust them to keep your personal data safe again?

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

Citing industry sources, Edge reported a growing trend of PS 3 consoles being traded in for cash or Xbox 360s. It also said sales of PSN points cards, which can be used to purchase games or movies on the PlayStation Network, have decreased. More than 77 million PSN registered users haven’t been able to play online games or download new content for more than three weeks because of a hacker attack that undermined the security of the network.

Hardcore gamers are pretty passionate about their game consoles. Many stuck with Microsoft even as it struggled through its billion-dollar Xbox 360 defects problem a couple of years ago. But some gamers are on the fence now with PSN. One store manager in the United Kingdom told Edge that from the second week of the outage onward, there has been a 200 percent increase in PS 3 consoles being traded in.

Tom Mestdagh, of Belgian indie retailer Gameswap, told Edge, “At the end of each month people come to sell their consoles. People that need money [to pay bills]. What’s different this time around is that they are bringing in PS3s together with all their games, and they don’t want money, they want an Xbox 360. In every case it is because of Black Ops and/or Modern Warfare 2.”

The timing is great for Microsoft because it landed an exclusive for the Call of Duty Black Ops map pack, Escalation, which launched last week. Millions of Call of Duty fans can play that new content on Xbox Live or the Xbox 360, but not on the PS 3.

Microsoft said it sold 297,000 Xbox 360s in April, up 60 percent from a year ago, according to market researcher NPD. The outage started on April 20. Sony sold 204,000 PS 3 consoles in April, up 13 percent from 180,800 a year earlier. Sony said PS 3 software sales were up 40 percent from a year ago.

Patrick Seybold, a spokesman for Sony, said, “We continue to see a lot of great momentum at the retail level and appreciate the support we’ve received from our partners, retailers and customers since the criminal attack on the PlayStation Network. We hope to have services restored as soon as we can.”

Mestdagh said the PS 3 returns are up 200 percent compared to March, even though that only includes 10 days of the month. Edge acknowledges that it has surveyed a small sample of retailers. What do you think? Please take our poll below. Please explain your vote in the comments.

Online Surveys – Zoomerang.com

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

While the company isn’t blaming Anonymous for the attack itself, it said the hackers that stole gobs of sensitive data about PSN users were able to break into the network while it was defending itself from denial of service attacks orchestrated by Anonymous. Anonymous took on Sony after the company went after famed PS3 hacker George Hotz, who reverse engineered the PlayStation 3 to run unauthorized programs.

When the PlayStation Network crashed on April 21, Anonymous said it was not behind the attack. Instead, the hacktivist group said, “Sony is incompetent.” But an observer of the IRC forum used by members of Anonymous said the attackers behind this current Sony outage appear to have learned their methods from Anonymous’ activities of two weeks ago.

This really is not the time for Sony to start playing with fire. Anonymous doesn’t regularly respond to blame and threats, but because the network of hackers has taken on Sony before, there is no guarantee Sony’s latest accusation won’t spark some kind of retaliation. Anonymous has proven time and again that it is a force to be reckoned with. Sony has to focus on beefing up its network, not trying to shift blame around and incite more attacks against the already feeble network.

Hackers attacked the PSN on April 19, forcing the Japanese company to bring down the network, which has more than 77 million registered users. The nightmare then continued after hackers broke into the company’s Station.com site, which serves as a host for its PC games like Everquest. Hackers were able to steal information from as many as 24.6 million accounts on that site, according to Sony. In all, more than 100 million accounts might have been compromised.

The PSN breach was a massive security gaffe that has caused the U.S. government to get involved and demand answers — such as who attacked the network and what users were affected. Sony has sent warnings to PSN users about the possible credit card theft. The whole ordeal spawned an apology from Sony that lasted more than an hour and a half.

The network has been down for more than a week, denying 77 million registered gamers the ability to play online games, watch movies, listen to music or download other entertainment to their PlayStation 3 consoles and PlayStation Portable handhelds. The PlayStation Network is a critical service that competes with Microsoft’s Xbox Live online gaming service — as well as other online gaming services. There are also 948 games now available in the PlayStation Network store, as well as 4,000 pieces of add-on content for games.

[Photo: Ars Technica]

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

Today, Sony’s Kaz Hirai repeatedly apologized to users for the data breach. The company said it would begin restoring the service for the PSN and Qriocity music and video service within a week. It offered to reimburse customers for the costs related to credit card reissuance and for any needed identity protection service. Sony is also offering a “welcome back” initiative to compensate users for their losses. That includes free content, which may vary based on the region where the user lives; it also includes 30 days free subscription to the paid PlayStation Plus service.

Sony also pledged to move its service to a new data center and to beef up its security so this doesn’t happen again.

The company has been criticized for acting too slowly to respond to user concerns about the break-in and whether or not personal information such as credit card numbers were stolen. Sony’s Hirai said as many as 10 million credit card numbers may have been stolen by the hackers.

Frankly, I think Sony did a good job here, although it should have done it much sooner. It pretty much said everything that I expected it to say. It probably could have disclosed more about the exact nature of the attack, but it erred on the side of not disclosing this information to prevent future attacks.

Are you satisfied that Sony said enough to mollify angry users?

Online Surveys – Zoomerang.com

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

Hirai and other Sony executives apologized to users immediately upon coming out on stage at a press conference in Tokyo. As they did so, they bowed in front of the press. The executives described what happened with a “highly sophisticated attack” against the PSN.

Hackers attacked the PlayStation Network on April 19, forcing the Japanese company to bring down the network, which has more than 77 million registered users. The outage has been one of the most humbling corporate events for Sony. The security gaffe, which happened at the same time that Amazon’s web services data center crashed, could shake the faith that consumers have in the internet cloud, where corporations say they will protect their personal data.

“This criminal act against our network had a significant impact not only on our consumers, but our entire industry,” Hirai said. “These illegal attacks obviously highlight the widespread problem with cyber-security.  We take the security of our consumers’ information very seriously and are committed to helping our consumers protect their personal data.  In addition, the organization has worked around the clock to bring these services back online, and are doing so only after we had verified increased levels of security across our networks.”

Speaking through a translator, Hirai said, “We advised customers to be vigilant” about the possibly stolen credit card numbers. He said that Sony is cooperating with authorities in a criminal investigation in pursuit of the hackers. Hirai said that as many as 10 million credit cards numbers may have been stolen. That number appears to be all of the credit card numbers that Sony has, since many of the 77 million registered users log into the network and play online for free.

In making the announcement, Sony pretty much followed the script I offered it. That doesn’t mean I knew a lot about what they would do or that I am especially smart. Rather, it was just so obvious what the company had to do, and it was surprising it took 10 days to do it. Still, it was good that Sony’s executives talked to the press and answered all questions from the media, at least as well as they could. They stayed on stage for an hour and 42 minutes. In that respect, Sony may have earned some good will tonight and begun the process of repairing its damaged reputation.

The apology press event took place in Japan at 2 pm Sunday Tokyo time, or 10 pm on Saturday evening Pacific time. Hirai, (pictured above), is representative corporate executive officer and executive deputy president of Sony as well as head of the game business. He was joined by Shinji Hasejima, senior vice president and chief information officer at Sony, and Shiro Kambe, senior vice president of corporate communications at Sony.

Sony said it wasn’t sure whether hackers had stolen users’ credit card numbers, which were encrypted. But users have begun to complain about false charges on their credit card accounts. Other corporations can’t be smug as they watch Sony and Amazon recover, since no one can be certain that their networks are secure from hackers or technical glitches.

The PlayStation Network is Sony’s all-important hub of the digital age. It enables PlayStation 3 and PlayStation Portable users to go online and find more content to download to their machines, from movies to TV shows. The PSN also hosts Sony’s Home virtual world and its console-based Sony Online Entertainment games: Free Realms and DC Universe Online. Also, the service allows users to store their saved single player games and engage in multiplayer combat online. The Qriosity service, which also went down, gives users access to online movies and music. In other words, the PSN and Qriosity are central to Sony’s survival in the digital age.

Back in March, Sony chief executive Howard Stringer concentrated more power in the hands of Hirai, who once headed the U.S. PlayStation business for Sony. Hirai is now the No. 2 executive next to Stringer and may be in line to succeed him when Stringer steps down around 2013.

The problem for Sony is that this story, like the outage itself, has refused to die. We’ve run 15 stories about it so far, mainly because users seem hungry for more information because there just hasn’t been enough detail coming from the official source. Sony has been good about putting updates on the PlayStation blog, but it hasn’t been fast enough.

For instance, the company said the attack occurred on April 19, forcing Sony to shut the services down. Sony notified users on April 22 that an “external intrusion” led to an outage. It hired forensic computer investigators to figure out what happened. After their initial inquiry, Sony announced on April 26 that personal user data for all 77 million of the PSN and Qriosity services had been compromised and their credit card numbers have been stolen. On April 26, Sony began informing all affected users of the possible credit-card data breach.

Hirai said that hackers penetrated a web application server and made a tool to give themselves illegal access to the database. They were able to access a database with data that included credit card numbers.

Hirai said the company is moving servers from San Diego, Calif., to a more advanced data center with better security. It is also installing more security systems with automated software management and enhanced levels of data encryption and better ability to detect data intrusions. The company is adding more firewalls too,and it is adding a new chief information security officer, reporting to Hasejima. PS 3 will have a new system software update requiring users to change their user names and passwords. The password can only be changed on the same PS 3 on which the account was created or via validated email.

To deter identity theft, the company is asking customers to be vigilant and check their credit card statements. Customers can check their purchase history on the PSN via Sony’s customer support system. Sony said it will not ask for credit card numbers and warned users to beware of possible phishing schemes sent via email. Sony will consider covering the cost of reissuing credit cards if consumers wish to do so. It will provide a complimentary offering for identity theft protection services in each affected country.

The company will roll out a program with a selection of premium services for consumers. This “welcome back” content will have free downloads, and 30 days of free PlayStation Plus network service. Normally, Sony charges extra for the Plus service, while membership in the PlayStation Network is free. Current members of PlayStation Plus will get 30 days of free service. Music Unlimited, powered by Qriocity, subscribers will get 30 days of free service in countries where it is available. Sony could not quantify the value of this free service and content offering for its users.

The company is planning on restoring the services as soon as it can, with some services starting this week. Hirai said that Sony’s network services are key to its strategy and that it will continue to strengthen them and learn from this incident. The attack targeted Sony’s data center in San Diego, Calif.

Separate from the attack that brought down the PSN, Hirai said that Sony’s sites had been subjected to attacks from Anonymous, the hacktivist group that targeted Sony during its litigation with “jailbreaking” hacker George “Geohot” Hotz. During these attacks, hackers dug out personal information on Sony executives and published it on the web. Sony is cooperating with authorities on those attacks as well.

In closing, Hirai bowed again and apologized again. Responding to press questions, Hirai said he had received questions about the matter from members of Congress in the U.S. and would answer the questions. In about a week’s time, the service is expected to restart. Hirai said he had not received reports that actual damages had been incurred related to the credit card exposure.

Hirai said Sony will advise users to change their passwords and not use the same ones over again. He said Sony has operated an online gaming network since the launch of the PlayStation 2 and has had to deal with online security for a long time. But he noted that the new situation is different. He noted that Anonymous has been attacking different corporate sites around the world for quite some time.

Sony’s network services are core to its strategy and it has to improve its security, Hirai said.

“We are living in a network society and we will deal with this kind of situation the best we can,” he said.

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

Sensitive information about PSN users was stolen by hackers during an extensive attack and intrusion on the online gaming network, according to Sony. It was a massive security gaffe that has caused the U.S. government to get involved and demand answers — such as who attacked the network and what users were affected. Sony has sent warnings about the possible credit card theft to PSN users.

But one of the most critical parts of the downtime was Sony’s extensive silence throughout the whole ordeal. It was several days between Sony’s first announcement that the network would be down indefinitely and coming clean about user information being stolen. Users were kept in the dark about their information being taken by hackers. It’s uncharacteristic of a massive gaming company, which usually has to rely on a huge trust surplus to survive in the hyper-competitive video game market.

Just this morning, VentureBeat’s gaming guru Dean Takahashi suggested Hirai hold a press conference to apologize to gamers everywhere and offer some kind of compensation. The PSN outage has done significant damage to the company’s credibility and it’s unlikely that gamers will trust them without thinking twice again, he said. There hasn’t been any indication about what the press conference tomorrow will be about, but it’s almost certain that there will be an apology in there somewhere.

The network has been down for more than a week, denying 77 million registered gamers the ability to play online games, watch movies, listen to music or download other entertainment to their PlayStation 3 consoles and PlayStation Portable handhelds. It’s a critical service that competes with Microsoft’s Xbox Live online gaming service — as well as other online gaming services. There are also 948 games now available in the PlayStation Network store, as well as 4,000 pieces of add-on content for games.

[Photo: ftchris]

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

July 9-10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Verizon Wireless subscribers on the carrier’s super fast LTE 4G network are now paying the price for being early adopters, as many users have been reporting sporadic 4G outages across the country.

In a statement released a few hours ago, Verizon said it has discovered the cause of the issue — which began last night and has persisted throughout Wednesday — and is working to resolve the outage.

As All Things Digital’s Ina Fried points out, the timing of the outage is unfortunate for Verizon VP Nicola Palmer, who boasted at VentureBeat’s Mobile Summit on Monday about how smoothly the company’s 4G rollout has been.

Verizon says that 4G users will still be able to make calls and that it’s working to restore data connections. Customers won’t be able to activate 4G devices at the moment, so hold off on picking up that new HTC Thunderbolt. The company says it will restore the network on a market by market basis, and it will offer updates soon.

But some of the data — 0.07 percent of the eastern region — hasn’t been recovered, and the company is contacting the limited number of customers who have lost data. One of the lessons being learned here is that backup sometimes isn’t enough, and customers who use web services may have to use multiple cloud vendors in the future.

The Amazon crash has raised questions about how much users will be able to trust cloud storage. Some companies using Amazon’s service are offering their own customers freebees to make the hassle a bit more palatable. For example, HootSuite is offering $50 credit for customers affected by the Amazon downtime.

The outage, which happened at the same time Sony suffered a hacker attack that brought down its PlayStation Network, brought down numerous high-profile Amazon customers, including Quora, Sencha, Reddit and Foursquare. Amazon said that a detailed post mortem is coming for the Elastic Block Store outage, and it is looking into the root causes. One status message report said that a “network event” caused the service to “re-mirror” a large number of EBS data in the eastern region.

On Monday, Amazon reported that its profits fell 33 percent as it spent to build more more and more warehouses.

The moderator, who posted the information on news aggregator Reddit under the user name chesh420, said the latest version of a custom firmware for the Playstation 3 called Rebug.me let Playstation 3 owners with that firmware installed access a developer network. They could then put in a fake credit card number and download content as they pleased without any kind of verification of the number on Sony’s back-end.

The latest version of Rebug.me was released at the end of March, and the first guides detailing how to get modified Playstation 3 consoles onto the developer network came out around a week after that, the moderator said. Sony brought its Playstation Network down four days ago without offering many details about what happened and said it was beefing up security. It has acknowledged that an “external intrusion” brought down the network and that it is trying to beef up its security. Meanwhile, it has said it is still trying to determine if any user credit card information was stolen.

In a blog post this morning, Sony spokesman Patrick Seybold said, “I know you are waiting for additional information on when PlayStation Network and Qriocity services will be online. Unfortunately, I don’t have an update or timeframe to share at this point in time. As we previously noted, this is a time intensive process and we’re working to get them back online quickly. We’ll keep you updated with information as it becomes available. We once again thank you for your patience.”

The PlayStation Network, or PSN, is Sony’s attempt to engage its video-game console and PlayStation Portable audience in a larger online entertainment network that includes online games, movies and TV shows, as well as Sony exclusives such as Qore, a digital magazine, and The Tester original TV show. It’s a strategic asset for the console maker as more of the-video game audience moves into the online world.

There are 948 games now available in the PlayStation Network store, as well as 4,000 pieces of add-on content for games. There are also 31,000 movies and TV shows available for download, not counting the content available on Netflix, Hulu, and sports content.

VentureBeat has contacted Sony for comment and will update the story as soon as more information is available.

---

GamesBeat 2013 is our fifth annual conference on disruption in the video game market. You'll get 360-degree perspectives from top gaming executives, developers, and analysts on what’s to come in the industry. Our theme this year is “The Battle Royal.” Check out full event details here, and grab your early-bird tickets here!

---

The explanation may not be clear if you’re not involved in IT. It describes several database errors and the team’s attempt to fix them. The main message, as summarized for less technical users, was that the downtime didn’t just involve a single error, but rather “a number of things going wrong at once.” Eventually, in order to fix the problems, Facebook had to shut the site down, fix the error, then slowly allow users to return.

Besides taking down the Facebook site itself, evidence of the downtime was visible elsewhere on the Web, as the outage broke the Facebook “Like” buttons that many websites (including VentureBeat) had incorporated.