At least 20,000 Gmail users were in a panic last weekend when they logged into their accounts to find that all of their email, chats, photos, and documents were no longer available.
Many of the affected users were simply greeted with the Gmail "welcome" message as if their accounts were newly created.
Google says the problem was due to a storage software update bug, and has offered assurances that the data was in fact backed up and will be restored.
"Imagine the sinking feeling of logging in to your Gmail account and finding it empty. That’s what happened to 0.02% of Gmail users [Monday], and we’re very sorry. The good news is that email was never lost and we’ve restored access for many of those affected. Though it may take longer than we originally expected, we're making good progress and things should be back to normal for everyone soon," wrote Ben Treynor, VP Engineering and Site Reliability Czar, on the Gmail blog.
What is quite interesting about the event is the revelation from Google that the data for Gmail is backed up on tapes. Yes, Google, which maintains some of the largest network storage systems in the world, backs up data on relatively antiquated tapes.
"To protect your information from these unusual bugs, we also back it up to tape. Since the tapes are offline, they’re protected from such software bugs. But restoring data from them also takes longer than transferring your requests to another data center, which is why it’s taken us hours to get the email back instead of milliseconds," Treynor explained.
Seth Weintraub of Fortune.com estimates that it would require somewhere in the neighborhood of 200,000 tapes to complete one backup of all Gmail accounts.
[Editors note: Corrected Text 3/2/11: Article had stated the 200,000 figure was to backup one account - that was incorrect. The figure is the estimate to backup all Gmail accounts one time. Thanks to some alert readers for pointing out the error.]
And the 20,000 low-ball estimate of affected users only represents .02% of all Gmail users - barely a drop in the bucket.
Google is not short on resources, and the company represents what is assumed to be the cutting edge of data storage and a working model for the future of cloud computing.
The revelation that the future of cloud-based data storage technologies and services could, for now, hinge on the use of tapes for backup should be cause for discussion, if not worry.