Monday, August 11, 2008

Gmail Outage: How to cope WHEN technology fails?

Outages happen.

Earlier this afternoon, the Twitter community was abuzz of cries that Google's popular email service, Gmail, was down. While this outage appeared system-wide and affected millions of users, some downtime can affect certain elements of a system with as few as a single user.

I thought I'd take this opportunity to pass along some basic troubleshooting tips. These, while certainly not exhaustive, may help you deal with future outages:
  • Plan -- The best thing you can do to prepare for any outage (power, email, computer malfunction) is to plan ahead. While this might seem incredibly obvious, think of the number of times you've fumbled around a dark, powerless, house looking for flashlight batteries? You probably promised yourself you'd put fresh batteries in an easy-to-find spot "just as soon as the lights come back on". (Did you?) In the case of technology we rely upon daily, planning ahead is just as critical. While some outages (this latest Gmail as an example) lasted just a few minutes, other outages may be days, weeks, longer. Think about each system you use daily and how you would live without it for minutes, hours, or days.
  • Backup -- While most free email systems are robust and unlikely to break (and if they do, will likely be back quickly), remember you "get what you pay for". Just like your hard drive, it's crucial to have some degree of redundancy. Now might be a great time to set up a backup email account (with a completely separate provider) in case your primary account goes down. Depending on how much you rely on your mobile provider, internet connection, or other online services -- a backup might be a good idea for these as well.
  • Diversify -- With many users switching to less-expensive telephone lines from Voice-over-IP providers like Vonage or services from internet providers like Comcast or Verizon, you may be putting all your eggs in one basket: If your connection to the Internet goes down, so too does your home phone service. Following the logic of backup, you'll probably be able to use your mobile phone should you lose internet connectivity at home. Some businesses may find having a few backup circuits from the telephone company worth the cost -- or even a backup internet connection for those accidental "fiber cuts" we dread. (Don't forget: telephone lines that run on internet connections will go out during a power outage if your cable/DSL modem and home networking gear do not have backup power from a UPS or standby generator!)
  • Diagnose -- I like to advise users to take a step back, thinking about a current problem or outage from an objective standpoint. Ask questions like: Does this work on another computer? Are my neighbours (or coworkers) having similar trouble? Did something change (new software installed, different internet connection, etc.) on my end? Did it work before? Think outside the box, you may be able to fix your own problem!
  • Report -- Tell someone about your experience. Report the facts, objectively as much as possible, and provide details. Helpful information: time, symptoms, duration, specific observations, troubleshooting you've tried. Less helpful information: frustration, anxiety, vague recollections, exaggerations, generalizations (Are you sure the ENTIRE Internet is down.... or perhaps just YOUR connection TO the Internet? ;-)).
  • Patience -- Now I understand there are mission-critical systems that absolutely cannot fail. I've supported these systems and understand that getting them back up and running is rarely an exercise in patience for management, users, and the pager-carrying IT guys tasked with fixing them... but for the rest of the world: email, blogs, web browsing, and certainly social networking sites -- can likely wait a few minutes. Take a break, catch up on your filing or offline projects, have an impromptu brainstorming session or just bang your head against the desk repeatedly. In many cases the system or service you "can't live without" will be back before you know it.
(I'm also a fan of bringing the IT Guys that get you back up and running home-baked treats... but that's entirely optional (and perhaps somewhat biased?) advice.)

Do you have any more outage-related advice? Horror stories? Feel free to post in the comments.

And hey, look at that... Gmail's back up and running (for me, anyway)! Yay!


Justin said...

I usually prefer to just cry.

corey said...

I love to just hit refresh thinking that this action will speed the process along. But it also gives me time to run downstairs from my desk and have a smoke with the DJ's from KISW.

Tim [ codearachnid ] said...

There is no coping - if Google (or any myriad of their services) goes down then the world ends.