What a day...

When I woke up today, I had two things I absolutely wanted to do today - do a blog post and update SportsLizard's mobile site. I figured those two things would have taken about an hour and I could spend the rest of my Friday working on writing a marketing plan for the next phase of marketing for iPrioritize. Boy was I wrong.

It started out great - I woke up to my inbox filled with the usual newsletters and realized that two of the biggest had published stories written by yours truly! With that comes a rush of traffic to SportsLizard and iPrioritize and a bunch of iPrioritize new registrations. And with those comes the typical customer service issues - comments, suggestions, and some minor problems. I love dealing with those emails because they mean good things are happening - traffic is peaking and people are actually using iPrioritize!

Then a quasi-disaster struck at 4:45 PM EST. I was actually using iPrioritize to enter my own tasks for the coming days and then it crashed. After 10 minutes it was still down so I sent an IM to my hosting companies customer service and they replied with a generic response about how they were working on it. Then a message appeared on their site saying that one of their generators crashed and they had to power down the facility, which housed 1/4 of their sites (including SportsLizard and iPrioritize). Lucky me.

I proceeded to spend the next hour or so cursing and throwing my little wrist pad thingy all over my room. Why? On this day of all days? I don't even want to think of how many potential lifelong users were visiting the site for the first time during those two hours, only to see it down and never come back. Even worse, it pains me to think of regular users like myself who had the freaking site crash in the middle of working on their lists. Aaaaahhhhhh!

When the site finally went live again around 7 PM, the database was a mess (presumably because people were entering tasks at the moment of the crash and the commands were never executed completely) and I had to spend quite a bit of time fixing the database so everything works properly. Now, at 8 PM nearly everything on the site is up and running (a few little things are still quirky).

My question for you guys - has anyone had an experience like this and what is the best solution? My web host, iPowerweb, is widely regarded as one of the best. I understand that outages are bound to happen, but I want to be more prepared in the future. How hard is it to set up a backup server in a different location? Is it even possible considering users are constantly updating my database with new tasks every second? How would I go about "mirroring" the database in real time? I'm sure there's a way - eBay comes to mind as an example of someone who has the same problem and I'm sure has a solution. I am, however, looking for a relatively inexpensive solution :)

Wow, it felt good to get all of that bitching off my chest. Thanks for listening :) I'm off to begin apologizing profusely to my customers...