Server failures and Epic Downtime

Or what happens when your Webhost’s server has a nervous breakdown…

So we’ve been offline since Tuesday evening about 7pm or so, due to the server that the site was hosted on (Evohosting’s Firestar server) having an epic breakdown. Apparently the server was tootling along absolutely fine then just dropped like a sniper had hit the target.

I don’t think the hosts have any idea what happened yet with it, because as mentioned above, their graphs and stats show the server running fine then suddenly the entire thing just died. It took 2½ days to run an FSCK (disk check), which admittedly was the hosts fault. The reason it took so long is because their data was on one huge partition, rather than having the disk partitioned off so it would have taken a lot less time to run what is apparently the longest FSCK in IT history.

So as you can tell, all the data was restored eventually (yesterday) and took me until this morning to patch up the SQL database errors following the migration to a new server.

To be honest, even though it’s been 4 days of downtime, I can’t really be too hard on the hosts as they’ve been brilliant in the 3½ years I’ve been with them. This is the first downtime (major or otherwise) that I’ve experienced with them. So fingers crossed lessons have been learnt on their part, and it shouldn’t happen again, which is good enough for me.