downtime noticed

Undertoad · 03-26-2017, 09:39 AM

The Cellar was offline from apx 3:30am to apx 10:30am EST due to a system crash. Root cause not identified, but it happened at the moment when system logs are rotated and processes restarted, and there happened to be an unusually large log file in the mix (no malice involved, just a log I hadn't rotated before).

I've taken this opportunity to upgrade the system as our host began offering double your system memory for free. (This actually may eliminate the source of the problem.)

Undertoad · 03-26-2017, 09:46 AM

This means Cellar does not qualify this year for the "five nines" category of "highly available", and only qualifies as "three nines". (It's only available 99.9% of the time, not 99.999% of the time.) In recent years, the regular failure of platforms like Amazon AWS has meant that even huge sites have had a hard time managing to reach that goal. So we're doing OK.

No single-point-of-failure solution is ever really highly available. To qualify we would need failover with at least one other server in a load-balancing platform.

sexobon · 03-26-2017, 10:11 AM

Quote:

Originally Posted by Undertoad

And, it may have been the first time there has ever been a present quorum of moderators. We got a lot done.

You're all banned.

Quote:

Originally Posted by Undertoad

The Cellar was offline from apx 3:30am to apx 10:30am EST due to an system crash. ...

We don't need the cover story, we're just glad you changed your mind.

monster · 03-26-2017, 10:31 AM

I blame xoB -when it can back up the list of new posts was all him, at that time.....

xoxoxoBruce · 03-26-2017, 11:10 AM

I'll take the blame, I have a broad's shoulders.
I'll admit I was late with my shtick yesterday.

Gravdigr · 03-26-2017, 01:19 PM

Quote:

Originally Posted by Undertoad

The Cellar was offline from apx 3:30am to apx 10:30am EST

Looks like I got here just in time. Late.

03-26-2017, 09:39 AM	#1
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	downtime noticed The Cellar was offline from apx 3:30am to apx 10:30am EST due to a system crash. Root cause not identified, but it happened at the moment when system logs are rotated and processes restarted, and there happened to be an unusually large log file in the mix (no malice involved, just a log I hadn't rotated before). I've taken this opportunity to upgrade the system as our host began offering double your system memory for free. (This actually may eliminate the source of the problem.)

03-26-2017, 10:31 AM	#4
monster I hear them call the tide Join Date: Dec 2005 Location: Perpetual Chaos Posts: 30,852	I blame xoB -when it can back up the list of new posts was all him, at that time..... __________________ The most difficult thing is the decision to act, the rest is merely tenacity Amelia Earhart

03-26-2017, 11:10 AM	#5
xoxoxoBruce The future is unwritten Join Date: Oct 2002 Posts: 71,105	I'll take the blame, I have a broad's shoulders. I'll admit I was late with my shtick yesterday. __________________ The descent of man ~ Nixon, Friedman, Reagan, Trump.

03-26-2017, 09:46 AM	#2
Undertoad Radical Centrist Join Date: Jan 2001 Location: Cottage of Prussia Posts: 31,423	This means Cellar does not qualify this year for the "five nines" category of "highly available", and only qualifies as "three nines". (It's only available 99.9% of the time, not 99.999% of the time.) In recent years, the regular failure of platforms like Amazon AWS has meant that even huge sites have had a hard time managing to reach that goal. So we're doing OK. No single-point-of-failure solution is ever really highly available. To qualify we would need failover with at least one other server in a load-balancing platform.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)