Unplanned outage 6-15 and 6-16

[patracy]

Hello everyone,

Sorry about Friday afternoon/Saturday morning. Our service provider had an issue in the data center we are hosted in. Instead of risking anything, they immediately started migrating us to another data center. I guess we learned a few things tonight. While it's probably standard protocol to do this for most forums. We're not most forums! Piping over 150gb+ of data to a new location takes a little time. Then I have to make things a little more difficult asking for some unique security settings. Well, it's now 3:25am EDT Saturday morning and we're back. Again, I'm sorry about the outage. The difference this time around is with our hosting provider. They have been very attentive to the issue and accommodating. A lot different than days of the past. (Not to mention way less frequent) Now back your regularly scheduled programming!

Thanks,
Drew

 

[gringeltaube]

Ah... at first I thought it was me!

I was in the middle of PMing someone when we had a sequence of grid power outages = no more WiFi. After that I tried everything possible to log on again but only kept getting this message "server is too busy..."


10 minutes later, after several attempts, this one...:


Got up early this morning - ready to send you an email - but then all was working normally again...


So I can only say a BIG THANK YOU to our Admin Drew - for your steady work and efforts to keep this site up and running!

 

[Tinwoodsman]

Thanks Drew!

 

[SETOYOTA]

THank you for your efforts

 

[CARNAC]

I learned my lesson (whatever I did wrong) and can I be unsuspended now? LOL

Thanks for the hard work on this.

 

[firefox]

You are one heck of a nice guy Drew!

 

[papabear]

May you live long and prosper Drew

 

[Flyingvan911]

That’s a lot of data!

 

[Floridianson]

Same here account suspended. I started to call all the neighbors to see what kind drugs they had for a panic attack.

 

[patracy]

Sorry about the "account suspended" message. It was just a means to deal with another problem we had when changing IP addresses. The old server IP and server are still "online". During that state of transition we thought it best to shut down the instance of the server, which caused the cpanel side of things to display that the server's actual account had been suspended. I'm still getting messages here and there from people not being able to get onto the site, but this is just part of how networking works. (DNS takes up to two days to update in some providers) If someone reaches out to you saying they're seeing the board is down, have them reboot to see if that helps flush their DNS. Or if they know enough about their routers, have them point their DNS server to something like google's DNS of 8.8.8.8.

 

[USAFSS-ColdWarrior]

Piping over 150gb+ of data to a new location takes a little time.

WOW !
I don't know what diameter pipe you've got to use to achieve any reasonable flow, but I'm glad we've got a troop of your caliber assigned to these duties!





And, as PB already wished unto you, Drew.....

Live long and prosper!

https://www.youtube.com/watch?v=DyiWkWcR86I

That's quite an explanation, huh?!?!?

 

[CMPPhil]

Look on the bright side from the number of people who said Thanks, shows we missed the forum. Would be far worse if it had been down for two days and no body notice.

Cheers Phil

 

[patracy]

Ugh, one new issue, with the migration to the new server, the mail lookback record is invalid. Right now mac.com, icloud.net, exc. have our emails blocked. I'm working on getting that resolved now, didn't even think about that at 3am in the morning. It might take a day or two, I just hope I get it corrected before more mail hosts block.

 

[patracy]

So I've been struggling with slowness from the server. Especially at night time when we're loaded more with users. Stupid me didn't even think to check how many CPU cores we were allocated. I have been looking at backend database and html server. So we've been running all this time on a single core. We're back to the allocated number of cores now. Server loads look to be better now too.

 

[marchplumber]

Thanks.....

Ya do more than 99.9% will ever know or understand.....

Thank you for the invisible work........

 

[USAFSS-ColdWarrior]

Our beloved I.T. Wizard, Drew, Is NOTHING like the one here......

https://www.youtube.com/watch?v=NZR64EF3OpA

Yet he DOES do magical things behind that curtain of his.

Carry on, Oh Worker Of Digital Miracles and Wonderments!!!

Carry on....

 

[papabear]

Sorry about the "account suspended" message. It was just a means to deal with another problem we had when changing IP addresses. The old server IP and server are still "online". During that state of transition we thought it best to shut down the instance of the server, which caused the cpanel side of things to display that the server's actual account had been suspended. I'm still getting messages here and there from people not being able to get onto the site, but this is just part of how networking works. (DNS takes up to two days to update in some providers) If someone reaches out to you saying they're seeing the board is down, have them reboot to see if that helps flush their DNS. Or if they know enough about their routers, have them point their DNS server to something like google's DNS of 8.8.8.8.

I tried to flush my DNS system on Independence Day (4th of July) and after the 4th glass of "flush" I noticed my vision was double normal and my speech was slurring a little. It didn't take me two days to update though, I was fine the next morning.