CloudatCost and Pertino

Danp

Just received the following via email:

 Cloud At Cost System Message:

04/30/2015 05:06 PM

We are having an issue accessing some of our network subnets due to a local car crash that caused fiber to be cut.




Network Update: Our fiber technicians have confirmed that the target for full restoration is 8 PM EST today. 

We will provide updates as we get them.

IRJ

@Danp said:

Just received the following via email:

 Cloud At Cost System Message:

04/30/2015 05:06 PM

We are having an issue accessing some of our network subnets due to a local car crash that caused fiber to be cut.




Network Update: Our fiber technicians have confirmed that the target for full restoration is 8 PM EST today.

We will provide updates as we get them.

There are alot of "uncontrollable" disasters that happen to C@C

Dashrender

Cloud At Cost System Message:

04/30/2015 05:06 PM
We are having an issue accessing some of our network subnets due to a local car crash that caused fiber to be cut.

Network Update: Our fiber technicians have confirmed that the target for full restoration is 8 PM EST today.
We will provide updates as we get them.

To Unsubscribe : If you don't want to receive these messages please change your email preferences here https://members.cloudatcost.com on your profile page.

scottalanmiller

I just got the email too. That's a pretty long time to think up "car crash". Where was the email when the outage started?

Danp

@scottalanmiller said:

Where was the email when the outage started?

Maybe they host their own email server. That would certainly explain the long delay.

Car Crash? So there is only one fiber connection.. No failover. I'm sure different providers are taking different routes so it's unlikely to take out multiples. And most fiber is below ground (at least in the US) unless it absolutely has to be run on the poles over a bridge or something.

@Danp said:

@scottalanmiller said:

Where was the email when the outage started?

Maybe they host their own email server. That would certainly explain the long delay.

They do. and it doesn't even have a vaild SSL Cert. It's self-signed. https://mail.cloudatcost.com/mail/

IRJ

@thecreativeone91 said:

@Danp said:

@scottalanmiller said:

Where was the email when the outage started?

Maybe they host their own email server. That would certainly explain the long delay.

They do. and it doesn't even have a vaild SSL Cert. It's self-signed. https://mail.cloudatcost.com/mail/

I wonder how many employees to accounts they have on staff?

scottalanmiller

@Danp said:

@scottalanmiller said:

Where was the email when the outage started?

Maybe they host their own email server. That would certainly explain the long delay.

Except the cloud platform is still down.

@IRJ said:

I wonder how many employees to accounts they have on staff?

2-5 is my guess.

scottalanmiller

@thecreativeone91 said:

@IRJ said:

I wonder how many employees to accounts they have on staff?

2-5 is my guess.

5 employees per account?

IRJ

@thecreativeone91 said:

@IRJ said:

I wonder how many employees to accounts they have on staff?

2-5 is my guess.

That would be cutting some serious corners, but we already know that they do cut corners with redundancy.

IRJ

@scottalanmiller said:

@thecreativeone91 said:

@IRJ said:

I wonder how many employees to accounts they have on staff?

2-5 is my guess.

5 employees per account?

I think he meant total

scottalanmiller

OH! LOL. That's very different. But makes way more sense.

gjacobse

Curious,..I know most 'cable' is on the pole... but isn't most fiber in ground?

@g.jacobse said:

Curious,..I know most 'cable' is on the pole... but isn't most fiber in ground?

Usually. It can be on a pole but it's rather rare.

Reid Cooper

This is a really significant outage. How many hours has it been?

@Reid-Cooper said:

This is a really significant outage. How many hours has it been?

About 12 hrs.

I filed a paypal dispute for my two Dev 1 and the BigDog 1 instance.

Now gotta figure out how to get some data off of one of them.

They said they expect to be up by 8PM EST.

Somehow you really question how the heck Cloud@cost is running. They are supposedly a fiber provider. They have stated they have two different providers of fiber backbone and they have a wireless backhaul yet, a small outage takes most of them out. Is this running out of someones house? Really. Ever heard of BGP C@C?

Rogers does have some outages but still they are suppose to plan for this.
https://twitter.com/RogersHelps/with_replies