Xen Server 6.5 + Xen Orchestra w. HA & SAN
-
-
@ntoxicator said:
Ok, why dont you come consult for us then? Explain why HA is not needed and list the negatives and upside.
I dont get why your so anti-HA?
So we get another single server, spec'd full of drives and hope that we dont have a hardware failure
What are chances of mobo dying on Dell R730? or integrated NIC card failing, etc? I suppose low percentage rate.
Of the approximately 100 servers that I personally have supported in the past 15 years I've seen exactly one motherboard failure, and zero RAID card failures. I've seen probably 5 Power Supply failures and perhaps 20-30 drives fail. This is a pretty low number of servers, but gives us easy things to make percentages out of.
So
motherboard failures = 1% over 15 years
RAID controllers = 0%
Power Supplies = 5%
drives - I can't give a real number here because I have no idea how many actual drives were on all these servers over the years. Assuming a minimum of 4, for a total of 400 drives (this is probably low) we'd be talking about a 4-5% failure rate.So looking at those numbers (as much BS as they really are) we can already see what we make certain parts redundant, and others not. Power Supplies and Drives fail often, at around 5% so we've setup multiple Power Supply systems and drives we've created RAID to keep the systems running in case of a failure.
But Mobo's and RAID cards fail so infrequently that we don't worry about it.
-
@Dashrender said:
I've seen probably 5 Power Supply failures and perhaps 20-30 drives fail.
And those are hot swap in any enterprise class server, even entry level. So they don't result in down time.
-
@scottalanmiller said:
@Dashrender said:
I've seen probably 5 Power Supply failures and perhaps 20-30 drives fail.
And those are hot swap in any enterprise class server, even entry level. So they don't result in down time.
Until they blow out the other drives!
-
@Dashrender said:
But Mobo's and RAID cards fail so infrequently that we don't worry about it.
High end server make those redundant too. they are the least likely to go bad and teh most expensive to make redundant so that is why they are avoided. But an Integrity, Oracle M or IBM i or z will all do redundant there.
-
@scottalanmiller said:
@ntoxicator tell them that without those numbers you have to assume that the losses would be minimal because of they were significant they would know how important it was for you to have them.
Instil in them that their actions are informing you where their words are not.
This is a hard thing to tell upper management without being fully prepared to be fired.
Sadly this feeling also proves that most businesses, even ones that appear successful, are really run poorly.
-
@scottalanmiller said:
You get more uptime moving a single server to a good datacenter than you do putting HA servers on premises. The facilities matter a lot.
That's why we generally see six nines from our standard servers. Six nines!!!
The problem with this is the cost of the Colo and the high speed internet to your main location are sometimes cost prohibitive, But that too also goes to Scott's point that HA isn't needed.
-
I didn't see it mentioned, if it was ignore this. Another way to look at it is the cost of the solution would directly be related to the amount of time saved by an HA solution. So for example:
You spend x number of dollars on an HA solution to fail over in 10 seconds. That means you would have to make x number of dollars every 10 seconds for it to be worth the cost.
-
For HA you would probably want 3 nodes also. If you are working on one node doing maintenance and it's offline, and the second node goes down you're out of luck.
-
@johnhooks said:
You spend x number of dollars on an HA solution to fail over in 10 seconds. That means you would have to make x number of dollars every 10 seconds for it to be worth the cost.
I think that you missed a number....
It would cost X to not have HA.
It would cost Y to have HA.
The downtime of X is Z
The downtime of Y is W
So the cost of HA is Y - X and the time to make up with Z - W.
So if R = Z - W and S = Y - X, then the cost S has to be justified in R downtime mitigated.
-
Eh I thought I read somewhere that you said something like that. I must have just misread one of your $1000 per minute posts.
-
@johnhooks said:
Eh I thought I read somewhere that you said something like that. I must have just misread one of your $1000 per minute posts.
Maybe it was the ten minute number. HA versus secondary server is a difference of normally about ten minutes of downtime.
-
@scottalanmiller said:
@johnhooks said:
Eh I thought I read somewhere that you said something like that. I must have just misread one of your $1000 per minute posts.
Maybe it was the ten minute number. HA versus secondary server is a difference of normally about ten minutes of downtime.
This is assuming some time of replication between the running and secondary servers?
-
@Dashrender said:
@scottalanmiller said:
@johnhooks said:
Eh I thought I read somewhere that you said something like that. I must have just misread one of your $1000 per minute posts.
Maybe it was the ten minute number. HA versus secondary server is a difference of normally about ten minutes of downtime.
This is assuming some time of replication between the running and secondary servers?
Or good backups. If you have a fast backup system, you can often restore quickly, too.
-
@scottalanmiller said:
@Dashrender said:
@scottalanmiller said:
@johnhooks said:
Eh I thought I read somewhere that you said something like that. I must have just misread one of your $1000 per minute posts.
Maybe it was the ten minute number. HA versus secondary server is a difference of normally about ten minutes of downtime.
This is assuming some time of replication between the running and secondary servers?
Or good backups. If you have a fast backup system, you can often restore quickly, too.
I suppose, but if 10 mins is your goal, I don't think backups are really in your game plan unless your VMs are pretty small, and few.
-
@Dashrender said:
I suppose, but if 10 mins is your goal, I don't think backups are really in your game plan unless your VMs are pretty small, and few.
You adjust as needed. How many people have large critical workloads? Some, not many. Most can be getting production back on line as systems return to normal.
-
A pair of FC interfaces can restore nearly 2TB of backups in 10 minutes when needed.
-
@scottalanmiller said:
A pair of FC interfaces can restore nearly 2TB of backups in 10 minutes when needed.
What kind of drive system do you have behind that?
Remember, most of us here are SMBs, we don't have FC - and if you're wanting to stick your hand up and say you have FC, before you post - do you have under 300 users? if not, just sit back down because you are not SMB (no matter what IBM or Norton Says).
-
@Dashrender said:
@scottalanmiller said:
A pair of FC interfaces can restore nearly 2TB of backups in 10 minutes when needed.
What kind of drive system do you have behind that?
Remember, most of us here are SMBs, we don't have FC - and if you're wanting to stick your hand up and say you have FC, before you post - do you have under 300 users? if not, just sit back down because you are not SMB (no matter what IBM or Norton Says).
FC isn't all that expensive these days, especially if you are doing host to host as there is no switching equipment involved. All you are adding are cards. If you were going to add 10GigE cards, FC is actually less effort and more performant for a backup. You'd be looking at pretty much break even on cost.
Pushing an SSH cache on a restore machine is not cheap but not bad at all. Your backups can often be a SuperMicro type box where that cache layer can use consumer drives.
Surprisingly, I think you'd find many SMBs paying more for less already.
-
And that's an extreme case, you can get nearly those speeds with SATA!
