Why the SMB Still Needs Hardware RAID
-
Once in a while someone asks me about this: If software RAID is, or can be, so good why do we ever use hardware RAID let alone recommend it so often?
Great question, and there are some really good reasons for it, most of which are not actually technical so much as practical business reasons.
First, the "hard" reasons; those reasons that can't be avoided and if we have those circumstances it's because "it's the only way."
These reasons are that some operating systems and hypervisors do not have any software RAID options or do not have enterprise class ones and so need to rely on hardware RAID in order to have RAID at all. The obvious culprits here are VMware ESXi, Windows Server and Hyper-V. ESXi has zero software RAID and the Microsoft software RAID stack is famously very poor and should be avoided. In fact, hardware RAID has only existed for the past few decades to specifically address this shortcoming on Windows. Had it not been for the lack of good software RAID on Windows, hardware RAID would have died out in the 1990s with the end of Netware.
Now the "soft" reasons. Assuming that we have the other server operating system or hypervisor options like Linux, FreeBSD, Solaris, Xen, XenServer, KVM or similar we often still recommend hardware RAID. But why? According to most reports software RAID is generally faster and more flexible and, in all real world cases, free. Why spend nearly a grand on hardware RAID if we have an equal or even better option in software RAID?
The answer is partly technical and partly business. The technical reason is blind swap.
The business reason is based around blind swap. Blind swap is easy and, far more importantly, expected. Sure, if you are the only IT person in a shop and know software RAID inside and out and know exactly what to do every time to do your hot swap, as most enterprise UNIX shops known how to do, you are all set. But if you are not certain that you will be the only admin to ever work in a shop, are in an SMB that might need to swap IT people or use an outside service, have your server hosted in a datacenter or any other flexible circumstance then blind swap is a big deal.
In some cases, it's a matter of making things easy so that you can call your datacenter and ask for a drive replacement, no interaction needed on your part. Or it allows the server vendor to do it for you on premises, no interaction needed again. Blind swap lets a non-technical "remote hands" person handle everything. Blind swap is what someone "looking at an enterprise server" is going to expect and often will not question that it must be there, even if there is no particular reason for them to assume so. Providing it protects against mistakes and oversights. It lets the receptionist replace drives, the secretary to watch for failures, the owner to handle things off hours, the on call tech to do immediate swaps or your replacement in five years to keep things running when they don't have the expertise that you have or don't know that they need to have learned about that.
If you are in the enterprise and have excellent training and documentation and procedures, software RAID is the go to solution for those times when RAID is still needed. In the SMB, consider that hardware RAID, while pricey, might remain your best option for the foreseeable future.
-
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
-
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
Automation scripts and limits on variables. General software RAID allows for tons more possibilities. They lock down the options to make it viable to automate.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
Automation scripts and limits on variables. General software RAID allows for tons more possibilities. They lock down the options to make it viable to automate.
So they always know that slot 1 in the chassis is connected to SATA bus 1. Makes sense.
-
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
Automation scripts and limits on variables. General software RAID allows for tons more possibilities. They lock down the options to make it viable to automate.
So they always know that slot 1 in the chassis is connected to SATA bus 1. Makes sense.
And they know the intention of the drives.
-
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
-
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
-
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
RAIN I think.
-
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
How are the drives configured in a RAIN setup? as JBOD? Does the RAIN setup (aka software managing the storage) handle the distribution of data over the drives? Does it work more or less like RAID 0 in the hopes of getting the best speed/performance?
-
@Dashrender said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
How are the drives configured in a RAIN setup? as JBOD? Does the RAIN setup (aka software managing the storage) handle the distribution of data over the drives? Does it work more or less like RAID 0 in the hopes of getting the best speed/performance?
I hate the term JBOD, it means nothing. All disks are JBOD at some level and not at others. All RAID starts as JBOD.
RAIN isn't a specific approach, so you can't answer at that level. RAID is specific and doesn't handle RAIN. So like relational databases and NoSQL, we have "this popular thing" and "everything else" categories, more or less.
-
@Dashrender said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
How are the drives configured in a RAIN setup? as JBOD?
To the host system, yes.
Does the RAIN setup (aka software managing the storage) handle the distribution of data over the drives?
http://www.computerworld.com/article/2579611/data-center/rain-explained.html
And let's not forget Erasure Coding to really understand what's going on under the hood of a RAIN system.
https://www.backblaze.com/blog/reed-solomon/Does it work more or less like RAID 0 in the hopes of getting the best speed/performance?
If you really want it to, it could.
-
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Ok that makes sense.. thanks.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
-
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
-
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
-
@coliver said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
Lustre is RAIN, but is closed. Gluster was the open replacement for Lustre.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
Lustre is RAIN, but is closed. Gluster was the open replacement for Lustre.
Just a quick search showed that Lustre was GPL 2.0, not sure if that is new or not.
