Announcing the Death of RAID
-
Friends, IT Pros, countrymen, lend me your ears;
I come to bury RAID, not to praise it.
The evil that tech does lives after them;
The good is oft interred with its bones;
So let it be with RAID. The noble S.A.M.
Hath told you RAID was ambitious:
If it were so, it was a grievous fault,
And grievously hath RAID answer’d it.Few people have written as extensively on RAID as I have, likely none have written so much and for so long. But we must address the fact that no matter how attached to it we are, no matter how much we talk about it or have studied it... RAID was never meant to last forever and sometime, in the last few years, while none of us were keeping watch: RAID slipped quietly away in its sleep. There were no death throws, no loud bangs, no sudden heart attack moment. Just peacefully, at home alone with the lights off RAID left us to go on to the big technology bin in the sky.
So what exactly happen to our beloved storage technology? The world changed around it.
When RAID was new, nearly everything in IT was expensive, really expensive. RAID was built around a need to accommodate large numbers of small disks in single enclosures. These were the realities of the age and even networking was far from ubiquitous at the time. And the idea that storage could pass over a network was nascent.
Today so much is different. Drives are huge, prices have changed dramatically, we rarely work from single enclosure systems and storage networking is very common and works great.
So first the basics: scale. RAID works great for eight disks used locally in an array big enough for local applications to run. It's perfect for that. But start pushing 40TB of storage or huge spindle counts and RAID starts to have all kinds of complications. The biggest problem is that parity based RAID options (the RAID F family of R4, 5, 6 and 7) don't scale well and two RAID levels there are so pointless at scale that they have been effective retired completely (R4 and R5) and R6 only works well at smaller scales and R7 is not widely implemented and has major rebuild problems because parity just does not scale that well.
In order combat scaling issues with parity RAID, we need to turn to mirrored RAID which lacks the capacity efficiency of parity RAID (in most cases) which leaves RAID open to "attack" at a business level from other approaches.
So scale has simply worked against RAID to a point to where RAID just rarely makes sense outside of small, special case deployments or for limited use like just for local boot disks.
The move from single enclosures to clusters. Outside of storage itself, businesses have broadly moved from running single enclosure systems (one server, on its own) to clusters of servers often in a virtualization stack. This move has made it critical that our storage be able to be accessed by multiple systems at once over the network and presents an opportunity for data protection between nodes (internodal reliability rather than purely intranodal reliability.)
We can address this to some degree with networked RAID (products like DRBD) but this does not scale well (or often at all) past two nodes and presents large problems - such as a need to consider node reliability and rebuilds both individually and together and how reads and writes will be impacted by the network. RAID quickly becomes very complex to implement.
Because systems where RAID fails to offer a straightforward, elegant solution have moved from niche to the norm, RAID itself now faces challenges that appear to be more than it will be able to withstand.
Enter RAIN. RAIN is really a catch all for many approaches to dealing with the RAID deficiency problems. But common approaches are beginning to emerge that are giving rise to many advantageous RAIN implementations.
RAIN refers to a "Redundant Array of Independent Nodes" and is a play on the RAID term. The RAIN term itself is too loose to be useful. RAIN's name only implies network RAID, but that is not what is meant by its use.
For a system to be generally accepted as RAIN, rather than Network RAID, it should be both node and drive aware and provide data protection (unlike technologies like RAID 0.) This awareness allows RAIN approaches to place data on disk in such a way as to protect against the failure of any one disk and/or any one node.
The most common or popular implementations of RAIN rely on disk mirroring, much akin to RAID 10, but doing so at a more granular level such as the block, rather than at a full disk or logical disk level. Mirroring has become the standard due to the low cost of disks today combined with the common advantages of mirroring, mostly around reliability and speed.
RAIN addresses the large scale reliability issues of RAID by making failure domains much smaller (individual blocks commonly) and leveraging mirroring which scales very well. RAIN additionally addresses scalability by allowing a single storage pool to scale far past the size of a single enclosure.
RAIN addresses the networking problem by having the entire storage stack by network aware, rather than simply tunneling existing RAID implementations over a network connection. RAIN can make choices such as only reading data from the local disk rather than waiting for part of the data to be transferred over the network and can make choices such as not waiting for remote writes before continuing with write confirmation. RAIN provides more flexibility and more choices.
I will explore RAIN technology, products and the future in another article. The future is not bleak, it is very good. The post-RAID world is not a bad one and RAID will, most likely, become mostly an anecdotal piece of IT history outside of a lingering use of RAID 1 for special cases. RAID is already niche today in larger businesses and this trend is pushing more and more into the SMB space. We can't expect everything that we do in IT to stay the same and RAID has had a good, long run as the top dog in server storage. But the times they are a changing and RAID should no longer be the assumed storage answer that it has been for so long.
-
Some RAIN vendor you may or may not know:
- Open Source RAIN Options: Gluster, Lustre, CEPH, Swift
- Closed Source RAIN Software: AetherStore
- RAIN Storage Appliance: Exablox
- RAIN Used Under the Hood: Scale HC3, VMware VSAN
-
Is StarWinds vSAN considered RAIN?
-
@coliver said in Announcing the Death of RAID:
Is StarWinds vSAN considered RAIN?
We'd have to dig in under the hood. I think that they are mostly focused on network RAID, just really advanced.
-
@scottalanmiller said in Announcing the Death of RAID:
RAIN addresses the networking problem by having the entire storage stack by network aware, rather than simply tunneling existing RAID implementations over a network connection.
I think you mean be
-
What's the smallest install where you would consider RAIN instead of RAID? and if that isn't enough details, what additional details are needed before you could answer a question like that?
-
@Dashrender said in Announcing the Death of RAID:
What's the smallest install where you would consider RAIN instead of RAID? and if that isn't enough details, what additional details are needed before you could answer a question like that?
When you have more than one server
-
RAID and RAIN have overlap in the two node world. But here is a quick breakdown of which makes the most sense where.
- Single Server: RAID
- Dual Server Cluster: Network RAID or RAIN
- Three or More Server Cluster: RAIN
It's the number of nodes in a related cluster (shared storage) that makes the difference. And RAIN may encourage systems that were not clustered before to become clustered, as well.
-
RAIN can be scaled down to a single node, but you lose the "requirement" of nodal protection. So we could think of it as "RAIN 0" or a RAIN in a failed state. Examples of this include the single node Scale HC3 product and the single node Exablox implementation.
-
@scottalanmiller said in Announcing the Death of RAID:
RAIN can be scaled down to a single node, but you lose the "requirement" of nodal protection. So we could think of it as "RAIN 0" or a RAIN in a failed state. Examples of this include the single node Scale HC3 product and the single node Exablox implementation.
Right... but ideally to convert a single node RAIN instance to a two-node RAIN cluster, you only need to add a second host and appropriate storage, right?
-
@dafyre said in Announcing the Death of RAID:
@scottalanmiller said in Announcing the Death of RAID:
RAIN can be scaled down to a single node, but you lose the "requirement" of nodal protection. So we could think of it as "RAIN 0" or a RAIN in a failed state. Examples of this include the single node Scale HC3 product and the single node Exablox implementation.
Right... but ideally to convert a single node RAIN instance to a two-node RAIN cluster, you only need to add a second host and appropriate storage, right?
Depends on if you are addressing the need for a witness node. In theory, yes two node can do RAIN. But it comes with complications.
-
@scottalanmiller said in Announcing the Death of RAID:
@dafyre said in Announcing the Death of RAID:
@scottalanmiller said in Announcing the Death of RAID:
RAIN can be scaled down to a single node, but you lose the "requirement" of nodal protection. So we could think of it as "RAIN 0" or a RAIN in a failed state. Examples of this include the single node Scale HC3 product and the single node Exablox implementation.
Right... but ideally to convert a single node RAIN instance to a two-node RAIN cluster, you only need to add a second host and appropriate storage, right?
Depends on if you are addressing the need for a witness node. In theory, yes two node can do RAIN. But it comes with complications.
OK considering the witness role, are you saying it's mostly wise to not worry about RAIN until you reach 3+ nodes?
-
@Dashrender said in Announcing the Death of RAID:
@scottalanmiller said in Announcing the Death of RAID:
@dafyre said in Announcing the Death of RAID:
@scottalanmiller said in Announcing the Death of RAID:
RAIN can be scaled down to a single node, but you lose the "requirement" of nodal protection. So we could think of it as "RAIN 0" or a RAIN in a failed state. Examples of this include the single node Scale HC3 product and the single node Exablox implementation.
Right... but ideally to convert a single node RAIN instance to a two-node RAIN cluster, you only need to add a second host and appropriate storage, right?
Depends on if you are addressing the need for a witness node. In theory, yes two node can do RAIN. But it comes with complications.
OK considering the witness role, are you saying it's mostly wise to not worry about RAIN until you reach 3+ nodes?
Generally, yes. Nothing wrong with it at one node, just not very much value to it there. And at two nodes, no one is focused on making that work well, it's just niche and unimportant.
-
@scottalanmiller said in Announcing the Death of RAID:
- Closed Source RAIN Software: AetherStore
It doesn't seem like you can use AetherStore for VM storage?
-
@FATeknollogee said in Announcing the Death of RAID:
@scottalanmiller said in Announcing the Death of RAID:
- Closed Source RAIN Software: AetherStore
It doesn't seem like you can use AetherStore for VM storage?
Hrm, is that a challenge? If it can be mounted on the hypervisor, it can be used as storage. Now I want to go test, and I just don't have the time!
-
I just looked at the website use cases & it didn't mention anything about virtualization, so I assumed it couldn't be used.
-
@FATeknollogee said in Announcing the Death of RAID:
@scottalanmiller said in Announcing the Death of RAID:
- Closed Source RAIN Software: AetherStore
It doesn't seem like you can use AetherStore for VM storage?
You CAN, but you sure wouldn't.
-
@travisdh1 said in Announcing the Death of RAID:
@FATeknollogee said in Announcing the Death of RAID:
@scottalanmiller said in Announcing the Death of RAID:
- Closed Source RAIN Software: AetherStore
It doesn't seem like you can use AetherStore for VM storage?
Hrm, is that a challenge? If it can be mounted on the hypervisor, it can be used as storage. Now I want to go test, and I just don't have the time!
It's a block device, it has to work. Would just be awful.
-
@FATeknollogee said in Announcing the Death of RAID:
I just looked at the website use cases & it didn't mention anything about virtualization, so I assumed it couldn't be used.
Just not a good use case for it. It's a block device, which should tell you everything that you need to know. It can be used for any block device task - including building a SAN on top of it.
-
@scottalanmiller
A SAN built on 5400-7200 rpm spindles(probably) of varying size/cache, over a network with all your other traffic on it.
Huge performance.
