When do we do away with Hardware RAID



  • So in another topic the question was raised, when would you consider doing away with Hardware RAID? Obviously there are many points that are critical to consider with this.

    Off the top of my head.

    • Your power infrastructure

    • Your Software RAID

    • Your Disk (SSD or Spinning Rust)

    • Support for software RAID

    Obviously this list isn't all inclusive. But the question is a simple one, when is hardware RAID something that is detrimental or un-usable in a business setting?



  • @DustinB3403 said:

    • Support for software RAID

    By software do you mean the Operating System? It was my understanding that the OS is the only application that knows you are using software RAID... everything else assumes it is a physical disk.





  • @MattSpeller Thanks!



  • Let's begin with a definition of software RAID so we're all clear.

    My understanding is it's a RAID which is transparent to the hardware* (seen as individual disks in BIOS) but controlled by software (OS?) to write data in such a fashion that it becomes a RAID.

    *fake RAID not withstanding, though worth mentioning for clarity. "Fake RAID" is another form of software RAID but run by another part of the computer (BIOS?).



  • @coliver Software as in MDADM Raid.

    It's installed into many linux distro's, but it is not an Operating System by its self.



  • @DustinB3403 said:

    @coliver Software as in MDADM Raid.

    It's installed into many linux distro's, but it is not an Operating System by its self.

    Right, the Operating System sets it up. But to any other application on the machine it is just another folder under "/". Or however you choose to mount it. I was thinking more along Support for Software RAID... where do you really care about that outside of the OS, does the OS support it or not?



  • With FakeRAID only BIOS has the details needed to access the array, which makes it extremely dangerous.

    Software RAID performs the same, regardless of the hardware you're running.

    As for support, if the software array were to crash, how do you recover your data. That is what I'd look for with Software RAID Support.



  • @DustinB3403 said:

    With FakeRAID only BIOS has the details needed to access the array, which makes it extremely dangerous.

    Software RAID performs the same, regardless of the hardware you're running.

    As for support, if the software array were to crash, how do you recover your data. That is what I'd look for with Software RAID Support.

    FakeRAID can generally be seen by the operating system. Hence why you have to install drivers in most Windows implementations to use it. You can actually dig into disk manager or fdisk to see them.


  • Service Provider

    @coliver said:

    By software do you mean the Operating System? It was my understanding that the OS is the only application that knows you are using software RAID... everything else assumes it is a physical disk.

    Doesn't have to be the OS, but any RAID is before the mounted block device so even if it is a third party RAID system (like ZFS would be) it gets a filesystem on top of it and presented to the rest of the system as files. No matter what you do with storage, it's all abstracted away by the time that it gets to applications.



  • @DustinB3403 said:

    With FakeRAID only BIOS has the details needed to access the array, which makes it extremely dangerous.

    If setup as RAID1, it's possible that it's not dangerous at all. Depends on if it writes any extra data or does anything differently than if it were just a single disk. My experience has been that it is completely transparent and you can turn off RAID and boot to the non-failed drive.

    Software RAID performs the same, regardless of the hardware you're running.

    ??? You lost me here

    As for support, if the software array were to crash, how do you recover your data. That is what I'd look for with Software RAID Support.

    Well, one of two things happens. Either you're running RAID1 and you boot to the working disk or for any other RAID level you recover from backups.



  • @scottalanmiller said:

    @coliver said:

    By software do you mean the Operating System? It was my understanding that the OS is the only application that knows you are using software RAID... everything else assumes it is a physical disk.

    Doesn't have to be the OS, but any RAID is before the mounted block device so even if it is a third party RAID system (like ZFS would be) it gets a filesystem on top of it and presented to the rest of the system as files. No matter what you do with storage, it's all abstracted away by the time that it gets to applications.

    Ok, that's where the software support bullet in the beginning confused me. I assumed @DustinB3403 meant application support.


  • Service Provider

    @coliver said:

    FakeRAID can generally always be seen by the operating system.


  • Service Provider

    @MattSpeller said:

    *fake RAID not withstanding, though worth mentioning for clarity. "Fake RAID" is another form of software RAID but run by another part of the computer (BIOS?).

    FakeRAID is a third party software RAID with a hardware component whose purpose is to mislead customers into thinking that they are purchasing hardware RAID.

    http://mangolassi.it/topic/6068/what-is-fakeraid


  • Service Provider

    @coliver said:

    @DustinB3403 said:

    @coliver Software as in MDADM Raid.

    It's installed into many linux distro's, but it is not an Operating System by its self.

    Right, the Operating System sets it up. But to any other application on the machine it is just another folder under "/". Or however you choose to mount it. I was thinking more along Support for Software RAID... where do you really care about that outside of the OS, does the OS support it or not?

    For example: Windows has poor support for software RAID while Linux and Solaris have excellent support. ESXi has no software RAID at all while Xen and KVM have very good software RAID.



  • @MattSpeller Software RAID is universal from one system to the next.

    IE MDADM RAID performs the same on System A or System X.

    Where as FakeRAID is often based on BIOS and whatever crap is in there.


  • Service Provider

    @DustinB3403 said:

    Where as FakeRAID is based on BIOS and whatever crap is in there.

    No, it's just faking hardware. Often it touches the BIOS, but often it does not. Not specific tie to the BIOS. Intel, for example, is famous for selling boxed, non-BIOS based FakeRAID cards.



  • @DustinB3403 said:

    @MattSpeller Software RAID is universal from one system to the next.

    OK I think you mean Software RAID has all the same capabilities regardless of what hardware it's run on.

    That's very different than "Software RAID performs the same, regardless of the hardware you're running."



  • Alright, we've had a good initial throat clearing

    When would you consider splitting off from hardware arrays?


  • Service Provider

    Hardware RAID is a full processor and memory and all RAID processing happens in dedicated hardware that does not rely on on the core CPU and system memory. It's a completely hardware abstracted, encapsulated and isolated storage subsystem that is interfaced to the computer as a single, discrete block device.

    Software RAID is a software based storage abstraction that utilized the core CPU and system memory for RAID functions and abstracts and encapsulated the block devices after they have been attached to the OS.


  • Service Provider

    This is a huge question and depends on a huge range of needs. Hardware RAID is the slower of the two and less flexible, but it is far easier to use and manage, especially in an SMB or in an enterprise using a datacenter staffed by bench pros. Hardware RAID offloads CPU and memory from the main system (up to about 10% of a core and up to about 2GB of RAM currently) and provides blind swapping which is pretty critical for SMBs and datacenter usage and has power loss prevention - typically via non-volatile caching.

    Sofware RAID requires you to handle all of the power protection externally to the chassis and pretty much precludes blind swapping requiring a knowledgeable system admin to be on hand for even the most trivial drive replacement. Software RAID leverages unused core CPU and memory for more processing power than a RAID card can realistically do. Software RAID is more flexible and has more options (RAID 7 is unique to software RAID, for example.)

    Software RAID is obviously cheaper in capex but requires more effort and experience. Hardware RAID costs more and is marginally slower but makes hardware support vastly easier.



  • So lets just imagine an infrastructure that has fault tolerant power system to run their systems.

    Everything from the Servers to the motion activated hand dryers.

    This business is looking to do a full system refresh from the ground up. You @scottalanmiller have been asked to design the system to reduce capex as much as possible for this project.

    How / what would you recommend? What techinical hurdles would the IT Team have to overcome in the event of a failed array. Single disks failures in a Software Array deployment.

    I'm just trying to think up possible use cases of an all Software RAID scenario. Hardware RAID has been tried and tested, it works, it's "simple to support" in comparison to Software RAID.

    Things of critical importance would be Array Performance.

    Array recoverability

    capex for the project


  • Service Provider

    @DustinB3403 said:

    I'm just trying to think up possible use cases of an all Software RAID scenario. Hardware RAID has been tried and tested, it works, it's "simple to support" in comparison to Software RAID.

    Not as tried and tested as software RAID. Software RAID is older (everything happens there first) and has been used in the big enterprise far more as all high end systems are and always have been software RAID (Sparc, Integrity / Itanium, SuperDome, VMS, AIX, HP-UX, IRIX, zOS, System i, etc.)

    Hardware RAID for the longest time was for the tiny end of the SMB only. It is only quite recently that hardware RAID has been considered adequate for mid-end processing loads and it is almost exclusively from the AMD64 servers pushing their way into bigger and bigger roles, not from hardware RAID earning its way up.


  • Service Provider

    @DustinB3403 said:

    How / what would you recommend? What techinical hurdles would the IT Team have to overcome in the event of a failed array.

    They would need to have systems administration resources available to do the drive swaps from the logical side rather than having only bench staff in the data center. You could never do the swap using just "remote hands", it requires active coordination with the person(s) controlling the RAID system who will deactive the drive and activate the new one.



  • So you are saying that software raid does not support hot blind swap?



  • @dafyre I don't think it can.

    There is no way that the hardware could prepare for a disk being ejected live without hardware to manage it that.



  • @DustinB3403 said:

    @dafyre I don't think it can.

    There is no way that the hardware could prepare for a disk being ejected live without hardware to manage it that.

    The idea would be that the hardware supports hot swap. As I seem to remember @scottalanmiller mentioning recently that pretty much any business class server should support hot swap at a minimum.



  • Here is the definition.

    http://mangolassi.it/topic/6816/hot-swap-vs-blind-swap/2

    Now which applies to MDADM RAID? If Cold swapping is the only way of swapping drives, then I guess it immediately excludes it from any Enterprise or even business solution.


  • Service Provider

    @dafyre said:

    So you are saying that software raid does not support hot blind swap?

    Some do, but it is super rare. I've never seen it outside of a NAS.


  • Service Provider

    @dafyre said:

    @DustinB3403 said:

    @dafyre I don't think it can.

    There is no way that the hardware could prepare for a disk being ejected live without hardware to manage it that.

    The idea would be that the hardware supports hot swap. As I seem to remember @scottalanmiller mentioning recently that pretty much any business class server should support hot swap at a minimum.

    Hot swap any business class RAID will handle. But blind swap, not so much.



Looks like your connection to MangoLassi was lost, please wait while we try to reconnect.