Home Lab Off-Host Backup Device



  • So for my home lab, I need to build a backup device for my Hypervisor. Something that I can run NAUBackup to, and have my full system backups stored on that is a separate "server" from XenServer.

    Using a purely hypothetical setup at the moment as I don't have the hardware.

    Consumer grade desktop, with 4 SR-Drives (Spinning Rust Drives) that I want in RAID 10. I have no hardware raid controllers.

    I'm considering using CentOS but need clarification on how I should configure the RAID. The board supports FakeRAID, the OS has several options. One being ZFS.

    The goal is to not only backup my Hypervisor, but to learn more about Software RAID solutions.



  • @scottalanmiller may be able to tell you more about the Linux software raid setup. I've done it a time or two, but it has been a long time. It seemed to work really well in the situations where it was needed.



  • As far as I know, no OS can software RAID the OS drive, but I could be completely wrong on that.



  • Well the obvious answer is to install XenServer on the machine, not CentOS, since installing CentOS directly would violate the "always virtualize" rule. Of course this is a lab so production rules (any rules in fact) do not not necessarily apply. But it is worth considering that if this was a business situation, which a good lab will generally mimic for maximum value, we would be questioning why you were considering installing a physical OS install rather than a virtual one.



  • @Dashrender said:

    As far as I know, no OS can software RAID the OS drive, but I could be completely wrong on that.

    Actually, they all can 🙂 Windows struggles hard with this. Other OSes don't have the slightest issue.



  • @Dashrender said:

    As far as I know, no OS can software RAID the OS drive, but I could be completely wrong on that.

    Though, I'm almost thinking this can't be right, how does big iron handle boot drives? Scott's always telling us that big iron only uses software RAID.



  • @Dashrender said:

    @Dashrender said:

    As far as I know, no OS can software RAID the OS drive, but I could be completely wrong on that.

    Though, I'm almost thinking this can't be right, how does big iron handle boot drives? Scott's always telling us that big iron only uses software RAID.

    Well, to be fair, Big Iron doesn't have boot drives 🙂 They boot from firmware (a la OpenBoot.)



  • @scottalanmiller said:

    @Dashrender said:

    @Dashrender said:

    As far as I know, no OS can software RAID the OS drive, but I could be completely wrong on that.

    Though, I'm almost thinking this can't be right, how does big iron handle boot drives? Scott's always telling us that big iron only uses software RAID.

    Well, to be fair, Big Iron doesn't have boot drives 🙂 They boot from firmware (a la OpenBoot.)

    Aww.. OK that makes sense.



  • This post is deleted!


  • @scottalanmiller said:

    Well the obvious answer is to install XenServer on the machine, not CentOS, since installing CentOS directly would violate the "always virtualize" rule. Of course this is a lab so production rules (any rules in fact) do not not necessarily apply. But it is worth considering that if this was a business situation, which a good lab will generally mimic for maximum value, we would be questioning why you were considering installing a physical OS install rather than a virtual one.

    So XenServer can setup a RAID set from the drives?



  • CentOS and XenServer (which leverages CentOS to make the RAID portion of this discussion easy) only has one integrated OS level software RAID option, MD. MD is the only RAID option that would generally be considered in a production Linux system today, especially with CentOS and/or XenServer. It's decades old, battle tested, used in most NAS and SAN products on the market, used in countless enterprise servers, well known, robust, etc.



  • So virtualize the storage server and build a CentOS VM off of that. Not that it's odd, but seems odd just saying it.

    What would happen in the event that the XenServer OS goes tits up? The only backup I'd have, would be if I cloned the USB Drive prior to "production".

    How is the VM Array protected in this case?



  • ZFS is a third party filesystem to Linux. Adding it for RAID functionality would be an extremely special case and really would only come up in a scenario where you absolutely had to have RAIDZ3 specifically, the only RAID option unique to ZFS (RAID 7 aka RAID 5.3.) Using a third party software RAID system is just as weird here as anywhere else. The OS has its own RAID, stick with that.



  • @DustinB3403 said:

    So virtualize the storage server and build a CentOS VM off of that. Not that it's odd, but seems odd just saying it.

    It sure does, but ask yourself, dig deep... WHY does it feel odd? What makes it special? We virtualize every server and when we say that the logic makes sense. When it is a simple file server like this, what about this would make it a special case? Pretty much we all get that same gut reaction that fileservers should be special cases, and yet they make perfect sense to virtualize depending on the direction from which we come to the discussion.



  • @DustinB3403 said:

    So virtualize the storage server and build a CentOS VM off of that. Not that it's odd, but seems odd just saying it.

    What would happen in the event that the XenServer OS goes tits up? The only backup I'd have, would be if I cloned the USB Drive prior to "production".

    How is the VM Array protected in this case?

    This is your backup repository. The same could be said if you were using a Drobo - what happens if the Drobo mobo dies?



  • @DustinB3403 said:

    So virtualize the storage server and build a CentOS VM off of that. Not that it's odd, but seems odd just saying it.

    What would happen in the event that the XenServer OS goes tits up? The only backup I'd have, would be if I cloned the USB Drive prior to "production".

    How is the VM Array protected in this case?

    This is your backup repository. The same could be said if you were using a Drobo - what happens if the Drobo mobo dies?
    There was a discussion recently, can't recall here or SW, where someone asked if the boot setup died, how do I recover my software based RAID. Assuming the drives are still good, when you reinstall the OS that created the RAID, you should be able to import the RAID back into the system with no data loss.



  • @DustinB3403 said:

    What would happen in the event that the XenServer OS goes tits up?

    Bad question to ask, sort of. This is a good thing to consider, of course, BUT the question is leading. It leads us to mentally jump to "XS introduces risk!!", but we know that it doesn't. One of the key reasons that we "always virtualize" is because it explicitly reduces risk. So instead of asking the question based on "what if this unlikely scenario plays out", instead ask "is reliability important?" If the answer is yes, then XenServer is a far better choice than not XenServer, right?

    Step back to the goal level. Reliability is the goal. So ask which approach is more reliable. Don't focus on an unlikely failure mode.

    That's like saying you need to drive two cars across the country. And someone says "well, you could fly." Don't say "what if the plane crashed", because that leads you to emotionally panic. When, if you look at the math, you know that two people on one plane is still far safer than driving separately over that distance. Looks at the resulting safety to avoid an unnecessary emotional reaction to easier perceived risks.

    On this chart: http://mangolassi.it/topic/6294/cognitive-biases-cause-bad-decisions

    What the XenServer failing risk is is called "salience #16"



  • @Dashrender said:

    This is your backup repository. The same could be said if you were using a Drobo - what happens if the Drobo mobo dies?
    There was a discussion recently, can't recall here or SW, where someone asked if the boot setup died, how do I recover my software based RAID. Assuming the drives are still good, when you reinstall the OS that created the RAID, you should be able to import the RAID back into the system with no data loss.

    I prefer to clone my running state USB.. but that's just my thinking.



  • @DustinB3403 said:

    How is the VM Array protected in this case?

    RAID 10 via the same MD software RAID system you would use with any Linux, Xen or KVM platform. As determined by your stated requirements in the OP.



  • @Dashrender said:

    what happens if the Drobo mobo dies?

    I really hope that when you open one of those up that their hardware revision is called Bobo and the board has a big label that says Drobo Bobo Mobo on it.



  • There actually are cases where you would not virtualize a file server, but not at this scale or type. Typically it would be massive systems where you are doing special case stuff like massive physical scale, Gluster, CEPH, etc.



  • @DustinB3403 said:

    @Dashrender said:

    This is your backup repository. The same could be said if you were using a Drobo - what happens if the Drobo mobo dies?
    There was a discussion recently, can't recall here or SW, where someone asked if the boot setup died, how do I recover my software based RAID. Assuming the drives are still good, when you reinstall the OS that created the RAID, you should be able to import the RAID back into the system with no data loss.

    I prefer to clone my running state USB.. but that's just my thinking.

    I definitely agree - saves you from having to start over, not that it's that big a deal if you do.



  • So the recommended solution is setup another XenServer host with this box, and build a VM that has full access to the RAID 10.

    I'm curious, how would I build the array on a VM, since I don't have a hardware RAID controller to do it at the Host level.



  • You would configure the RAID 10 using XenServer's gui at the Hypervisor level...

    Then just install the File Server VM as a regular single disk vm.



  • @DustinB3403 said:

    So the recommended solution is setup another XenServer host with this box, and build a VM that has full access to the RAID 10.

    I'm curious, how would I build the array on a VM, since I don't have a hardware RAID controller to do it at the Host level.

    No, do nothing of the sort. This is what I call "being weird." I can't define it but you can feel it. In SE it's called "smelling bad."

    Why are you doing anything abnormal here? Install XenServer on a RAID 10 array, make a VM. Done. Never give VMs access to make RAID arrays. That's fundamentally not how virtualization is meant to work.



  • @dafyre said:

    You would configure the RAID 10 using XenServer's gui at the Hypervisor level...

    Then just install the File Server VM as a regular single disk vm.

    Exactly. Same way you would do this in any other instance. The problem comes, I assume, but feeling like this is a special case ... but it is not. It is just a normal VM on a normal hypervisor doing a normal task. Treat it like an normal VM.



  • OK than how do you build the Software Array at the host level using XenServer? Any guides I can follow?



  • @DustinB3403 said:

    OK than how do you build the Software Array at the host level using XenServer? Any guides I can follow?

    Identical to normal CentOS, as you will be doing it from the CentOS CLI.

    Here are some guides:

    https://blog.linuxnet.ch/xenserver-software-raid/
    https://major.io/2012/01/16/xenserver-6-storage-repository-on-software-raid/



  • @scottalanmiller said:

    @dafyre said:

    You would configure the RAID 10 using XenServer's gui at the Hypervisor level...

    Then just install the File Server VM as a regular single disk vm.

    Exactly. Same way you would do this in any other instance. The problem comes, I assume, but feeling like this is a special case ... but it is not. It is just a normal VM on a normal hypervisor doing a normal task. Treat it like an normal VM.

    Uh, I don't know about Dustin, But I've never used a hypervisor to make the array, I've always used hardware to make the array, and the hypervisor sees what the hardware presents as a single large disk.



  • @Dashrender said:

    Uh, I don't know about Dustin, But I've never used a hypervisor to make the array, I've always used hardware to make the array, and the hypervisor sees what the hardware presents as a single large disk.

    Maybe because you are coming from the VMware world where that is the only option? The lack of enterprise software RAID is unique to VMware.