OS/Filesystem for file server ~ 8Tb



  • Hi all,

    I'm hoping to get some (wise!) opinions on what I should or could do here.

    I've got a 7 year old HP SAN that's nearing it's capacity. It's connected to a HP DL360 G7 running an old version of openindiana which operates as our fileserver. The SAN doesn't host VMs or databases, just home and shared directories for file storage. Now rather than try to increase the capacity of this SAN, I think it would make more sense to move our file's onto a new platform.

    I know virtualisation is a must and we run VMware here so most likely I'd do something like this

    1. get a newer HP DL360 or Dell RX730 or something similar
    2. install ESXi 6.5 onto SD card
    3. Setup a Raid 10 array using a hardware raid card.
    4. Install a VM with one vmdk for the OS and one for the storage area.

    Now I'm looking for advice on what the best OS and filesystem to use for the file server VM? Capacity would need to be in the region of 8 TB. We're almost entirely a linux operation here though we have a few windows clients. The current file server uses NFS v3 and samba for the fileshares, and the filesystem is ZFS. I'd like and have been asked, to keep the snapshot functionality that ZFS offers.

    So one option is to stick with Openindiana and ZFS. One problem here is that it's an OS I'm not so familiar with. However it's been pretty solid for us except that once the storage pool where our home directories live gets close to full then our workstations start to crawl.
    I could go with BTRFS as a filesystem to keep the snapshot ability. Am I best to use one of Opensuse or Suse enterprise server in this case?
    Is there an issue using either of these filesystems on top of hardware raid? I guess not if I'm not using their RAID functions?
    Centos and XFS with LVM? Do LVM snapshots provide similar functionally to the ZFS ones?
    Or stick with ZFS but use FreeBSD.
    Or finally just get a Synology and use that.

    I'd be grateful for opinions :-) including if it's just to say definitely don't do one of the options for some obvious reason that I'm not seeing.

    cheers!



  • CentOS 7 is the best choice for this.

    You could use other bundled systems as well, but for a basic file server. An enterprise supportable OS is the best choice.



  • @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    install ESXi 6.5 onto SD card

    Have you looked at other Hypervisors for this task? Just out of curiosity seems like KVM would be a great fit since you're already a *nix shop.



  • While ESXi is a good solution, are you taking advantage of the features it comes with at the paid levels. Or are you using the free version which is handicapped?

    There are a lot of choices that offer a lot of flexibility to manage your infrastructure outside of ESXi.



  • Are you already virtualized? I'm pretty sure the LVM snapshots would work for you with XFS but it would be something to test out to make sure. No reason not to spin up a new VM to test it with.



  • @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    I'd like and have been asked, to keep the snapshot functionality that ZFS offers.

    You get this with any Virtualization platform... If you must have it at the file system level, then I believe that you can do this with LVM or BtFRS... Having little experience with either of those, I don't know which would be better.

    Pretty much any distribution of Linux would be good for file serving. I believe the current favorites around here are either CentOS or Fedora.

    As for the Data disk, you'll need to make sure that your VMware data store is set for VMFS-5 or better for support of disk images > 2 TB.



  • We're running ESXi essentials here. One reason I'd favour sticking with VMWare is that it's what I know best. Another is I'm kind of a 1 man band here as a sysadmin but I do have a deputy who works a tester who I've trained up a little bit in ESXi in case I'm not around and something happens.
    Yes we're on ESXi 6.5 so it'd be a VMFS 5 datastore.



  • and yes it's snapshotting at the file level I'd need to replicate the daily zfs snapshots we've got now.



  • With Essentials do you get the ability to backup your VM's without an agent? Or really any other benefit?

    I thought it was Essentials Plus.

    With Essentials, isn't that just to allow for more cores and memory?



  • Yes I can use Veeam (I use the free version) with vcenter on essentials to backup VMs



  • Yeah with essentials you're just paying for the hypervisor. Why? It's wasted money at that point as you get none of the benefits.

    Hyper-V offers more than this and is free (granted its Microsoft's solution). XenServer and KVM way out offer this as well.

    0_1495205972129_chrome_2017-05-19_10-58-28.png



  • Using Essentials is just wasted money. (sorry to be blunt). You need at least Essentials Plus for it to make sense to use VmWare.

    The reason is that the software is gimped. Broken kneecaps in the middle of the street. Gimped.

    If you're at a refresh interval for your servers and NAS you could get a solution from @scale which is highly available, with backup capabilities with all of the bells and whistles you could want.

    Alternatively you could go with XenServer and 2 or 3 servers, get Xen Orchestra and have just about the same exact set of functionality.



  • Also ESXi is in no way *nix, any longer.



  • I get the centralised management, plus the agentless backup. And it's quite a bit cheaper than essentials plus as I remember. High availabilty is not a major concern for us.



  • @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    I get the centralised management, plus the agentless backup. And it's quite a bit cheaper than essentials plus as I remember. High availabilty is not a major concern for us.

    How are you getting those features as Essentials doesn't include it? Using Veeam is an outside provider. It's good software, but its not within the Hypervisor.



  • Standard availability is likely all you need, which I completely understand. Its the base that any business really does need.

    But spending for software, that you're unable to use or scale up as needed is painful.



  • @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    I get the centralised management, plus the agentless backup. And it's quite a bit cheaper than essentials plus as I remember. High availabilty is not a major concern for us.

    So why spend the money on a hypervisor when all major hypervisors are provided for free?



  • Without wanting to draw out the virtualisation platform questions anymore let me just say that the £600 or something we paid for essentials is not outrageous given that it's the platform

    1. I'm most familar with and
    2. the people in the company who would have to step in if I wasn't around are most familiar with

    I completely accept that Xenserver and it's bells and whistles come for free and I'll definitely take it into consideration when I'm implementing a solution.

    Now onto the filesystem and OS for the VM :-) !



  • @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    Without wanting to draw out the virtualisation platform questions anymore let me just say that the £600 or something we paid for essentials is not outrageous given that it's the platform

    1. I'm most familar with and
    2. the people in the company who would have to step in if I wasn't around are most familiar with

    I completely accept that Xenserver and it's bells and whistles come for free and I'll definitely take it into consideration when I'm implementing a solution.

    Now onto the filesystem and OS for the VM :-) !

    XenServer with Xen Orchestra, use CentOS for the VM.

    All linux, all day, and save that £600 for Beer-Friday!



  • £600 will keep me in beer for quite a while. I don't drink like I used to :-(

    So with Centos as the VM, would you use LVM snapshotting? ZFS on linux and BTRFS are not quite production ready seems to be a common (possibly wrong) consensus. There would be complaints if we lost the file system snapshotting that ZFS allows.



  • @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    £600 will keep me in beer for quite a while. I don't drink like I used to :-(

    So with Centos as the VM, would you use LVM snapshotting? ZFS on linux and BTRFS are not quite production ready seems to be a common (possibly wrong) consensus. There would be complaints if we lost the file system snapshotting that ZFS allows.

    LVM will do snapshots at the filesystem level yes.



  • @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    £600 will keep me in beer for quite a while. I don't drink like I used to :-(

    So with Centos as the VM, would you use LVM snapshotting? ZFS on linux and BTRFS are not quite production ready seems to be a common (possibly wrong) consensus. There would be complaints if we lost the file system snapshotting that ZFS allows.

    Use CentOS as the server, use Veeam for your backups. You get a bit more functionality with it, especially if you aren't familiar with the process to restore files using the backed in snapshot functionality.



  • Yes I'd keep using Veeam for backups. I'd class the snapshotting and backups as different functions really. I'd be hoping to use the snapshots for restoring single files or directories, the backups more for restoring the file server in a disaster.


  • Service Provider

    @DustinB3403 said in OS/Filesystem for file server ~ 8Tb:

    With Essentials do you get the ability to backup your VM's without an agent? Or really any other benefit?

    I thought it was Essentials Plus.

    With Essentials, isn't that just to allow for more cores and memory?

    Essentials Plus gives you HA and support. Essentials is no support, and all you get are backups.


  • Service Provider

    @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    and yes it's snapshotting at the file level I'd need to replicate the daily zfs snapshots we've got now.

    Why replicate here rather than with Veeam? Veeam does this well. Are you using ZFS' internal syncing protocol?


  • Service Provider

    @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    Yes I'd keep using Veeam for backups. I'd class the snapshotting and backups as different functions really. I'd be hoping to use the snapshots for restoring single files or directories, the backups more for restoring the file server in a disaster.

    That can make sense. Although I'd consider re-evaluating your Veeam usage instead. Veeam will do this in a single place, more powerfully than doing snaps inside the VM will.

    This is really Veeam's strong suite.


  • Service Provider

    @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    £600 will keep me in beer for quite a while. I don't drink like I used to :-(

    So with Centos as the VM, would you use LVM snapshotting? ZFS on linux and BTRFS are not quite production ready seems to be a common (possibly wrong) consensus. There would be complaints if we lost the file system snapshotting that ZFS allows.

    They are both production ready, but the big question is always... why look for niche solutions when the mature, normal ones are SO good at it? LVM and XFS are the mature choice for this for two decades.

    ZFS has always been considered silly on Linux because snapshotting was always there, since the 1990s. ZFS doesn't offer new functionality there. It was new on Solaris, either. Solaris had snaps in the 1990s as well. Long before they added ZFS.


  • Service Provider

    @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    Without wanting to draw out the virtualisation platform questions anymore let me just say that the £600 or something we paid for essentials is not outrageous given that it's the platform

    That's a very unhealthy way of looking at it. What you should ask is "what did you get for that money?" And the answer is "screwed."

    In absolute terms, £600 is nothing for any viable business to spend. So it sounds reasonable when looked at in that way and if it was getting you valuable stuff, then absolutely without question, just spend it.

    But we have to work in relative terms. For £600 what did you actually get? You got a very limited system without the features you would get for free from any other system, that's it. You are literally getting a negative value out of spending £600.

    To put it another way, if ESXi Essentials was free instead of £600... we would still be having this conversation about choosing the only platform out there that doesn't provide a certain base set of features. The £600 is a red herring, it's that ESXi isn't up to snuff here, it's not a viable option. ESXi isn't really a consideration until you are talking about Essentials Plus for much, much more money and mostly that is because you are getting support. It's not that ESXi is bad, it's awesome. But you are getting screwed if you are trying to use it below the Essentials Plus tier.


  • Service Provider

    @Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

    We're almost entirely a linux operation here though we have a few windows clients.

    Just have to mention that this fact makes ESXi a weird choice as you need a range of skills instead of being able to focus on the ones that you have. If you have Linux skills, you are ready for KVM or Xen :)


  • Service Provider

    CentOS, Fedora, openSuse and Ubuntu are your choices here and are all fine. If you want to use ZFS, honestly I'd probably use Ubuntu.

    If I was building from scratch what you want to do here, I'd likely use openSuse with XFS. All of the choices are fine.


Log in to reply
 

Looks like your connection to MangoLassi was lost, please wait while we try to reconnect.