Hi all,

I'm hoping to get some (wise!) opinions on what I should or could do here.

I've got a 7 year old HP SAN that's nearing it's capacity. It's connected to a HP DL360 G7 running an old version of openindiana which operates as our fileserver. The SAN doesn't host VMs or databases, just home and shared directories for file storage. Now rather than try to increase the capacity of this SAN, I think it would make more sense to move our file's onto a new platform.

I know virtualisation is a must and we run VMware here so most likely I'd do something like this

1. get a newer HP DL360 or Dell RX730 or something similar
2. install ESXi 6.5 onto SD card
3. Setup a Raid 10 array using a hardware raid card.
4. Install a VM with one vmdk for the OS and one for the storage area.

Now I'm looking for advice on what the best OS and filesystem to use for the file server VM? Capacity would need to be in the region of 8 TB. We're almost entirely a linux operation here though we have a few windows clients. The current file server uses NFS v3 and samba for the fileshares, and the filesystem is ZFS. I'd like and have been asked, to keep the snapshot functionality that ZFS offers.

So one option is to stick with Openindiana and ZFS. One problem here is that it's an OS I'm not so familiar with. However it's been pretty solid for us except that once the storage pool where our home directories live gets close to full then our workstations start to crawl.
I could go with BTRFS as a filesystem to keep the snapshot ability. Am I best to use one of Opensuse or Suse enterprise server in this case?
Is there an issue using either of these filesystems on top of hardware raid? I guess not if I'm not using their RAID functions?
Centos and XFS with LVM? Do LVM snapshots provide similar functionally to the ZFS ones?
Or stick with ZFS but use FreeBSD.
Or finally just get a Synology and use that.

I'd be grateful for opinions including if it's just to say definitely don't do one of the options for some obvious reason that I'm not seeing.

cheers!

Thanks!
Long time listener - first time caller!

I'm sysadmin for a small software company. It's a great job but covers so many areas that sometimes I feel I'm stretched a little thin keeping up with everything. I do tend to visit here quite a lot. I almost always find interesting and new IT topics to read...

I've got the Huawei P20 Pro.
I've had it for 9 months or so. Previous phones were a Samsung Galaxy S7 Edge which my 2 year old son threw down a stone stairs and a OnePlus 2 which he threw into the toilet. I can honestly say the Huawei phone is the best I've had by distance. The camera setup was a step above the other falgships at the time and I believe the P30 continues that trend. When my son finds a way to destroy this one I'll definitely be looking at Huawei again, if they're not completely crippled by the Google stuff.

In the interests of "closure", the plethora of libvirt updates which came down the line yesterday from fedora seem to have fixed the problem. I can now create virtual networks again. Yay!

Congrats Scott. Is there any extra benefit to you if I buy from Packt Publishing rather than Amazon, or vice versa?

Congrats! and Thanks!

We're running ESXi essentials here. One reason I'd favour sticking with VMWare is that it's what I know best. Another is I'm kind of a 1 man band here as a sysadmin but I do have a deputy who works a tester who I've trained up a little bit in ESXi in case I'm not around and something happens.
Yes we're on ESXi 6.5 so it'd be a VMFS 5 datastore.

@scottalanmiller Sorry, my choice of terminology didn't make things very clear there. By replicating I mean replicating the functionality rather than the snapshots themselves. At the moment if I developer deletes a file by mistake they can go to the .zfs folder in the root of their home and restore the file from one of the daily snapshots. I think if they were no longer able to do the equivalent of this, that I might get some resistance.

@scottalanmiller said in OS/Filesystem for file server ~ 8Tb:

@Doyler3000 said in OS/Filesystem for file server ~ 8Tb:

We're almost entirely a linux operation here though we have a few windows clients.

Just have to mention that this fact makes ESXi a weird choice as you need a range of skills instead of being able to focus on the ones that you have. If you have Linux skills, you are ready for KVM or Xen

In my (shaky) defence I had stronger VMWare skills than Linux skills at the time

Thanks all, I very much appreciate lots of great advice.
I've already got a XenServer installed and I'm going to take a look at Xen Orchestra now. As regards OS/Filesystems OpenSuse (or Centos) with XFS and LVM will get some testing in the near future.

I've moved most of our important stuff onto Proxmox over the last 2 to 3 years. It's been great.
I've been using the PBS as well for the backups - it's so much better than manually doing full backups of VMs (which was the case with KVM and it's tools).

So thumbs up for Proxmox from me.

We moved from LastPass to 1Password. We liked the flow of it better.
As a company they've been very attentive and responsive even after they got our money which isn't always the case!
The subscription we got includes free home accounts as well that link to the business ones.

Congrats Scott. Is there any extra benefit to you if I buy from Packt Publishing rather than Amazon, or vice versa?

@DustinB3403
I don't doubt it.
There wasn't really any added complexity though. PVE was installed with all the defaults.

@JaredBusch
Yes - I've done that a few times. I'm running with the default LVM.
I think it might be a little less picky about CPU options (during a migration) than raw KVM too. Though I definitely don't have enough data to say that with lots of confidence.

That's true but being able to schedule backups easily from the console (even if they are full backups) is a benefit over raw KVM which was the question.

Actually another benefit is being able to live migrate between cluster hosts (without shared storage) from the console. I could live migrate stuff from raw KVM but only from the command line on the host.

@JaredBusch
The reason I'm in the process of moving over to Proxmox is the ease of setting up backups. It's very straightforward and if a backup doesn't work then it doesn't require someone who understands scripting to work out why.

Since Proxmox backup server was introduced a few months ago, PVE has the ability to do incremental backups too.

Apart from that there's not much extra functionality.

Nice!
I'm on day 9 :man_getting_massage_medium-light_skin_tone:

Well since I use this forum a lot and I don't want to be seen as 'weird' I'll try to explain where I was coming from.
Yes I've been using vanilla KVM for a few years and I've always used qcow2. There was some talk about qcow2 earlier in the conversation with stacksofplates being quite positive about their advantages and then SAM saying that he uses qcow2 on lvm-thin mostly and it was the default for Proxmox.
So I thought - those guys generally know what they are talking about, let me see if I can make a qcow2 on lvm-thin.
But proxmox wouldn't let me until I created some directory storage on the thinpool.
Ok great that works - but that seems a little strange - let me ask the guys and gals on the forum what they think.
And here we are.

As regards moving to an appliance - I'm comfortable with KVM but I run our VM infrastructure mostly on my own. On occasions where I'm on holiday or otherwise unavailable, it would be helpful if at least a couple of others could administer the system (particularly backups).

@scottalanmiller said in Reconsidering ProxMox:

@Doyler3000 said in Reconsidering ProxMox:

The method of creating a volume group on the thinpool and creating the qcow2 files in that works for me. Just wondered if anyone had thoughts on whether that's the right thing to do.

Nothing wrong with that at a technical level, but makes no sense to try to work around ProxMox' mechanisms if using ProxMox.

So I'm wondering what I've missed. You use qcow2 on lvm-thin but I don't seem to have that option unless I create directory storage on top of the lvm-thin volume.
I'll keep playing around.