To ZFS or not to ZFS... that is my question.
-
So, I am contemplating a new NAS system, and I am debating what type of file system I would use for it. As I read and read about NAS storage, and specifically RAID setups, I recognize that RAID != backups... I have read about some of the issues for long-term storage on NAS systems, including issues like bitrot, etc. So you've got to do your own backups...
Then I read about ZFS. It seems to allow you to setup RAID so that it could be used as a backup for critical data. That's because it is both a volume manager and file system and has actual knowledge of the drives. Apparently, you get SAN like reliability/redundancy. BUT I understand it is insanely slow...
So I guess I'm wondering what are the use cases for ZFS? Does ZFS provide everything it promises? End-to-end date integrity? Zero bitrot? Is this the ideal solution for storing high-value data that you want to protect? But not the solution for serving of media, etc?
Curious what @scottalanmiller has to say... I hear so much about the SAM-SD, but most implementations don't use ZFS (I assume for speed issues).
-
@markl said in To ZFS or not to ZFS... that is my question.:
Then I read about ZFS. It seems to allow you to setup RAID so that it could be used as a backup for critical data.
ZFS does nothing special. And certainly is just normal RAID. And RAID is never a backup. So no.
Bottom line, ZFS changes nothing. It's a good product, but there is nothing special or groundbreaking.
-
@scottalanmiller So the combined volume manager/file system has no advantages?
What is your filesystem of choice for NAS?
-
@markl said in To ZFS or not to ZFS... that is my question.:
That's because it is both a volume manager and file system and has actual knowledge of the drives. Apparently, you get SAN like reliability/redundancy.
SAN, on average, are incredibly non-reliable. The high reliability of SAN is a myth as a starting point. Redundancy is SANs is actually what often causes the lack of reliability. Redundancy for its own sake is a bad thing, only redundancy that creates reliability is good.
That ZFS acts as both volume manager and file system has some minor advantages, but they are trivial. That it acts as the software RAID, LVM and FS gives it the ability to do some CRC on files which has a trivial advantage that almost never matters in the real world. It does have an advantage. But you MUST understand that all of these things are "background noise" in the world of reliability. Is it a positive? Yes. Will you ever notice? Nope.
Think about the improvement that you get in fuel efficiency by washing your car. Do you get some? yes, absolutely. Will no notice? no. Which car you choose, how much load you carry and how you drive are the real factors. Yes, a clean car is more efficient than a dirty one, but it's a really, really silly thing to worry about when picking which car to buy.
-
@markl said in To ZFS or not to ZFS... that is my question.:
So I guess I'm wondering what are the use cases for ZFS?
Like all filesystems, you choose your platform for other reasons, then choose the filesystem on the platform of choice. Don't let filesystems be the "lead" in your decision making. They are not that important of a factor. So you choose ZFS "when you are on a platform on which it makes sense." Normally for a NAS this means that you choose FreeBSD as your platform and you wanted the features of ZFS over the features of UFS (UFS is slightly faster for traditional workloads). You can use ZFS on Ubuntu, but as Ubuntu is not the best server platform, especially for storage, you normally would not choose it in the Linux realm and would therefore not be using ZFS.
ZFS is great, but "great" in filesystem terms doesn't mean that you switch operating systems or fundamental approaches in order to use it.
And remember that is nearly all cases in the SMB market, you would still implement your ZFS on FreeBSD both as a VM and on top of hardware RAID... so all of those special "advantages" of ZFS don't get used in most applications anyway.
-
@markl said in To ZFS or not to ZFS... that is my question.:
Does ZFS provide everything it promises? End-to-end date integrity? Zero bitrot? Is this the ideal solution for storing high-value data that you want to protect? But not the solution for serving of media, etc?
Yes, it can, if you implement everything end to end as ZFS needs. But this requires a lot of compromises and begs basic questions like...
- Does bit rot really concern you? Why?
- Is end to end data integrity a problem that you are facing? Why?
Almost no enterprise systems have these features for a reason. They are not things that most businesses need to worry about. There are big items of concern, these are minor ones.
-
@markl said in To ZFS or not to ZFS... that is my question.:
@scottalanmiller So the combined volume manager/file system has no advantages?
Nothing of significance for normal use cases.
-
@scottalanmiller said in To ZFS or not to ZFS... that is my question.:
So you choose ZFS "when you are on a platform on which it makes sense." Normally for a NAS this means that you choose FreeBSD as your platform and you wanted the features of ZFS over the features of UFS (UFS is slightly faster for traditional workloads)
Awesome reply. Seriously, I am loving this forum and appreciate the expertise.
So here's my followup question: I have hear this statement, "Redundancy for its own sake is a bad thing, only redundancy that creates reliability is good" a lot in RAID forums... so what do you feel actually generates the best reliability when discussing NASes for SMBs? How should I be thinking about it.
-
@markl said in To ZFS or not to ZFS... that is my question.:
What is your filesystem of choice for NAS?
I start be choosing the function and needs of the NAS, then getting the operating system that is more appropriate for that need. Then choosing a filesystem that best fits the needs that is part of that OS.
My go to operating systems for NAS are Fedora and openSuse Linux, Windows and FreeBSD. Others are fine too, but those 3 and a half (two Linux that are very similar) options cover essentially all of the bases. Each has a different default file system choice.
Fedora you expect to use XFS. It's fast and mature. Super reliable.
openSuse you expect to use XFS or BtrFS.
Windows you expect to use NTFS, but in very rare cases you might consider ReFS, but very rare.
FreeBSD you almost always use ZFS, but you certainly can use UFS where you have a very basic system and performance matters. -
So the real question would be... what's the use case of the NAS? That can lead us in many directions. If you need SMB3, for example, we must go with Windows. If it will be supported by lots of people who don't know UNIX, we'll definitely need hardware RAID. What hardware do you plan to use? How big will it be?
-
Awesome. I have another question, but I'll throw it into a diff thread as you've pretty much resolved my thoughts on this one so far.
-
@markl said in To ZFS or not to ZFS... that is my question.:
So here's my followup question: I have hear this statement, "Redundancy for its own sake is a bad thing, only redundancy that creates reliability is good" a lot in RAID forums... so what do you feel actually generates the best reliability when discussing NASes for SMBs? How should I be thinking about it.
What you want to think about is "resultant reliability." That is, the reliability of the final product. A great example that I love is the brick house versus two straw houses that are near each other.
Now imagine two disasters... a hurricane or a fire. Hurricane... the brick house is not redundant but will almost certainly be more reliable than two or even five straw houses. The wind will just sweep them away. Redundancy didn't help enough to overcome the advantages of building with better materials.
In the second example of a fire, the straw houses are not just more likely to catch fire and burn than the brick house, but having additional straw houses actually increases the chances that any of them will catch file and each has a high chance of catching the others on fire. So the more straw houses you have, the higher the chance of failure!
This is how many SANs work. They have straw house parts where one failing causes others to fail. So you might end up with much more risk than if you had not had redundancy at all.
So when it comes to storage, you have loads of factors to consider and no simple equation to figure it out. But you have to consider the whole stack from the protocols to the hardware to the filesystems, RAID, RAIN, and so forth.
-
@scottalanmiller So let's get specific, here's my next post: My Home NAS
-
@markl said in To ZFS or not to ZFS... that is my question.:
@scottalanmiller So let's get specific, here's my next post: My Home NAS
I'm on it
