ZFS is Perfectly Safe on Hardware RAID
-
@saniplastic said in ZFS is Perfectly Safe on Hardware RAID:
hello to all
in my freenas server, each disk is on Raid0 Hardware Raid P410i and separate raid0 disk present to zfs and use freenas raid and disable S.M.A.R.T services on freenas and let RAID card handles the SMART monitoring, handles the alerting.
I khow i can use hardware raid and present single logical unit to freenas. but our server is under load. should i use hardware raid or continue with exist config ?
what is your idea about this?Quoting here so that you will see this if you come back to this thread, but going to split out the question to its own thread as it is a unique question.
-
Recently I am noticing a change in you when it comes to ZFS.
Also Poutine is overrated.
-
@Emad-R said in ZFS is Perfectly Safe on Hardware RAID:
Also Poutine is overrated.
OMG, that's it, I can't take anyone seriously on this forum! Sacré bleu!
:face_with_stuck-out_tongue_winking_eye:
-
@Emad-R said in ZFS is Perfectly Safe on Hardware RAID:
Also Poutine is overrated.
For sure, it's never been something that I liked.
-
Hmm, the actual designers and implementers of ZFS disagree with you. Here's their take on hardware RAID controllers and ZFS without a FreeNAS/iX sales pitch: "Hardware RAID controllers should not be used with ZFS. While ZFS will likely be more reliable than other filesystems on Hardware RAID, it will not be as reliable as it would be on its own." They then go on to explain why this is so with a reasonably brief amount of technical detail.
-
@pjrobar said in ZFS is Perfectly Safe on Hardware RAID:
Hmm, the actual designers and implementers of ZFS disagree with you. Here's their take on hardware RAID controllers and ZFS without a FreeNAS/iX sales pitch: "Hardware RAID controllers should not be used with ZFS. While ZFS will likely be more reliable than other filesystems on Hardware RAID, it will not be as reliable as it would be on its own." They then go on to explain why this is so with a reasonably brief amount of technical detail.
Even more of a reason to use ZFS in a VM if ZFS is necessary, or HBA in hardware if having to go down that road.
-
@pjrobar said in ZFS is Perfectly Safe on Hardware RAID:
Hmm, the actual designers and implementers of ZFS disagree with you. Here's their take on hardware RAID controllers and ZFS without a FreeNAS/iX sales pitch: "Hardware RAID controllers should not be used with ZFS. While ZFS will likely be more reliable than other filesystems on Hardware RAID, it will not be as reliable as it would be on its own." They then go on to explain why this is so with a reasonably brief amount of technical detail.
I've spoken on why this is a BS sales pitch many times. Yes, it's a slightly different team, but still one "selling" something. And remember, these are developers, not operations people, so what the developers of something feel is really worth listening to, but then we have to put on our IT hats and evaluate it carefully.... and in doing so it is really obvious that ZFS on its own is fine, but ZFS on hardware RAID is also fine and that their points aren't a reason to always choose one or the other. It is with their concerns 100% evaluated that we come to the conclusions.
If their concerns were correct, then their theory is that all production systems in the world must not be good enough to use. It's an absurd point and makes their point invalid.
-
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@pjrobar said in ZFS is Perfectly Safe on Hardware RAID:
Hmm, the actual designers and implementers of ZFS disagree with you. Here's their take on hardware RAID controllers and ZFS without a FreeNAS/iX sales pitch: "Hardware RAID controllers should not be used with ZFS. While ZFS will likely be more reliable than other filesystems on Hardware RAID, it will not be as reliable as it would be on its own." They then go on to explain why this is so with a reasonably brief amount of technical detail.
Even more of a reason to use ZFS in a VM if ZFS is necessary, or HBA in hardware if having to go down that road.
Because the maintainers aren't understanding operations and we can't trust them? Hubris is one of the most dangerous things in production.
-
@pjrobar said in ZFS is Perfectly Safe on Hardware RAID:
While ZFS will likely be more reliable than other filesystems on Hardware RAID, it will not be as reliable as it would be on its own
This is where we have to be careful. You see, if this is true, then they either agree with my point, or else believe that ALL other filesystems, ALL operating systems that don't support ZFS, and ALL hardware RAID is not safe enough to use in production.
But generally what we see is the ZFS team stating something reasonable... like that ZFS is slightly safer than other options... and then it being misinterpreted that because it is unmeasurably safer, that that must be the sole determining factor in all decisions, which is not even close to being true. And to be safer, it has to ignore certain operational realities that often make ZFS far less safe in the real world.
For example the "which is generally safer" reason, hardware RAID often trumps ZFS. ZFS might have a technical advantage, but hardware RAID often has a human one that is vastly more significant.
But, I never really see the ZFS people saying that. It's just how people read into it. But their own statements normally don't suggest that. They simply point out that ZFS on hardware RAID is maybe a tiny bit safer than say NTFS or EXT4 on it. And I think most of us would agree with that. What most people would also agree with is that all of those combinations are so safe that under all normal conditions, it doesn't matter at all.
-
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
-
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
-
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
-
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
BTW, we use linux software RAID (md) and on the LSI cards we have in several machines you just don't define any arrays and the drives show up as expected and are handled by the OS directly with full access to the drives. OS automatically loads the mpt2sas driver which is included in the kernel.
So it's a RAID card but the RAID logic lies dormant and the OS handles the drives directly.
-
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
How else would you connect 24 hard drives, or connedlct a DAS box, in that manner, without an HBA?
Why would you pay good money for a raid card just to not use it?
Ideally you would purchase a server vendor supported card.
-
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
How else would you connect 24 hard drives, or connedlct a DAS box, in that manner, without an HBA?
Why would you pay good money for a raid card just to not use it?
Ideally you would purchase a server vendor supported card.
Some servers comes with integrated RAID. At least on newer machines most of them are able to passthrough disk directly to the OS, just as I mentioned above on our LSI cards. So the RAID card works exactly like a dumb HBA - if you want.
-
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
The problem is, many RAID cards won't turn off the RAID logic and encapsulation.
-
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
In that scenario, where you've already chosen to use the software RAID and aren't in a position of having hardware that makes it problematic, yes.
-
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
How else would you connect 24 hard drives, or connedlct a DAS box, in that manner, without an HBA?
Why would you pay good money for a raid card just to not use it?
Ideally you would purchase a server vendor supported card.
Some servers comes with integrated RAID. At least on newer machines most of them are able to passthrough disk directly to the OS, just as I mentioned above on our LSI cards. So the RAID card works exactly like a dumb HBA - if you want.
Sure, but I'm guessing that's not @Obsolesce point. If you don't have an integrated solution - HBA would/should be less expensive than buying a RAID card, plus it's one less piece of complication to get in the way.
additionally - most systems don't have the ability to take 24 drives with an integrated solution.
-
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
The problem is, many RAID cards won't turn off the RAID logic and encapsulation.
Hence the need for RAID 0, which could shield the actual drive from ZFS, I'm guessing.
-
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
HBA would/should be less expensive than buying a RAID card, plus it's one less piece of complication to get in the way.
HBAs aren't always less expensive. And the complexity is identical no matter what.... you shift the complexity from hardware to OS or vice versa, but that's it. Same pieces, you are just choosing which place they go in.
But managing hardware RAID is standard and every bench tech knows how to do it, and every Windows admin, and everyone that uses FreeNAS.
ZFS is highly complicated and if you aren't a UNIX admin with ZFS experience, which essentially includes anyone that would consider something like FreeNAS, then ZFS is quite complicated from a human usability perspective. It exposes way more risk to the admin.
THis is why we see humans failing at ZFS RAID functions causing data loss all the time.
