How would you build a File server with 170TB of Usable Storage?
-
If you need more storage density you could also go with another chassis using the NF1 format SSDs from Samsung.
That's a whopping 0.54 PB in 1U.@dafyre I checked our supplier and fully stocked with 36 15TB Samsung PM983 NVMe drives it would be priced at around $115K with 768GB RAM and two Xeon CPUs and a couple of 25Gb NICs.
https://www.supermicro.com/a_images/products/views/1029P-NMR36L_angle.jpg
-
I need to dig in and see what Chasis our new Nimble Storage units are (They're the new ones form HPE). I know those run about $60k each. I need to see how much storage we get in each.
-
@Pete-S said in How would you build a File server with 170TB of Usable Storage?:
I'd just put twenty 15TB NVMe drives in my little 1U server and call it a day.
https://www.supermicro.com/a_images/products/views/1029UZ-TN20R25M_angle.jpgThis is the best answer for the given question. Anything else is going into assumption land which was specifically not allowed.
-
@JaredBusch said in How would you build a File server with 170TB of Usable Storage?:
@Obsolesce said in How would you build a File server with 170TB of Usable Storage?:
Misread. So how much of the 170TB is used immediately and what's the projected growth rate?
Not relevant. The concept is that we need 170TB usable. Usable implies available now.
You are asking a different question to resolve a different problem.
Your own configuration does not provide 170TB usable. You calculated using unformatted capacity, and didn't take into account that even unformatted, a 14TB drive is only a decimal representation used for marketing, which takes another chunk out of the usable capacity
-
@Vlinderbeest said in How would you build a File server with 170TB of Usable Storage?:
@JaredBusch said in How would you build a File server with 170TB of Usable Storage?:
@Obsolesce said in How would you build a File server with 170TB of Usable Storage?:
Misread. So how much of the 170TB is used immediately and what's the projected growth rate?
Not relevant. The concept is that we need 170TB usable. Usable implies available now.
You are asking a different question to resolve a different problem.
Your own configuration does not provide 170TB usable. You calculated using unformatted capacity, and didn't take into account that even unformatted, a 14TB drive is only a decimal representation used for marketing, which takes another chunk out of the usable capacity
Plus RAID and FS overhead. With 14TB drives using mirroring, you'd need at least 26 drives, and 28 to be safe.
-
Just for the amusement factor, I just checked out one of the 90 bay SuperMicro chassis. With 14TB drives, that's 500TB usable in a single 4u chassis. All for around $60k
-
Was working with HPE today on a Nimble Storage controller... Turns out they use SuperMicro something or another. I forgot to bookmark the page before I logged out of the machine I was using. I'll find it tomorrow.
-
@dafyre said in How would you build a File server with 170TB of Usable Storage?:
Was working with HPE today on a Nimble Storage controller... Turns out they use SuperMicro something or another. I forgot to bookmark the page before I logged out of the machine I was using. I'll find it tomorrow.
Tee hee. Now that's pretty funny. Even HPE uses SuperMicro!
-
@scottalanmiller said in How would you build a File server with 170TB of Usable Storage?:
@dafyre said in How would you build a File server with 170TB of Usable Storage?:
Was working with HPE today on a Nimble Storage controller... Turns out they use SuperMicro something or another. I forgot to bookmark the page before I logged out of the machine I was using. I'll find it tomorrow.
Tee hee. Now that's pretty funny. Even HPE uses SuperMicro!
Don't know about the chasis itself, but the mobo's are, for sure.
I'll get a screen cap if I'm able.
-
@dafyre said in How would you build a File server with 170TB of Usable Storage?:
@scottalanmiller said in How would you build a File server with 170TB of Usable Storage?:
@dafyre said in How would you build a File server with 170TB of Usable Storage?:
Was working with HPE today on a Nimble Storage controller... Turns out they use SuperMicro something or another. I forgot to bookmark the page before I logged out of the machine I was using. I'll find it tomorrow.
Tee hee. Now that's pretty funny. Even HPE uses SuperMicro!
Don't know about the chasis itself, but the mobo's are, for sure.
I'll get a screen cap if I'm able.
Oh okay HP has all kinds of people make mobos.
-
Bringing something up from the "other" thread. Would RAID rebuild times play a factor into choosing a method of supplying this kind of storage amount?
-
@hobbit666 said in How would you build a File server with 170TB of Usable Storage?:
Bringing something up from the "other" thread. Would RAID rebuild times play a factor into choosing a method of supplying this kind of storage amount?
My first thought is YES! DEFINITELY! However, it actually may depend on how you set it up. Systems like Ceph and Gluster don't necessarily need RAID since they can work with different disks in the same node.
-
@hobbit666 said in How would you build a File server with 170TB of Usable Storage?:
Bringing something up from the "other" thread. Would RAID rebuild times play a factor into choosing a method of supplying this kind of storage amount?
Absolutely, that's why at scale you are generally limited to RAID 10. RAID 60 can work, but with so many RAID 6 arrays in the pool that you save extremely little compared to RAID 10, but you lose so much that the balance is rarely worth it.
RAID 10 is the only level that really scales because of the rebuild problems.
-
@dafyre said in How would you build a File server with 170TB of Usable Storage?:
Systems like Ceph and Gluster don't necessarily need RAID since they can work with different disks in the same node.
They definitely don't need RAID. That's their point.
