Add 2.5" U.2 (NVMe) SSDs to custom build?



  • Have some parts laying around and want to construct a new lab for playing with. I really want to go with 2.5" U.2 NVMe SSDs but I'm confused on how to actually connect them to the MB. Are there no PCIe HBAs (perhaps the wrong term) that allows you to add NVMe U.2 drives to the system?

    I saw this which I guess could work. But I really want to go with two drives and run a MDADM mirror so I'd have to get two of those "interfaces" and stuff two drives in the back of the chassis over the PCIe lanes if I went this route.

    Icy Dock makes these cool little kits that I can add into the existing 3.5" external bays in the case. But, again, what do I connect the backplanes to?

    Surely it can't be this complicated to add U.2 2.5" NVMe SSDs to a custom build?

    Edit: Found this which I wouldn't give a $1 much less the ridiculous price that they are asking.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    Surely it can't be this complicated to add U.2 2.5" NVMe SSDs to a custom build?

    It totally would be. Those are enterprise drives really only meant for use with enterprise OEM builds.



  • In general, U.2 devices require a full how swap drive chassis and matching back plane. That's not really "built it yourself" style gear. What hardware are you using that you want to add U.2 into?

    If you think about why and when this hardware exists (U.2 drives), as it is purely for high end, enterprise, hot swap use cases, you'd never expect to find hardware to support it in a white box kind of scenario. Nothing wrong with the idea, just not something that the market is going to support.

    If you want to "build for yourself" a server with these kinds of parts, you'd always start with an enterprise chassis that has at least the majority of the pieces to this built in.



  • @scottalanmiller Well there is tons of Enterprise stuff that makes its way to the consumer market. ICY DOCK selling U.2 style racks/bays means they expect you to connect it to "something" and I don't think anyone thinks of ICY DOCK as Enterprise.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    Well there is tons of Enterprise stuff that makes its way to the consumer market.

    Like what? This isn't "enterprise first", it's "enterprise only". There's no consumer use case for this gear.



  • @scottalanmiller Who branded it as "Enterprise Only"? There must be some demand for the prosumer version since they are building those "adapters" that I listed in the OP.

    My other option if I want to go NVMe is M.2 I guess if its really this complicated but I still want two drives.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    Who branded it as "Enterprise Only"?

    The market. It's a product with no consumer purpose. There is a lot of gear that no consumer will use, so no company will spend engineering dollars designing and building, because there is no potential customer base.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    My other option if I want to go NVMe is M.2 I guess if its really this complicated.

    That's the consumer market equivalent. For good reason. Just think through what use case anyone would have for buying U.2 outside of the enterprise.

    And use cases like "I want to use old leftover parts" really doesn't cut it. Nothing wrong with it, it's just that it's an insanely niche market that doesn't justify making parts. In most cases, it's just as cheap for the customer to buy new M.2 drives as to buy adapters for U.2 drives. So why would they buy less useful U.2 adapters, instead of new M.2 drives?

    It's all about market forces. U.2 are expensive, rare parts because they add a function, hot swap, that only enterprises need and is very costly and complex to offer.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    Icy Dock makes these cool little kits that I can add into the existing 3.5" external bays in the case. But, again, what do I connect the backplanes to?

    that's just a drive holder, it doesn't "do" anything. So super cheap to make and doesn't solve any issues. So not really relevant to the discussion. Presumably mounting the drive wasn't an issue already.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    There must be some demand for the prosumer version since they are building those "adapters" that I listed in the OP.

    Those aren't U.2 adapters, they are just 2.5 to 3.5 size adapters. Quite a different thing. It still leaves you with the U.2 problem. So that doesn't indicate what you are thinking that it indicates.



  • @scottalanmiller Its a drive holder but what are they expecting you to connect it to? They wouldn't make them if they didn't see some type of demand.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    @scottalanmiller Its a drive holder but what are they expecting you to connect it to?

    They expect you to already have U.2 stuff, otherwise, why do you have U.2 drives 🙂 It's just for people with U.2 chassis that are 3.5" setup and they got 2.5" drives. It's just a piece of metal that allows small drives to fit in large drive spaces. All the "what it connects to" is completely outside of their realm of concern as they aren't that kind of company.







  • @scottalanmiller Ok....

    Yeah I've seen that Supermicro HBA reviewed over at servethehome but it was such an old review I figured there would be more stuff out since then.

    That Highpoint card is over $400 on Amazon!

    Alright, I guess M.2 it is. Just need to figure out a two drive setup.



  • The problem that you'll run into is that you don't have a physical backplane, because the entire chassis has to be built to handle U.2. So just having an HBA doesn't fix the fundamental issue of lacking the enterprise server chassis to make it all come together.

    What you can do is just cable the drives directly to the HBA. This will work but doesn't allow for your hot swap.

    This is where the Icy dock is useful. It provides a stable "one drive" backplane of a sort tha tyou can connect to with this cable.

    https://highpoint-tech.com/USA_new/series-ssd7120-overview.htm

    https://www.amazon.com/High-Point-SSD7120-Dedicated-Controller/dp/B0774WLSH4



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    That Highpoint card is over $400 on Amazon!

    Yeah, there really isn't a market for cheap U.2 gear. Think of it like a Ferrari specific part. There's no call for a cheap version. Anyone who owns a Ferrari is going to pony up for genuine high performance parts. The number of Ferrari owners who need parts on the cheap is so few that there is no market to support them.



  • @scottalanmiller Yeah but, Highpoint? Man, I'd have a hard time paying them anything!



  • @scottalanmiller ICY DOCK sent me this .pdf that shows compatible HBA/RAID cards with that bay/tray and none of them are inexpensive.

    Alright, back to M.2. Not worth spending this kind of money and not getting Enterprise level quality from end-to-end.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    I figured there would be more stuff out since then.

    I've never seen a company adopting U.2. It's a very rare standard. Certainly out there, but even in the SSD space it remains pretty niche. I'm not sure if it is gaining adoption, or losing it, at this point. It's certainly a valid platform, and has important use cases, but it remains pretty uncommon. In the very large enterprise we see it, in the SMB it's pretty much non-existant.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    Alright, back to M.2. Not worth spending this kind of money and not getting Enterprise level quality from end-to-end.

    Exactly, and that's what drives cheaper makers out of the market. There's no money to be made making cheaper components. The drives are uber expensive, and their benefits don't matter to the consumer, prosumer, or even SMB markets generally. So no one is buying super expensive drives, just to them be saddled with all the additional complexity of supporting them.

    Chicken and egg. In this kind of market, there is both no chicken and no egg, so the cycle never kicks off to make it affordable.



  • We've added U2 drives to servers that were not originally made for it, so I have a pretty good idea what you need.

    U2 is physically a 2.5" drive. NVMe is physically a PCIe bus connection.

    Those are the two things you need to consider - how to mount it and how to connect it.



  • The difference between M2 and U2 is the physical format and thermal capability. The U2 is made to have efficient cooling as it is basically made like a heat sink.



  • U2 drives are the same price as enterprise SAS drives but faster and better.

    They are usually not more expensive than M.2 drives either, if you pick the same model drives.

    For instance Intel P4511 is the read intensive enterprise M.2 drive and Intel P4510 is the U.2 drive and the price is the same. U2 drives comes in larger capacities though.

    Enterprise M.2 drives are usually a little longer than their consumer counterparts (22x110mm instead of 22x80mm).



  • A third option for NVMe drives is the HHHL format which is when the SSD is built directly on a PCIe card and you put it in a PCIe slot.

    alt text



  • Yeah I wanted U.2 simply because I like the "physical rectangular" drive and it can be slipped in and out of that ICY DOCK drive tray without having to break open the case to swap a drive out (although I can't imagine I'd be doing this ever). With M.2, you have to screw them in and they just feel "flimsy" to me. Again, this was a lab so I'm not too worried about reliability. Just wanted functionality.

    There are also some speed differences if your're concerned with IOPS (as far as the Samsung Data Center drives are concerned) where the U.2 NVMe drives seem to be a bit faster than their M.2 NVMe counterparts.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    Yeah I wanted U.2 simply because I like the "physical rectangular" drive and it can be slipped in and out of that ICY DOCK drive tray without having to break open the case to swap a drive out (although I can't imagine I'd be doing this ever). With M.2, you have to screw them in and they just feel "flimsy" to me. Again, this was a lab so I'm not too worried about reliability. Just wanted functionality.

    There are also some speed differences if your're concerned with IOPS (as far as the Samsung Data Center drives are concerned) where the U.2 NVMe drives seem to be a bit faster than their M.2 counterparts.

    Yes U.2 are usually faster but the real difference is that they are faster with intensive workloads because the U.2 drives can stay cool while the M.2 drives get hot and have to slow down.

    The question is what drives did you have in mind? While small capacity U.2 drives were made in the past, now they usually aren't available in smaller capacities than 1TB.



  • @Pete-S
    Well before @scottalanmiller ruined my life I was looking at twin MZ-QLB1T9NE. 🙂

    But that was also before I had to figure spending another $600+ on bays/HBA/cables etc. I was figuring I could get a cheap HBA off Amazon.

    I'll have close to $1500 in the system in just drives. Seems a bit silly for a lab of parts at this point.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    With M.2, you have to screw them in and they just feel "flimsy" to me. Again, this was a lab so I'm not too worried about reliability. Just wanted functionality.

    They are quite solid. but that same Icy Dock unit takes M.2 drives as well.



  • @biggen said in Add 2.5" U.2 (NVMe) SSDs to custom build?:

    I'll have close to $1500 in the system in just drives. Seems a bit silly for a lab of parts at this point.

    unless NVMe itself is something you are labbing... stick with spinners or really cheap SSDs in a lab.


Log in to reply