@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:

I see on xbyte, the PM863a is a lot cheaper, though I can't tell if that's a used/refurb part. What other places would you suggest I look?

It's just a bit long in the tooth at this point (That drives been around 2-3 years since the refresh). It's also low endurance TLC with a smallish DRAM and SLC buffer. It's not going to take sustained write throughput very well.

@stacksofplates said in Discovering FakeRAID in the Real World:

@scottalanmiller there isn’t a software based raid option on ESXi unless you pass it to a VM or use vSAN. The cheapest raid controllers supported are the BOSS modules M.2 mirroring things.

I think that's what our VxRails are coming with.

14Gen VxRAIL and ReadyNodes use's BOSS for the boot modules. VxRAIL keeps the factory re-install image I think on the SD Cards in case you need to factory nuke a node.

@scottalanmiller there isn’t a software based raid option on ESXi unless you pass it to a VM or use vSAN. The cheapest raid controllers supported are the BOSS modules M.2 mirroring things.

@scottalanmiller said in Discovering FakeRAID in the Real World:

Dell and HPE both make some really silly software RAID devices that require hardware dongles and configure through the BIOS, but are NOT FakeRAID.

You talking about the S series, or the Intel CPU ROC functions for NVMe drives (which have a dongle for certain features, or the ability to use non-Intel drives). I thought the OEM's were shunning VROC.

@scottalanmiller said in Discovering FakeRAID in the Real World:

@pete-s said in Discovering FakeRAID in the Real World:

@dustinb3403 No, Broadcom 3008 has it's own integrated cpu, on board memory etc. It's just not enough performance for it to do all kinds of raid. It's the same controller as Dell H330 and a bunch of others make cards with it as well.

From the specs:

The SAS 3008 offered with the Avago Integrated RAID (IR) feature is a
low cost, high performance RAID solution designed for blade, entry and
mid-range servers that require redundancy and high availability but where
a full featured RAID implementation is cost prohibitive or not desired. The
Avago advanced Integrated RAID options include Integrated Mirroring
(IM), which is RAID 1, Integrated Mirroring Enhanced (IME), known as RAID
1E, Integrated Striping (IS), which is RAID 0, and Integrated Mirroring and
Striping (IMS) which is RAID 10. By simplifying the RAID configuration
options, Integrated RAID is easy to install and configure and meets the
needs of most internal RAID requirements.

https://mangolassi.it/topic/6375/examining-the-dell-perc-h310-controller/

The H3xx series is hardware. Terribly slow hardware, but hardware.

The H330 is a 3008 based product but it has a different firmware package, and some cut down internals so it has a much lower queue depth, and a bastardized mega-raid capability (no write cache). There is memory on the card, but it's for the queues.

The HBA330 isn't slow it @#%@% flies but it's pass through only. It is the full queue'd (exposes the raw possible queue based on the memory which is crazy high), and has no firmware option for a RAID firmware.

The H310 was a LSI2008 based ASIC package which had its queue crippled to something pathetic like 25.

Dell runs their own trunk for HBA firmware's rather than just grabbing various checkpoints on the tree is my understanding, so beyond these customizations, you will sometimes see different behavior. Sometimes they will selectively ship a fix, and avoid a feature they found problematic or didn't want (Tri-Mode RAID wasn't supported on the newest PERC's at first).

The controller industry is crazy opaque.

@dustinb3403 said in Discovering FakeRAID in the Real World:

Software RAID works across any OS, Linux, Windows and (probably) MAC all include this. It's hardware agnostic and could in theory allow you to mount an array to recover data.

This big thing to be careful of is drive swaps with software RAID. If you can't flag the drive to replace (especially dangerous with SAS Expanders, and JBODs where the numbering schemes sometimes start with 0 sometimes with 1).

@dashrender said in SAS SSD vs SAS HDD in a RAID 10?:

This really does boil down to math, but odds are of course never zero, and someone does have to be the one who suffers the failure outside of the typical odds from time to time.

Human error tends to be the biggest cause. People go to replace a drive while a rebuild is going on and swap the wrong drive.

@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:

OBR5 is the standard if you are going to be using an SSD

A URE isn't the only failure or corruption mode on a SSD. You can have drives that are not dead, but you want to shoot (firmware acting squirrel and you get 500ms). Also, 16TB SSDs that have deduplication and other data services in front of them can take a LONG TIME to rebuild (making that 7+1 a non-fun rebuild).

Throw in people using cheap TLC and QLC (crap write speed and latency after the DRAM and SLC buffer exhausted) and I wouldn't say as a rule RAID 5 for traditional RAID groups of SSDs is always a good idea. If you have an SDS layer that wide stripes across multiple servers, and limits the URE domain to an individual object this is a bit more controlled. If I have a small log file that writes in a circle a lot (My Casandra/REDIS systems) erasure codes may not be worth it has given the volume of ingestion.

I'm a bigger fan of RAID 5 on SSD in systems where I can pick and chose my RAID level on a single object, LUN etc so I can break up the write outliers that are small.

@scottalanmiller said in Which skills is more valuable to you? Hard skills or Soft skills?:

But in the enterprise, so few people need to talk to anyone but other tech people, the interfaces to the outside world are few and far between.

I agree here, but there is a minimum viable product of soft skills where even other tech people don't want to put up with your shit, or your ability to communicate with other tech people is crap and so you don't get good feedback or collaboration. The 10x engineer who belongs in a closet is something that fewer and fewer companies will put up with. Hell Linus recently admitted he needs to stop being an asshole. Netflix has an official policy to not keep them.

Our engineering have an internal-only R&D conference every year where they pitch idea's and their work to each other (it's like an academic conference with papers and quick talks) and there is a lot of value in them being able to pitch their new crazy idea. We have Internal reading groups, and other presentations that are done weekly where people talk about their work. A ton of good feedback on projects comes from this. Technical skills can be advanced by soft skills.

@scottalanmiller said in Which skills is more valuable to you? Hard skills or Soft skills?:

This, HUGELY depends on role. For some roles, I don't care at all. For others, soft is all that matters, more or less.

The problem is roles where you need more technical than soft skills, in consulting end up with that person "fronted" by a project manager etc. If that person doesn't communicate with the PM then eventually that can be a problem. If they only work with other technical team members, eventually toxic personalities cause problems. There are fewer "truly isolated" IT jobs than I think people realize.