When to replace hard drive in a RAID array
-
@Texkonc said in When to replace hard drive in a RAID array:
@RojoLoco said in When to replace hard drive in a RAID array:
@EddieJennings said in When to replace hard drive in a RAID array:
@Texkonc 3 TB (WD Red)
Ouch... if you try to rebuild that array and it works (don't hold your breath), I'd go out and buy a ticket to every lottery you can, because you'll never have that kind of luck again.
I had StoreVirtual dual node SAN with 24 4TB drives take 7.5 days to repair without issue or popping another drive. Raid 6 thankfully though.
I don't know if I could handle 7.5 days without sleep!
-
@scottalanmiller The chance of failure on the resilver is what frightens me, which, contrary to what I posted a couple of minutes ago, makes me want to make the drive swap happen when I redo the RAID as RAID 10. For that matter, I'll also look and see what the cost would be to add drives to the server that connects to the NAS via iSCSI and just have the data stored locally.
-
@EddieJennings said in When to replace hard drive in a RAID array:
@scottalanmiller The chance of failure on the resilver is what frightens me...
That's why to deal with the whole thing now. Consider it an emergency situation.
-
Wealth of knowledge has been gained in the last few minutes -- in particular how long it would take to resilver an array, which puts into perspective how dangerous RAID 5 is.
-
@RojoLoco said in When to replace hard drive in a RAID array:
If you have the space, back the data up to another location and blow away the RAID 5, toss the sick drive, rebuild into a RAID 10. Far less risk that way vs. add a new drive and pray that it rebuilds, plus no extra disks needed. The sooner the better on making a new array, I don't know if I would risk replacing a drive in a RAID 5 array (I'm making the assumption that these are 1TB + drives, which means that you have about as much chance of a successful rebuild as you have of getting hit by lightning).
^ this exactly. Order larger drives today if you have too little space after RAID10 conversion.
-
@RojoLoco said in When to replace hard drive in a RAID array:
@Texkonc said in When to replace hard drive in a RAID array:
@RojoLoco said in When to replace hard drive in a RAID array:
@EddieJennings said in When to replace hard drive in a RAID array:
@Texkonc 3 TB (WD Red)
Ouch... if you try to rebuild that array and it works (don't hold your breath), I'd go out and buy a ticket to every lottery you can, because you'll never have that kind of luck again.
I had StoreVirtual dual node SAN with 24 4TB drives take 7.5 days to repair without issue or popping another drive. Raid 6 thankfully though.
I don't know if I could handle 7.5 days without sleep!
Trust me, I woke up some nights to see if my VPN was still up. (an RRAS VM) then log into storage and check the percent complete.
-
@EddieJennings said in When to replace hard drive in a RAID array:
@scottalanmiller For that matter, I'll also look and see what the cost would be to add drives to the server that connects to the NAS via iSCSI and just have the data stored locally.
That would likely make way more sense.
-
Yeah they're still going to be of the same quality (NAS drives) but you'd be in a non-parity array.
Adding more drives would be a boost if you can fit them in as the entire system will operate that much more quickly.
-
@Texkonc said in When to replace hard drive in a RAID array:
@RojoLoco said in When to replace hard drive in a RAID array:
@Texkonc said in When to replace hard drive in a RAID array:
@RojoLoco said in When to replace hard drive in a RAID array:
@EddieJennings said in When to replace hard drive in a RAID array:
@Texkonc 3 TB (WD Red)
Ouch... if you try to rebuild that array and it works (don't hold your breath), I'd go out and buy a ticket to every lottery you can, because you'll never have that kind of luck again.
I had StoreVirtual dual node SAN with 24 4TB drives take 7.5 days to repair without issue or popping another drive. Raid 6 thankfully though.
I don't know if I could handle 7.5 days without sleep!
Trust me, I woke up some nights to see if my VPN was still up. (an RRAS VM) then log into storage and check the percent complete.
This is why I keep a bottle of Pepto + sleeping pills in my tech emergency kit.
-
When I started at my current place, we had an 8TB OBR5 with a failed disk, and our MSP didn't even notice it during maintenance...
How they weren't fired immediately I have no clue.... restoring the entire thing would've ticked me off to no end. We got super lucky and the resilver completed without issue. Something like 23% chance of success... . .
This system is now gone..
-
What is your storage footprint like? % used/free.
Can you buy a two bay synology for under $200 on amazon and get some drives for it, copy your stuff there and blow away?
Remember you cant have 5 drives in R10, with your setup at least.
2 pairs of the 3tb drives, so you only get 6TB then a hotspare, but the space you loose. So think of that.. Will you need larger drives? What is your growth like? -
@Texkonc Yeah, I know I can't have 5 in a RAID 10. Currently we're using under 500 GB of this storage, so losing that drive isn't significant. Once my current two hair-on-fire esque tasks are done, I'll get on figuring out the procedure.
-
@EddieJennings said in When to replace hard drive in a RAID array:
@Texkonc Yeah, I know I can't have 5 in a RAID 10. Currently we're using under 500 GB of this storage, so losing that drive isn't significant. Once my current two hair-on-fire esque tasks are done, I'll get on figuring out the procedure.
Only 500GB!
Go to staples now and buy a 1TB external and move the data off of this array.
-
@EddieJennings And growth? 500 GB in about 6 years (which I believe is the age of the NAS and its drives).
-
@EddieJennings said in When to replace hard drive in a RAID array:
@EddieJennings And growth? 500 GB in about 6 years (which I believe is the age of the NAS and its drives).
Moving the 500GB off of this and onto a USB 1TB would likely be way faster than trying to resilver too.
Send someone out to get a drive now, don't "wait to finish putting out those fires".
-
@EddieJennings said in When to replace hard drive in a RAID array:
@EddieJennings And growth? 500 GB in about 6 years (which I believe is the age of the NAS and its drives).
If you have 500GB in use today, and its been in place over the past 6 years, your delta is tiny.
You'd be fine with a 2 bay with 2x2TB drives in RAID1.
-
I'm with Dustin, go to Walmart and fix this, this instant. If the data isn't worth $100, then scrap the whole thing. If the company can't come up with $100, find a different job. In all other cases, just fix it right now
-
@scottalanmiller said in When to replace hard drive in a RAID array:
I'm with Dustin, go to Walmart and fix this, this instant. If the data isn't worth $100, then scrap the whole thing. If the company can't come up with $100, find a different job. In all other cases, just fix it right now
Hell $100, I'm seeing drives in stock at my local staples and walmart for under $60!
-
@EddieJennings said in When to replace hard drive in a RAID array:
@EddieJennings And growth? 500 GB in about 6 years (which I believe is the age of the NAS and its drives).
That's awesome news man, you have 4 good drives left to make a sexy OBR10 array that will serve them well for years
-
Hell, he has a Synology. He should be able to add the spare drive he has as a standalone disk. Move the share. nuke the array and rebuild with the 4 good drive and then move the share back.
