@Dashrender said in I have to change cloud drive service yet again:

@Donahue said in I have to change cloud drive service yet again:

Now if the OS had the ability for tags to be created automatically based on say the words in a document, then you might start down a better path. But let's say that your data was images. This cannot be done outside of things like neural networks, and even then only to a point.

Not sure you need to leave this to the OS - but I get your point. And just look at google photos - it does a pretty awesome job or knowing what a bird vs a dog is.

As for auto tagging - some apps can and do have that - I have no personal experience, I just know that some can do it.

But regarding making a user 'tag correctly' well they aren't tagging or naming correctly today. Creating a list for them to choose from, plus allowing them to make their own hopefully only makes things better for them, I can't see how it would be worse than what they have today.

I'm not saying it would be worse, I'm saying you can't fundamentally make it better via programming.

TANSTAAFL

@scottalanmiller said in Large or small Raid 5 with SSD:

@Dashrender said in Large or small Raid 5 with SSD:

@scottalanmiller said in Large or small Raid 5 with SSD:

@Dashrender said in Large or small Raid 5 with SSD:

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

And while say a 24-48 hour decision window plus rebuild time is a lot more exposure than an instant rebuild time, it is still quite low?

So that the first drive has failed is unrelated. Once we hit this window, it is, say, 48 hours of "decision" and say 8 hours of rebuilding. During which time, there is no protection.

1. Why would you add 48 hours of exposure with NO RAID at all, for no reason?
2. There is only one possible outcome of the decision, to replace the drive. There is no condition under which you would not replace the drive, so why introduce a two day risk window without potential benefit?

perhaps that comes from what I have read, and perhaps what I have read would have made sense with spinners and initializing the rebuild inducing the second drive failure. Presumably that extra time would be to make sure all my ducks are in a row with fresh backups and such, but perhaps that is where my error is, and I should know my ducks were in a row long before the first failure.

With spinners, resilvering can take weeks or months of time, rather than hours, and generally has 6TB+ to resilver with high URE rates. SSDs take hours to resilver, with generally under 1TB of capacity, with low URE rates. So the factors of one apply poorly to the other.

why only 1 TB of capacity?

How big do you expect SSDs to be when you have many in an array realistically?

so you're talking about the single drive, not the array. Got it.

Though when resilvering, you still read the entire array worth.

Correct, the time to resilver is primarily based on the size of the drive being rebuild. That's the bottleneck, the time to write data back to the one drive.

So if 4x 10TB drives takes 2 days to replace a drive.
8x 5TB drives would take 1 day to replace a drive.

It's not exact, but it is really close.

but with twice the chance of having to rebuild.

@scottalanmiller said in Adding tape drive:

@Donahue said in Adding tape drive:

@scottalanmiller said in Adding tape drive:

@Donahue said in Adding tape drive:

ok, If I understand this, Scott's championing OBR10 as the starting point for a production load...

OBR10 when using spinners, as a starting point for decision making. No RAID should be chosen without clear planning, every situation is unique. But if you can't find something that mathematically is clearly better for you when using spinners, you fail to OBR10 because it is the safest and fastest and without clear financial benefit, you don't want something else.

I interpret all that as OBR10 = gold standard, or as you would put it, "best practice". I have probably been guilty of over simplifying for the sake of laziness efficiency.

Right, a lot of people have repeated "starting point" as "best practice" which are wholly different things. A BP means you have no discussion, there is only one way to do it. Starting point means the opposite, there is no BP and no given answer, you have to figure it out.

Think of it as a starting point for choosing a car. "Well, you should look at the dealer nearest your home first, since you know it is conveniently located and you have nothing else to go on before test driving some different cars." Buying whatever car is closest to your home is hardly a best practice, but it is a reasonable way to start a process of test driving different models.

I loosely define "standard" to mean the same thing that you say when you say "best practice". I would call "starting point" the same as "default"

For the sake of this thread, I am probably going to use 3.84TB SSD's, but the point remains.

@scottalanmiller said in Adding tape drive:

@Donahue said in Adding tape drive:

ok, If I understand this, Scott's championing OBR10 as the starting point for a production load...

OBR10 when using spinners, as a starting point for decision making. No RAID should be chosen without clear planning, every situation is unique. But if you can't find something that mathematically is clearly better for you when using spinners, you fail to OBR10 because it is the safest and fastest and without clear financial benefit, you don't want something else.

I interpret all that as OBR10 = gold standard, or as you would put it, "best practice". I have probably been guilty of over simplifying for the sake of laziness efficiency.

@hobbit666 said in What Are You Doing Right Now:

Wondering how you get the "old" generation to see other options as valid business options and not what the "sales" people tell them?

e.g.
Hyperconverged (e.g. Scale) over 3-2-1 SAN setup.
Using FreePBX instead of paid for onprem BCM/AVAYA phone systems?

wait for them to retire?

@RojoLoco said in Non-IT News Thread:

@dafyre said in Non-IT News Thread:

@strongbad said in Non-IT News Thread:

@obsolesce said in Non-IT News Thread:

@jaredbusch said in Non-IT News Thread:

@mlnews said in Non-IT News Thread:

Amritsar: India train mows down crowd killing scores

I am not sorry that you stood on train tracks and got run over by a train. I can feel sympathy for the families, but not for the stupid.

But the kids... Just standing with their parents.

Very sad, they think people couldn't hear the train coming.

Even if they couldn't hear it, you'd think they would still feel it.

Yeah.... trains are loud and cause a lot of vibration on both the tracks and the surrounding ground. Not sure how people couldn't hear it coming.

who stands on an active track and doesn't assume a train could come at any minute? That's like darwin award worthy.

@scottalanmiller read the print. This guys was arrested because he said something "that could be offensive"

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

And while say a 24-48 hour decision window plus rebuild time is a lot more exposure than an instant rebuild time, it is still quite low?

So that the first drive has failed is unrelated. Once we hit this window, it is, say, 48 hours of "decision" and say 8 hours of rebuilding. During which time, there is no protection.

1. Why would you add 48 hours of exposure with NO RAID at all, for no reason?
2. There is only one possible outcome of the decision, to replace the drive. There is no condition under which you would not replace the drive, so why introduce a two day risk window without potential benefit?

perhaps that comes from what I have read, and perhaps what I have read would have made sense with spinners and initializing the rebuild inducing the second drive failure. Presumably that extra time would be to make sure all my ducks are in a row with fresh backups and such, but perhaps that is where my error is, and I should know my ducks were in a row long before the first failure.

ok, If I understand this, Scott's championing OBR10 as the starting point for a production load, but it may not be the starting point, or in this case the end point on a backup storage, especially if IOPS are not the primary concern? All this seems like just doing 7 drives in raid 6 and using the tape drive on the extra channel is the right way to go.

Its very easy for someone like me to read some of the stuff that Scott has written as "the most prolific raid author in history" as authoritative and think of it as the gold standard.

https://community.spiceworks.com/storage/articles/2801-one-big-raid-10-the-new-standard-in-server-storage

@scottalanmiller said in Adding tape drive:

@Donahue said in Adding tape drive:

@Obsolesce said in Adding tape drive:

@Donahue said in Adding tape drive:

between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.

If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.

[[storage] + [backup server]] --> Network --> [restore location/server]

the backup server will have onboard storage, but will be backing up from and restoring to a 10G network.

Right, so very likely the speed of the backup system's drives and networking will be the bottleneck.

Specifically the drives themselves, and not the array?

am I wrong to think that the probability of two drives failing is much less than the probability of just one drive failing? And while say a 24-48 hour decision window plus rebuild time is a lot more exposure than an instant rebuild time, it is still quite low?

@Obsolesce said in Adding tape drive:

@Donahue said in Adding tape drive:

between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.

If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.

[[storage] + [backup server]] --> Network --> [restore location/server]

the backup server will have onboard storage, but will be backing up from and restoring to a 10G network.

@scottalanmiller said in Adding tape drive:

@Donahue said in Adding tape drive:

between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.

Not really, speed doesn't vary in a meaningful way to you. What factor do you feel is driving you to RAID 10?

you shouting from the mountain tops that OBR10 is the gold standard.

@scottalanmiller said in I have to change cloud drive service yet again:

@Donahue said in I have to change cloud drive service yet again:

Now if the OS had the ability for tags to be created automatically based on say the words in a document, then you might start down a better path. But let's say that your data was images. This cannot be done outside of things like neural networks, and even then only to a point.

Would maybe still know when it was taken, where it was taken, etc.

true.

between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.

I am also going to be adding a lot more storage space to this host, and I've got a raid card in it now that will support 8 drives. I go with a 7 drive raid 6 (HDD's) and have an open spot for the tape drive, or I can stick with 8 drives in raid 10 and just buy another HBA.

We currently use veeam in a VM on ESXi, but I am changing my whole setup soon. I am currently backing up to disk, mostly to a few synology boxes that are presenting their storage to ESXi to distribute to the VM's. In the role of offsite, which is where the tapes would come in, I am doing weekly backups to a external SSD via usb3 and then taking this offsite. But the size of these backups are very limited and they take forever because of the way my network is configured (also fixing this). I am not currently using veeam for these offsite backups, I am forced to do file level copies because of my limited storage space on the usb drives. But inside of veeam, I am using the GFS scheme to keep a limited number of backups on disk at our two physical locations.

I still need to decide if I want to do the GFS scheme with tape, or with onsite disk. I originally had the idea to use disk for those, but Scott tried to talk me into using the tapes for this.

I use reverse incremental in veeam, and I dont ever have to take "fulls".

Now if the OS had the ability for tags to be created automatically based on say the words in a document, then you might start down a better path. But let's say that your data was images. This cannot be done outside of things like neural networks, and even then only to a point.