Does block level sync exist?
-
@Pete-S Good description.
-
@Fredtx said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Right, which is just a fancy way of saying it uses VSS. Everything does that, that's not considered application aware, because absolutely everything has that level of awareness - the agent that has the awareness is part of the OS. Barracuda isn't aware of any third party applications, including those that run on top of MS SQL.
Yea, I'm aware it uses VSS. I thought that's what you were referring to when talking about application awareness. But looks like you are referring to something else that I have a lack of knowledge or understanding on.
VSS is application aware, just one only one or two applications that we rarely care very much about. It's not aware of YOUR applications, but it knows about SOME applications. In a world of hundreds of thousands of applications, it probably knows a few dozen and that's about it. Nearly all are MS applications like AD and Exchange. That's about it.
In all these years, I've never once run into any application outside of the stock MS apps that any vendor took the effort to be application aware of natively out of the box for a general purpose backup software. It's just impossible to address everyone (let alone anyone's) needs, so there's no point in pretending.
-
@Fredtx said in Does block level sync exist?:
I may look at something else such as Veeam Backup And Replication, but I would need to do further research to see how their backup product works in the back end.
That won't change the awareness issue. Do they have a sync option that meets your needs?
-
@scottalanmiller said in Does block level sync exist?:
That won't change the awareness issue. Do they have a sync option that meets your needs?
What I'm looking for is adhering to the Backup 3-2-1 best practice. The 3-2-1 strategy that Barracuda offers is a backup is copied to a local backup appliance, which is then deduplicated to save storage space on the appliance. Then, the data from the appliance is compressed and replicated to an offsite storage, which can be another appliance (our setup in my division) or their cloud storage, which I believe is in AWS.
-
@Fredtx said in Does block level sync exist?:
What I'm looking for is adhering to the Backup 3-2-1 best practice.
Well, the FIRST step in 3-2-1 is to get a reliable backup to do the 3-2-1 with. 3-2-1 itself is a really trivial part of the overall picture. That's just talking about where and how many copies you have. But it only matters if the copies you are making are good ones. You are worrying that the copies of the copies don't change, but not worrying if the original copy is any good.
-
@Fredtx said in Does block level sync exist?:
The 3-2-1 strategy that Barracuda offers is a backup is copied to a local backup appliance, which is then deduplicated to save storage space on the appliance. Then, the data from the appliance is compressed and replicated to an offsite storage, which can be another appliance (our setup in my division) or their cloud storage, which I believe is in AWS.
Just for reference, we do this using Duplicati. Rolling it out right now after testing. We've used it for years, but now making it our primary tool. It doesn't do the EXACT same process you are describing, but very similar. But single tool doing the 3-2-1 in an essentially similar way.
We are doing one local copy, one copy to NAS, a snapshot of the NAS, and an offsite to S2 or Wasabi.
-
But we are aware that Duplicati, like Veeam or Baraccuda, isn't application aware (of our applications) and that we have to accommodate for that and take a backup using application aware processes locally, then let Duplicati take a backup of THAT backup.
-
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
-
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
-
-
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
How do you keep track of all the backups if you have hundreds of duplicati installations running? There are no central backup server or UI right? All installations are independent of each other?
-
@scottalanmiller said in Does block level sync exist?:
You are worrying that the copies of the copies don't change, but not worrying if the original copy is any good.
My main concern is that the copies are good in all locations. In my case with Barracuda, the copies were not good. Actually, even worse, they were not even there.
-
@Fredtx said in Does block level sync exist?:
My main concern is that the copies are good in all locations
No your main concern is that your primary backup is good.
After that, it is just a copy operation. If you are worried about a copy operation, your solution is doing something wrong.
-
@Fredtx said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
You are worrying that the copies of the copies don't change, but not worrying if the original copy is any good.
My main concern is that the copies are good in all locations. In my case with Barracuda, the copies were not good. Actually, even worse, they were not even there.
Why do you care if you have copies if you don't know what they are copies of? Unless politically you are responsible for the copies but not the backup. Then, that makes sense. If they backup is bad, that's "Betty's fault", but if the sync of the backup is bad, that's your fault. That would make sense.
If that's not the case, you are looking at this all wrong. As a company, the backup is what matters and the sync is completely irrelevant until that is known to be good and once it is good, the sync can be pretty important, but never as important as the thing that is being sync.
It's similar to caring about how good the lock on your safe is but not caring that the back of the safe is missing and people can just walk by and steal everything without opening the door in the first place.
-
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
How do you keep track of all the backups if you have hundreds of duplicati installations running? There are no central backup server or UI right? All installations are independent of each other?
Ah, excellent question. So there is no central server, that is correct. All installations are independent, also correct. But they are open source and have some APIs and messaging options. That's great when you have one or two. But with hundreds (like we will have once the conversion is done) you need something more.
Thankfully there is a non-open source Duplicati monitoring service that we are using that is cheap and does just that. It does NOT control them, only monitors. But honestly, that's perfect. It is so easy to use MeshCentral to hop on and manage the instances when they need something. The central system just tells us that all is well or not. So while Duplicati ITSELF doesn't do that, third party tools do and that's what we are doing now.
Also talking about building our own open source solution to do that for reasons of integration into other things (namely helpdesk) but that hasn't gone very far since the other solution is cheap and working.
-
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
How do you keep track of all the backups if you have hundreds of duplicati installations running? There are no central backup server or UI right? All installations are independent of each other?
Ah, excellent question. So there is no central server, that is correct. All installations are independent, also correct. But they are open source and have some APIs and messaging options. That's great when you have one or two. But with hundreds (like we will have once the conversion is done) you need something more.
Thankfully there is a non-open source Duplicati monitoring service that we are using that is cheap and does just that. It does NOT control them, only monitors. But honestly, that's perfect. It is so easy to use MeshCentral to hop on and manage the instances when they need something. The central system just tells us that all is well or not. So while Duplicati ITSELF doesn't do that, third party tools do and that's what we are doing now.
Also talking about building our own open source solution to do that for reasons of integration into other things (namely helpdesk) but that hasn't gone very far since the other solution is cheap and working.
Great, thanks for clarifying. I often find myself looking at a product amazed at how badly they manage to explain what it's for and how it works. But you summed it up nicely.
-
@scottalanmiller Let me clarify. I want to make sure the "good" backups are copied to the offsite storage. So if the building were to catch on fire or something, and the good copies are destroyed. I would want to be able to restore from the offsite storage. In my case, some of the data was missing from the offsite storage that should have been replicated from the local "good" backup. Not sure what happened, and why it was not copied over, but it did not. I figured there would be some kind of sync mechanism that would have caught that ahead of time, which Barracuda said there is no such sync. That is why I reached out to the community.
-
@Fredtx said in Does block level sync exist?:
@scottalanmiller Let me clarify. I want to make sure the "good" backups are copied to the offsite storage. So if the building were to catch on fire or something, and the good copies are destroyed. I would want to be able to restore from the offsite storage. In my case, some of the data was missing from the offsite storage that should have been replicated from the local "good" backup. Not sure what happened, and why it was not copied over, but it did not. I figured there would be some kind of sync mechanism that would have caught that ahead of time, which Barracuda said there is no such sync. That is why I reached out to the community.
We understand. And that's important because clearly your sync failed. It's just that it also exposed the fact that the original backups are not application aware (unless there is no application) so something that you should see as a very, very large issue. If you are responsible for the backups, that is. Otherwise, not your monkeys, not your circus.
-
@scottalanmiller said in Does block level sync exist?:
@Fredtx said in Does block level sync exist?:
@scottalanmiller Let me clarify. I want to make sure the "good" backups are copied to the offsite storage. So if the building were to catch on fire or something, and the good copies are destroyed. I would want to be able to restore from the offsite storage. In my case, some of the data was missing from the offsite storage that should have been replicated from the local "good" backup. Not sure what happened, and why it was not copied over, but it did not. I figured there would be some kind of sync mechanism that would have caught that ahead of time, which Barracuda said there is no such sync. That is why I reached out to the community.
We understand. And that's important because clearly your sync failed. It's just that it also exposed the fact that the original backups are not application aware (unless there is no application) so something that you should see as a very, very large issue. If you are responsible for the backups, that is. Otherwise, not your monkeys, not your circus.
You're making an assumption that there's an app to backup - which wasn't 100% clear until this post. As you mention - he might just be backing up file servers - so no app involved - just files to backup.
-
@Dashrender said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
@Fredtx said in Does block level sync exist?:
@scottalanmiller Let me clarify. I want to make sure the "good" backups are copied to the offsite storage. So if the building were to catch on fire or something, and the good copies are destroyed. I would want to be able to restore from the offsite storage. In my case, some of the data was missing from the offsite storage that should have been replicated from the local "good" backup. Not sure what happened, and why it was not copied over, but it did not. I figured there would be some kind of sync mechanism that would have caught that ahead of time, which Barracuda said there is no such sync. That is why I reached out to the community.
We understand. And that's important because clearly your sync failed. It's just that it also exposed the fact that the original backups are not application aware (unless there is no application) so something that you should see as a very, very large issue. If you are responsible for the backups, that is. Otherwise, not your monkeys, not your circus.
You're making an assumption that there's an app to backup - which wasn't 100% clear until this post. As you mention - he might just be backing up file servers - so no app involved - just files to backup.
Even a pure file server is normally accessed. "File server" is a form of "database". A very specific form, but surprisingly similar to a document database. It would be super weird, but not actually impossible, to have a file server that holds files but is never accessed. but once you start accessing files, it's an application doing the accessing and we are right back to where we started. File servers tend to have known usage patterns and accepted backup failure modes, but the issue hasn't changed. It just feels that way. No file exists without an application.
