Does block level sync exist?
-
@scottalanmiller said in Does block level sync exist?:
The issue you are having is that someone bought a bad product that doesn't work and doesn't have support. If you used Rsync, for example, you'd have that functionality. Robocopy does, too.
I'm definitely seeing how horrible this product is. At first, I was like hey this is pretty good. Now that I'm getting a more understanding of it, and working with their support I see it sucks. Heck, even today the cloud console that is used to check the status of all the backups was unresponsive. I reached out to support who was aware of the issue, and I asked for a technical answer once they resolve it, and here's what I got from the tech.
I did also reach out to the team for a state, the public statement and I was given the following statement, but I do not believe we have a concrete reason as of yet on why this occurred.
We were experiencing an internal issue which has now been resolved and we are taking steps to prevent issues like this from occurring moving forward.
Barracuda Support
I may look at something else such as Veeam Backup And Replication, but I would need to do further research to see how their backup product works in the back end.
-
@Fredtx said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Right, which is just a fancy way of saying it uses VSS. Everything does that, that's not considered application aware, because absolutely everything has that level of awareness - the agent that has the awareness is part of the OS. Barracuda isn't aware of any third party applications, including those that run on top of MS SQL.
Yea, I'm aware it uses VSS. I thought that's what you were referring to when talking about application awareness. But looks like you are referring to something else that I have a lack of knowledge or understanding on.
It's not that complicated. Imagine you are running your own desktop in a VM and you want to take a backup. What about the files you are working on and haven't saved yet? They only exist in RAM and not on any disk. So any backup that backups just your files or blocks on the disk will never be complete.
It's the same on a server. You don't know where the data is that you are trying to backup. Only the application developers knows how it works and where the data is.
VSS is a set of Windows components that communicate with applications so that the OS can tell the application when it need to prepare for a snapshot of the data by writing files to disk etc. But that only works IF the developers actually use the VSS components in their application and that is not always the case. But if they do, the backup should be good concerning that particular application. However there are many things running on a typical server.
-
@Pete-S Good description.
-
@Fredtx said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Right, which is just a fancy way of saying it uses VSS. Everything does that, that's not considered application aware, because absolutely everything has that level of awareness - the agent that has the awareness is part of the OS. Barracuda isn't aware of any third party applications, including those that run on top of MS SQL.
Yea, I'm aware it uses VSS. I thought that's what you were referring to when talking about application awareness. But looks like you are referring to something else that I have a lack of knowledge or understanding on.
VSS is application aware, just one only one or two applications that we rarely care very much about. It's not aware of YOUR applications, but it knows about SOME applications. In a world of hundreds of thousands of applications, it probably knows a few dozen and that's about it. Nearly all are MS applications like AD and Exchange. That's about it.
In all these years, I've never once run into any application outside of the stock MS apps that any vendor took the effort to be application aware of natively out of the box for a general purpose backup software. It's just impossible to address everyone (let alone anyone's) needs, so there's no point in pretending.
-
@Fredtx said in Does block level sync exist?:
I may look at something else such as Veeam Backup And Replication, but I would need to do further research to see how their backup product works in the back end.
That won't change the awareness issue. Do they have a sync option that meets your needs?
-
@scottalanmiller said in Does block level sync exist?:
That won't change the awareness issue. Do they have a sync option that meets your needs?
What I'm looking for is adhering to the Backup 3-2-1 best practice. The 3-2-1 strategy that Barracuda offers is a backup is copied to a local backup appliance, which is then deduplicated to save storage space on the appliance. Then, the data from the appliance is compressed and replicated to an offsite storage, which can be another appliance (our setup in my division) or their cloud storage, which I believe is in AWS.
-
@Fredtx said in Does block level sync exist?:
What I'm looking for is adhering to the Backup 3-2-1 best practice.
Well, the FIRST step in 3-2-1 is to get a reliable backup to do the 3-2-1 with. 3-2-1 itself is a really trivial part of the overall picture. That's just talking about where and how many copies you have. But it only matters if the copies you are making are good ones. You are worrying that the copies of the copies don't change, but not worrying if the original copy is any good.
-
@Fredtx said in Does block level sync exist?:
The 3-2-1 strategy that Barracuda offers is a backup is copied to a local backup appliance, which is then deduplicated to save storage space on the appliance. Then, the data from the appliance is compressed and replicated to an offsite storage, which can be another appliance (our setup in my division) or their cloud storage, which I believe is in AWS.
Just for reference, we do this using Duplicati. Rolling it out right now after testing. We've used it for years, but now making it our primary tool. It doesn't do the EXACT same process you are describing, but very similar. But single tool doing the 3-2-1 in an essentially similar way.
We are doing one local copy, one copy to NAS, a snapshot of the NAS, and an offsite to S2 or Wasabi.
-
But we are aware that Duplicati, like Veeam or Baraccuda, isn't application aware (of our applications) and that we have to accommodate for that and take a backup using application aware processes locally, then let Duplicati take a backup of THAT backup.
-
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
-
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
-
-
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
How do you keep track of all the backups if you have hundreds of duplicati installations running? There are no central backup server or UI right? All installations are independent of each other?
-
@scottalanmiller said in Does block level sync exist?:
You are worrying that the copies of the copies don't change, but not worrying if the original copy is any good.
My main concern is that the copies are good in all locations. In my case with Barracuda, the copies were not good. Actually, even worse, they were not even there.
-
@Fredtx said in Does block level sync exist?:
My main concern is that the copies are good in all locations
No your main concern is that your primary backup is good.
After that, it is just a copy operation. If you are worried about a copy operation, your solution is doing something wrong.
-
@Fredtx said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
You are worrying that the copies of the copies don't change, but not worrying if the original copy is any good.
My main concern is that the copies are good in all locations. In my case with Barracuda, the copies were not good. Actually, even worse, they were not even there.
Why do you care if you have copies if you don't know what they are copies of? Unless politically you are responsible for the copies but not the backup. Then, that makes sense. If they backup is bad, that's "Betty's fault", but if the sync of the backup is bad, that's your fault. That would make sense.
If that's not the case, you are looking at this all wrong. As a company, the backup is what matters and the sync is completely irrelevant until that is known to be good and once it is good, the sync can be pretty important, but never as important as the thing that is being sync.
It's similar to caring about how good the lock on your safe is but not caring that the back of the safe is missing and people can just walk by and steal everything without opening the door in the first place.
-
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
How do you keep track of all the backups if you have hundreds of duplicati installations running? There are no central backup server or UI right? All installations are independent of each other?
Ah, excellent question. So there is no central server, that is correct. All installations are independent, also correct. But they are open source and have some APIs and messaging options. That's great when you have one or two. But with hundreds (like we will have once the conversion is done) you need something more.
Thankfully there is a non-open source Duplicati monitoring service that we are using that is cheap and does just that. It does NOT control them, only monitors. But honestly, that's perfect. It is so easy to use MeshCentral to hop on and manage the instances when they need something. The central system just tells us that all is well or not. So while Duplicati ITSELF doesn't do that, third party tools do and that's what we are doing now.
Also talking about building our own open source solution to do that for reasons of integration into other things (namely helpdesk) but that hasn't gone very far since the other solution is cheap and working.
-
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
@Pete-S said in Does block level sync exist?:
@scottalanmiller said in Does block level sync exist?:
Duplicati
What is Duplicati? It sound like something you install locally on each server that will send backups to NAS/cloud/wherever. Is that correct?
Yup, it's backup software. Does both image and file based. Can send to local, remote, or cloud destinations. It's an agent, so it installs ON Windows, MacOS or Linux.
How do you keep track of all the backups if you have hundreds of duplicati installations running? There are no central backup server or UI right? All installations are independent of each other?
Ah, excellent question. So there is no central server, that is correct. All installations are independent, also correct. But they are open source and have some APIs and messaging options. That's great when you have one or two. But with hundreds (like we will have once the conversion is done) you need something more.
Thankfully there is a non-open source Duplicati monitoring service that we are using that is cheap and does just that. It does NOT control them, only monitors. But honestly, that's perfect. It is so easy to use MeshCentral to hop on and manage the instances when they need something. The central system just tells us that all is well or not. So while Duplicati ITSELF doesn't do that, third party tools do and that's what we are doing now.
Also talking about building our own open source solution to do that for reasons of integration into other things (namely helpdesk) but that hasn't gone very far since the other solution is cheap and working.
Great, thanks for clarifying. I often find myself looking at a product amazed at how badly they manage to explain what it's for and how it works. But you summed it up nicely.
-
@scottalanmiller Let me clarify. I want to make sure the "good" backups are copied to the offsite storage. So if the building were to catch on fire or something, and the good copies are destroyed. I would want to be able to restore from the offsite storage. In my case, some of the data was missing from the offsite storage that should have been replicated from the local "good" backup. Not sure what happened, and why it was not copied over, but it did not. I figured there would be some kind of sync mechanism that would have caught that ahead of time, which Barracuda said there is no such sync. That is why I reached out to the community.
-
@Fredtx said in Does block level sync exist?:
@scottalanmiller Let me clarify. I want to make sure the "good" backups are copied to the offsite storage. So if the building were to catch on fire or something, and the good copies are destroyed. I would want to be able to restore from the offsite storage. In my case, some of the data was missing from the offsite storage that should have been replicated from the local "good" backup. Not sure what happened, and why it was not copied over, but it did not. I figured there would be some kind of sync mechanism that would have caught that ahead of time, which Barracuda said there is no such sync. That is why I reached out to the community.
We understand. And that's important because clearly your sync failed. It's just that it also exposed the fact that the original backups are not application aware (unless there is no application) so something that you should see as a very, very large issue. If you are responsible for the backups, that is. Otherwise, not your monkeys, not your circus.
