Nextcloud Backup System Idea
-
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 and @travisdh1 are both missing the fucking point that while this may be a single 1u server in colo the point is to be a full lab design.
He absolutely should have a “backup” VM as this is what exists in reality.
It is replicating the real world on site copy.
Jared is correct about my intent
-
@Dashrender said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 and @travisdh1 are both missing the fucking point that while this may be a single 1u server in colo the point is to be a full lab design.
He absolutely should have a “backup” VM as this is what exists in reality.
It is replicating the real world on site copy.
This was my assumption as well - he's trying to mimic real life as best he can.
It isn’t even an assumption. It was clearly stated.
-
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
-
@EddieJennings said in Nextcloud Backup System Idea:
@travisdh1 said in Backup System Idea:
@EddieJennings said in Backup System Idea:
@DustinB3403 said in Backup System Idea:
To ask and to be that guy;
Why does it sound as if NextCloud is going to be installed to the bare metal with the potential for a separate "Backup VM"?
As for the backup solution, you can use tools like Restic, Duplicati, UrBackup or really anything you want. Have that tool backup to a local storage media and then copy those to a cloud provider.
NextCloud isn't going to be installed on bare metal. I could just attach another qcow2 disk to the VM, mount it somewhere, and use that as the backup target.
The idea of the Backup VM is emulating something like what I might find in a production environment using something like Veeam, where you have a dedicated server (VM) that runs the backup software and is a target for the backed up data. Of course, this is overkill for what I'm doing (just backing up my NextCloud instance).
Why not send the backup to either Backblaze B2 or Wasabi? If you want practice doing backups, might as well do it right. My B2 is costing me ~$1.25/month
So skip having a 3rd copy of the data (live, copy on aforementioned Backup VM, copy on Backblaze storage or some other provider)?
Number of copies is never really a factor. It's the resulting durability that matters.
-
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
Again, missing the point. The Colo is replicating an on site deployment of VM server, on site primary backup and then offsite backup.
-
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
Again, missing the point. The Colo is replicating an on site deployment of VM server, on site primary backup and then offsite backup.
. . .
-
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
Again, missing the point. The Colo is replicating an on site deployment of VM server, on site primary backup and then offsite backup.
. . .
You can be as whiny bitch as you want. it doens't change the facts of the stated case.
-
@JaredBusch The stated case is, how can I create backups and do so in a fashion that would work.
While using the same Hypervisor and backing up to a separate VM on the same Hypervisor as the host does not create a backup.
Since it isn't detached, I'm sure if I look hard enough I can find several instances of you spouting this.
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
I also never stated to not take backups, I implied it was stupid to take backups to the same hypervisor that your production set is made from.
-
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch The stated case is, how can I create backups and do so in a fashion that would work.
While using the same Hypervisor and backing up to a separate VM on the same Hypervisor as the host does not create a backup.
Since it isn't detached, I'm sure if I look hard enough I can find several instances of you spouting this.
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
I also never stated to not take backups, I implied it was stupid to take backups to the same hypervisor that your production set is made from.
On the contrary, a lab is exactly the place to set up something like this and test processes. That is the entire fucking point of a lab.
Shoving off to a VPS somewhere is a totally different approach that would require different tools.
The tools used when staying within the same network are not the same. Could they be? Sometimes.
I don't know WTF crawled up your ass, but go take a shit and let it out.
-
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch The stated case is, how can I create backups and do so in a fashion that would work.
While using the same Hypervisor and backing up to a separate VM on the same Hypervisor as the host does not create a backup.
Since it isn't detached, I'm sure if I look hard enough I can find several instances of you spouting this.
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
I also never stated to not take backups, I implied it was stupid to take backups to the same hypervisor that your production set is made from.
On the contrary, a lab is exactly the place to set up something like this and test processes. That is the entire fucking point of a lab.
But you would never do this for a production workload. So consider the workload as any other and find the issues with the plan, punch holes in it, and realize that the plan may not hold water. A lab is a place to test process, but process needs to be considered before it is tested. One such issue with this process as discussed is if the Hypervisor or Host fails you risk everything.
Shoving off to a VPS somewhere is a totally different approach that would require different tools.
It's the approach that would likely be taken in cases like this where "I only have a 1U in a colo and am looking for ways to protect it".
The tools used when staying within the same network are not the same. Could they be? Sometimes.
I don't know WTF crawled up your ass, but go take a shit and let it out.
-
@DustinB3403 said in Nextcloud Backup System Idea:
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
Of course in a real environment it wouldn't be smart to backup to a VM that's on the same hypervisor as the VMs that are being backed up. The goal is practicing a process, which is have a target for backups that are then replicated to
$cloudStorage. The way to achieve this given the hardware I have is to backup to a VM (on the colocated server), then from that VM send copies of the data away.Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
-
@EddieJennings said in Nextcloud Backup System Idea:
Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
This is likely not a better approach. Because the major point of local backup is to have fast recovery.
Without fast recovery, there is less reason to have a local backup.
-
@EddieJennings said in Nextcloud Backup System Idea:
Thinking through RPO / RTO this is what I think is reasonable for this data. I could handle NextCloud not being available for 24 hours as well losing 24 hours worth of data. The initial amount of data that will be stored in NextCloud will be around 1.5 TB. There will likely be far less than an a gigabyte / day of new data created.
All of this implies this is not just a lab, at least not for the long term. look at setting up your systems in a manner which can be easily protected.
Rather than going through the process you stated in the OP.
-
@JaredBusch said in Nextcloud Backup System Idea:
@EddieJennings said in Nextcloud Backup System Idea:
Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
This is likely not a better approach. Because the major point of local backup is to have fast recovery.
Without fast recovery, there is less reason to have a local backup.
Quick is relative when he's stated he can be without this NC instance for 24 hours.
-
@JaredBusch said in Nextcloud Backup System Idea:
@EddieJennings said in Nextcloud Backup System Idea:
Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
This is likely not a better approach. Because the major point of local backup is to have fast recovery.
Without fast recovery, there is less reason to have a local backup.
Exactly. As Scott said, the multiple copies of backups then is more about durability than anything else. Do you really not trust your first cloud provider that much?
Fast recovery would be the main goal I can think of for local backup.
-
@DustinB3403 said in Nextcloud Backup System Idea:
But you would never do this for a production workload. So consider the workload as any other and find the issues with the plan, punch holes in it, and realize that the plan may not hold water. A lab is a place to test process, but process needs to be considered before it is tested. One such issue with this process as discussed is if the Hypervisor or Host fails you risk everything.
You're correct in that risk is the problem with this idea. Part of the point of the thread is to get some other ideas on how to do what I'm wanting -- which you've provided. Backing up to the VM would serve a purpose as I've mentioned. It let me emulate how to do that process, with the exception that the target VM is on the same hypervisor rather than a different hypervisor. What I can take away is that I can design / do the task, and if I'm ever in an environment where I have more than one device with which to work, I can do the same process, but have the backup target on a different device, which is how it should be done.
-
@EddieJennings said in Nextcloud Backup System Idea:
The goal is practicing a process, which is have a target for backups that are then replicated to
$cloudStorage. The way to achieve this given the hardware I have is to backup to a VM (on the colocated server), then from that VM send copies of the data away.Practicing process also requires that you practice developing process and having holes punched into said idea.
-
@DustinB3403 said in Nextcloud Backup System Idea:
@EddieJennings said in Nextcloud Backup System Idea:
The goal is practicing a process, which is have a target for backups that are then replicated to
$cloudStorage. The way to achieve this given the hardware I have is to backup to a VM (on the colocated server), then from that VM send copies of the data away.Practicing process also requires that you practice developing process and having holes punched into said idea.
Only helpful when said holes are valid to the target practice being developed. WHich yours are not.
-
@DustinB3403 said in Nextcloud Backup System Idea:
@EddieJennings said in Nextcloud Backup System Idea:
Thinking through RPO / RTO this is what I think is reasonable for this data. I could handle NextCloud not being available for 24 hours as well losing 24 hours worth of data. The initial amount of data that will be stored in NextCloud will be around 1.5 TB. There will likely be far less than an a gigabyte / day of new data created.
All of this implies this is not just a lab, at least not for the long term. look at setting up your systems in a manner which can be easily protected.
Rather than going through the process you stated in the OP.
It can be tough to fully describe intent in the OP, when the point is discussion. I figured, even for just a lab, it would've been wise to decide an RPO/RTO goal. Rather than just picking some random time for that, I thought through the idea of "if this were production, I could likely handle . . . "
-
@EddieJennings said in Nextcloud Backup System Idea:
@DustinB3403 said in Nextcloud Backup System Idea:
But you would never do this for a production workload. So consider the workload as any other and find the issues with the plan, punch holes in it, and realize that the plan may not hold water. A lab is a place to test process, but process needs to be considered before it is tested. One such issue with this process as discussed is if the Hypervisor or Host fails you risk everything.
You're correct in that risk is the problem with this idea. Part of the point of the thread is to get some other ideas on how to do what I'm wanting -- which you've provided. Backing up to the VM would serve a purpose as I've mentioned. It let me emulate how to do that process, with the exception that the target VM is on the same hypervisor rather than a different hypervisor. What I can take away is that I can design / do the task, and if I'm ever in an environment where I have more than one device with which to work, I can do the same process, but have the backup target on a different device, which is how it should be done.
But you already have the ability to access as many additional hosts as you want. You could even backup to your on-premise desktop or a VPS.
I agree that testing the click-this, input that steps are worthwhile. I disagree in the approach you're looking to take with what has been stated in the OP and hence.
