Nextcloud Backup System Idea
-
@travisdh1 said in Nextcloud Backup System Idea:
But you don't have a 3rd copy currently with 1 of the 3 being on the same server.
Yeah. Since this lab server is just a 1U in co-location, the best I could do for having an additional copy off of the Nextcloud VM is having another VM as a backup target. Of course, there's no real protection with that.
Apparently I missed that you already have an offsite happening, so good on you.
Nothing offsite yet. I'm trying to think all of this through before I create the Nextcloud VM and start storing data on it.
Being that it's your home lab, is it really worth the expense of getting a real 3rd copy? Might not matter (value of the data compared to time to restore.)
That's something I'm weighing as well. The only way I think I could get a real 3rd copy is have one on a second cloud storage provider, since having additional hardware in colo really isn't an option. I suppose I could store a copy at home as the 3rd copy, but that's more of make-a-way-to-have-a-3rd-copy-just-to-have-a-third-copy rather than as you suggested before "do it right."
-
@EddieJennings said in Nextcloud Backup System Idea:
@travisdh1 said in Nextcloud Backup System Idea:
But you don't have a 3rd copy currently with 1 of the 3 being on the same server.
Yeah. Since this lab server is just a 1U in co-location, the best I could do for having an additional copy off of the Nextcloud VM is having another VM as a backup target. Of course, there's no real protection with that.
So the question to ask is why add the complexity of backing up to a VM that would reside on the same host? It's more complex for no gain.
Nothing offsite yet. I'm trying to think all of this through before I create the Nextcloud VM and start storing data on it.
This is good to do.
That's something I'm weighing as well. The only way I think I could get a real 3rd copy is have one on a second cloud storage provider, since having additional hardware in colo really isn't an option. I suppose I could store a copy at home as the 3rd copy, but that's more of make-a-way-to-have-a-3rd-copy-just-to-have-a-third-copy rather than as you suggested before "do it right."
Or you could setup a NextCloud instance on something like Vultr and replicate between your 1U and the actual remote VM.
-
@DustinB3403 and @travisdh1 are both missing the fucking point that while this may be a single 1u server in colo the point is to be a full lab design.
He absolutely should have a “backup” VM as this is what exists in reality.
It is replicating the real world on site copy.
-
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
-
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 and @travisdh1 are both missing the fucking point that while this may be a single 1u server in colo the point is to be a full lab design.
He absolutely should have a “backup” VM as this is what exists in reality.
It is replicating the real world on site copy.
This was my assumption as well - he's trying to mimic real life as best he can.
-
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 and @travisdh1 are both missing the fucking point that while this may be a single 1u server in colo the point is to be a full lab design.
He absolutely should have a “backup” VM as this is what exists in reality.
It is replicating the real world on site copy.
Jared is correct about my intent
-
@Dashrender said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 and @travisdh1 are both missing the fucking point that while this may be a single 1u server in colo the point is to be a full lab design.
He absolutely should have a “backup” VM as this is what exists in reality.
It is replicating the real world on site copy.
This was my assumption as well - he's trying to mimic real life as best he can.
It isn’t even an assumption. It was clearly stated.
-
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
-
@EddieJennings said in Nextcloud Backup System Idea:
@travisdh1 said in Backup System Idea:
@EddieJennings said in Backup System Idea:
@DustinB3403 said in Backup System Idea:
To ask and to be that guy;
Why does it sound as if NextCloud is going to be installed to the bare metal with the potential for a separate "Backup VM"?
As for the backup solution, you can use tools like Restic, Duplicati, UrBackup or really anything you want. Have that tool backup to a local storage media and then copy those to a cloud provider.
NextCloud isn't going to be installed on bare metal. I could just attach another qcow2 disk to the VM, mount it somewhere, and use that as the backup target.
The idea of the Backup VM is emulating something like what I might find in a production environment using something like Veeam, where you have a dedicated server (VM) that runs the backup software and is a target for the backed up data. Of course, this is overkill for what I'm doing (just backing up my NextCloud instance).
Why not send the backup to either Backblaze B2 or Wasabi? If you want practice doing backups, might as well do it right. My B2 is costing me ~$1.25/month
So skip having a 3rd copy of the data (live, copy on aforementioned Backup VM, copy on Backblaze storage or some other provider)?
Number of copies is never really a factor. It's the resulting durability that matters.
-
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
Again, missing the point. The Colo is replicating an on site deployment of VM server, on site primary backup and then offsite backup.
-
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
Again, missing the point. The Colo is replicating an on site deployment of VM server, on site primary backup and then offsite backup.
. . .
-
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch said in Nextcloud Backup System Idea:
@brandon220 said in Nextcloud Backup System Idea:
As important as backups are - I'm really surprised that NC has not integrated them into the software.
This thread is helpful as I was not aware of Restic. I normally make exports of the VM for NC backups.I am surprised that they have not partnered with some provider.
I wonder is someone could write a backup app that would backup the DB and up it to Backblaze
So mimic real fucking life and hostthe VM on a remote platform like I stated.
See my previous post for the details as to what I said.
Also happy new years
Again, missing the point. The Colo is replicating an on site deployment of VM server, on site primary backup and then offsite backup.
. . .
You can be as whiny bitch as you want. it doens't change the facts of the stated case.
-
@JaredBusch The stated case is, how can I create backups and do so in a fashion that would work.
While using the same Hypervisor and backing up to a separate VM on the same Hypervisor as the host does not create a backup.
Since it isn't detached, I'm sure if I look hard enough I can find several instances of you spouting this.
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
I also never stated to not take backups, I implied it was stupid to take backups to the same hypervisor that your production set is made from.
-
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch The stated case is, how can I create backups and do so in a fashion that would work.
While using the same Hypervisor and backing up to a separate VM on the same Hypervisor as the host does not create a backup.
Since it isn't detached, I'm sure if I look hard enough I can find several instances of you spouting this.
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
I also never stated to not take backups, I implied it was stupid to take backups to the same hypervisor that your production set is made from.
On the contrary, a lab is exactly the place to set up something like this and test processes. That is the entire fucking point of a lab.
Shoving off to a VPS somewhere is a totally different approach that would require different tools.
The tools used when staying within the same network are not the same. Could they be? Sometimes.
I don't know WTF crawled up your ass, but go take a shit and let it out.
-
@JaredBusch said in Nextcloud Backup System Idea:
@DustinB3403 said in Nextcloud Backup System Idea:
@JaredBusch The stated case is, how can I create backups and do so in a fashion that would work.
While using the same Hypervisor and backing up to a separate VM on the same Hypervisor as the host does not create a backup.
Since it isn't detached, I'm sure if I look hard enough I can find several instances of you spouting this.
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
I also never stated to not take backups, I implied it was stupid to take backups to the same hypervisor that your production set is made from.
On the contrary, a lab is exactly the place to set up something like this and test processes. That is the entire fucking point of a lab.
But you would never do this for a production workload. So consider the workload as any other and find the issues with the plan, punch holes in it, and realize that the plan may not hold water. A lab is a place to test process, but process needs to be considered before it is tested. One such issue with this process as discussed is if the Hypervisor or Host fails you risk everything.
Shoving off to a VPS somewhere is a totally different approach that would require different tools.
It's the approach that would likely be taken in cases like this where "I only have a 1U in a colo and am looking for ways to protect it".
The tools used when staying within the same network are not the same. Could they be? Sometimes.
I don't know WTF crawled up your ass, but go take a shit and let it out.
-
@DustinB3403 said in Nextcloud Backup System Idea:
Even for a lab, this is a bad approach to take and the better approach would be to setup a VPS and do your backups to that as the detached backup and from that push the backups to a cloud if the goal would be to follow the 3-2-1 rule.
Of course in a real environment it wouldn't be smart to backup to a VM that's on the same hypervisor as the VMs that are being backed up. The goal is practicing a process, which is have a target for backups that are then replicated to
$cloudStorage. The way to achieve this given the hardware I have is to backup to a VM (on the colocated server), then from that VM send copies of the data away.Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
-
@EddieJennings said in Nextcloud Backup System Idea:
Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
This is likely not a better approach. Because the major point of local backup is to have fast recovery.
Without fast recovery, there is less reason to have a local backup.
-
@EddieJennings said in Nextcloud Backup System Idea:
Thinking through RPO / RTO this is what I think is reasonable for this data. I could handle NextCloud not being available for 24 hours as well losing 24 hours worth of data. The initial amount of data that will be stored in NextCloud will be around 1.5 TB. There will likely be far less than an a gigabyte / day of new data created.
All of this implies this is not just a lab, at least not for the long term. look at setting up your systems in a manner which can be easily protected.
Rather than going through the process you stated in the OP.
-
@JaredBusch said in Nextcloud Backup System Idea:
@EddieJennings said in Nextcloud Backup System Idea:
Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
This is likely not a better approach. Because the major point of local backup is to have fast recovery.
Without fast recovery, there is less reason to have a local backup.
Quick is relative when he's stated he can be without this NC instance for 24 hours.
-
@JaredBusch said in Nextcloud Backup System Idea:
@EddieJennings said in Nextcloud Backup System Idea:
Another, and probably better approach, would be to just backup the data to two different cloud storage providers, or follow your suggestion to backup to a VPS and then push from there. The beauty of a lab is I can tear stuff down and try multiple approaches.
This is likely not a better approach. Because the major point of local backup is to have fast recovery.
Without fast recovery, there is less reason to have a local backup.
Exactly. As Scott said, the multiple copies of backups then is more about durability than anything else. Do you really not trust your first cloud provider that much?
Fast recovery would be the main goal I can think of for local backup.
