Storage and Data Locality
-
@scottalanmiller said in Storage and Data Locality:
@dafyre said in Storage and Data Locality:
Current latency is between 200 and 250ms over ZT.
That's a lot of latency, that'll make things hard. But an async replica is an option.
That's the term I was looking for. Async replica -- that's what I'm going for... In the event that my home network burns down, the goal is to be able to spin up anything I need in France.
-
@dbeato said in Storage and Data Locality:
If I can offer any input, please do not use DRBD, it can corrupt your data pretty easily.
Thanks for the forewarning -- that was one of the things I was considering.
-
I was thinking Veeam B&R but don't think you want to go to either Hyper-V or ESXi and dont have piles of cash just laying around, do you?
-
@nerdydad said in Storage and Data Locality:
I was thinking Veeam B&R but don't think you want to go to either Hyper-V or ESXi and dont have piles of cash just laying around, do you?
Yeah, this is a personal lab, so I don't have piles of cash lying around. Since my hosted server is limited in OS choices, I'm sticking with KVM. (ESXi and Hyper-V are not offered).
-
rsync?
-
I think it's just assumed that if your using KVM, that your also competent enough to write your own backup scripts. If you already have a backup script setup, just create a local backup and then rsync those files to the other box.
If you need a backup script still, well, I've been meaning to work on one, you just might be the motivation to get it done.
-
@travisdh1 said in Storage and Data Locality:
I think it's just assumed that if your using KVM, that your also competent enough to write your own backup scripts. If you already have a backup script setup, just create a local backup and then rsync those files to the other box.
This is just a steaming pile of shit.
Anyone can backup any system easy enough. KVM has nothing to do with it.
What you cannot do with a simple script is do differential or incremental backups such as Veeam and Unitrends do.
That is why backups are hard, and why there are not bazillions of backup solutions out there. -
@jaredbusch said in Storage and Data Locality:
@travisdh1 said in Storage and Data Locality:
I think it's just assumed that if your using KVM, that your also competent enough to write your own backup scripts. If you already have a backup script setup, just create a local backup and then rsync those files to the other box.
This is just a steaming pile of shit.
Anyone can backup any system easy enough. KVM has nothing to do with it.
What you cannot do with a simple script is do differential or incremental backups such as Veeam and Unitrends do.
That is why backups are hard, and why there are not bazillions of backup solutions out there.Also, usable automation. It's easy to automate a straight backup copy job; it's very hard to automate a system for restoring a specific file from a specific point in time.
-
I’ve used rsnapshot before and it works really well. Uses hard links to save space. Won’t help with your VM disks but will help with files.
-
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
-
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
-
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
-
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
Don't need it for DR. Then the question becomes, why did someone make that policy?
-
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
With stateless machines, wouldn't it be easier to automate the deployment, especially if your data is stored on another virtual disk or storage appliance?
-
@black3dynamite said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
With stateless machines, wouldn't it be easier to automate the deployment, especially if your data is stored on another virtual disk or storage appliance?
Exactly, that's the theory. Have a system like Salt, Ansible, or just a build script that can restore the box automatically and another script or command that can restore the data, rather than the system. This provides not just lower cost and faster backups, but faster and more flexible restores.
-
@scottalanmiller said in Storage and Data Locality:
@black3dynamite said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
With stateless machines, wouldn't it be easier to automate the deployment, especially if your data is stored on another virtual disk or storage appliance?
Exactly, that's the theory. Have a system like Salt, Ansible, or just a build script that can restore the box automatically and another script or command that can restore the data, rather than the system. This provides not just lower cost and faster backups, but faster and more flexible restores.
I keep forgetting how much easier state machines make backups for servers, keep beating it into my head, I promise I'll get reprogrammed soon.
-
@travisdh1 said in Storage and Data Locality:
@scottalanmiller said in Storage and Data Locality:
@black3dynamite said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
With stateless machines, wouldn't it be easier to automate the deployment, especially if your data is stored on another virtual disk or storage appliance?
Exactly, that's the theory. Have a system like Salt, Ansible, or just a build script that can restore the box automatically and another script or command that can restore the data, rather than the system. This provides not just lower cost and faster backups, but faster and more flexible restores.
I keep forgetting how much easier state machines make backups for servers, keep beating it into my head, I promise I'll get reprogrammed soon.
Takes a while to get used to thinking of them in that way.
-
@scottalanmiller said in Storage and Data Locality:
@black3dynamite said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
With stateless machines, wouldn't it be easier to automate the deployment, especially if your data is stored on another virtual disk or storage appliance?
Exactly, that's the theory. Have a system like Salt, Ansible, or just a build script that can restore the box automatically and another script or command that can restore the data, rather than the system. This provides not just lower cost and faster backups, but faster and more flexible restores.
Ya. I just back up GitLab. There’s a couple VMs with data but not many. All of our workstations are stateless. If there’s an issue I just kickstart again and Ansible does the work.
-
@scottalanmiller said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
Don't need it for DR. Then the question becomes, why did someone make that policy?
They want every physical and virtual server backed up at least weekly.
If a stateless server goes down, I don't have time to rebuild it. I am also not at a place to fully implement Salt to redeploy every damn server we have... we have 100+ servers and most of them are all different. That's a lot of work.
Yes, in an ideal environment it wouldn't be like that. But it takes time to get there, and there aren't enough of us to do it now, and there are many higher priority things going on atm. But it's getting closer.
There's also the issue of me being "the man behind the curtain" like the Wizard of OZ. It was brought up that implementing so many different things like SS, NC, this, that, and the other thing isn't desirable because of the hit by the bus issue...
So politics are another thing.
Anyways, ReaR works great for servers... I use it, and am glad I do. It's already proved itself!!
-
@scottalanmiller said in Storage and Data Locality:
@black3dynamite said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
@stacksofplates said in Storage and Data Locality:
@tim_g said in Storage and Data Locality:
ReaR is great as agent based backup for image based recovery and inceementals. For file based, I like FWBackups.
ReaR works great on stateless VMs, really easy and quick to recover.
Why are you backing up stateless machines?
Policy and DR.
With stateless machines, wouldn't it be easier to automate the deployment, especially if your data is stored on another virtual disk or storage appliance?
Exactly, that's the theory. Have a system like Salt, Ansible, or just a build script that can restore the box automatically and another script or command that can restore the data, rather than the system. This provides not just lower cost and faster backups, but faster and more flexible restores.
Yes you should have seen from my post hostory here that i fully understand and prefer that....
