KVM in Production - Build it yourself
-
I think what @DustinB3403 feels bad is the mix of config and logic in the same file. If you would be able to separate the working logic from config, then edit a bunch of lines in a txt would not be so terrible .
Conceptually it should mot be so terrible:
1- snap
2- mount snap and backup location
3- run borg backup or similar
4- unmount everythingTesting is the most long part. With stuff like borg you should be confident in both backup and recovery.
Acutally recovery is trickier?! Don't know, should be like:
1- shutdown the machine
2- mount machine and backup location
3- restoreIssue is: how to programmatically tell borg which snap to use to recover...
I guess 2 of days of work... And I'm not super fast at writing code.
What has stopped me from using kvm in production was: what if I have to off load stuff to other people? Will they be
skilled enough for this crap?! So I just moved to hyperv+altaro... -
@matteo-nunziati said in KVM in Production - Build it yourself:
What has stopped me from using kvm in production was: what if I have to off load stuff to other people? Will they be
skilled enough for this crap?! So I just moved to hyperv+altaro...That's sad. I hope some user friendly solution will come out soon. I'm in the KVM boat since 2010, KVM is SO good today. Maybe it will get faster adoption after the AWS move.
-
@stacksofplates said in KVM in Production - Build it yourself:
I have 12 KVM hosts in production. For the very few machines that have stateful data, I use either the backup options built into the software/service that's running to a mounted location, my script, or I use ReaR. The rest of the systems are stored in Git.
Bare KVM is just fast and easy. I second that, but the issue there is no standard way to manage it, especially backups, so this leaves KVM a solution for person that knows much about, and usually the sole IT in that location, cause if there was other IT folks, they just pick ESXi or Hyper-V to ensure continuity.
-
@francesco-provino said in KVM in Production - Build it yourself:
Seems that Bacula has a KVM plugin: https://www.baculasystems.com/enterprise-backup-solution-with-bacula-systems/virtual-machine-backup-software . Any experience with that?
No experience but reading the online man suggests a lot of pita to setup bacula, while the kvm plugin seems quite abysmal. Not much more then the famous perl script or @stacksofplates script
-
Mine isn’t meant to be a full backup replacement. It’s a way to get full disk images once a week or so. I think most people that are using KVM are either using cloud infra or using state machines and agents when needed. I don’t think there is much of a demand for this type of backup solution.
However I firmly believe any org can use automation. It’s not just for large orgs. If I can do it for everything in my house, any size business can.
-
@stacksofplates said in KVM in Production - Build it yourself:
Mine isn’t meant to be a full backup replacement. It’s a way to get full disk images once a week or so. I think most people that are using KVM are either using cloud infra or using state machines and agents when needed. I don’t think there is much of a demand for this type of backup solution.
However I firmly believe any org can use automation. It’s not just for large orgs. If I can do it for everything in my house, any size business can.
So what do you do with your stateless data - like photos? video files, etc?
-
@dashrender said in KVM in Production - Build it yourself:
@stacksofplates said in KVM in Production - Build it yourself:
Mine isn’t meant to be a full backup replacement. It’s a way to get full disk images once a week or so. I think most people that are using KVM are either using cloud infra or using state machines and agents when needed. I don’t think there is much of a demand for this type of backup solution.
However I firmly believe any org can use automation. It’s not just for large orgs. If I can do it for everything in my house, any size business can.
So what do you do with your stateless data - like photos? video files, etc?
For home? Amazon. It’s automatically backed up there. I do have a bunch of music but that’s on one NFS VM that’s running CrashPlan. But like I said it’s one of the few that would require an agent.
-
I am starting to play with KVM after using XenServer for a while. While looking for a good KVM backup solution I found:
https://github.com/jagane/qemu-kvm-livebackup
https://www.linux-kvm.org/images/b/b6/2011-forum-LiveBackup.pdfI will be testing as soon as possible (probably in 1 week). Does anyone has any experience with it?
-
@dave_c said in KVM in Production - Build it yourself:
I am starting to play with KVM after using XenServer for a while. While looking for a good KVM backup solution I found:
https://github.com/jagane/qemu-kvm-livebackup
https://www.linux-kvm.org/images/b/b6/2011-forum-LiveBackup.pdfI will be testing as soon as possible (probably in 1 week). Does anyone has any experience with it?
The last commit was in 2011. Is it still usable?
-
Yes, I know. I am reading documentation and some of the source to determine if it is viable or not. So far it seems like some of the features it implements have been some how included in libvirt. Perhaps it would be a good challenge to bring this up to date.
The most important question is: Does anyone know a complete open source backup system for KVM? Like xen-orchestra for XS
I know about convirture and vprotect; seems like SEP has something but I am looking for open source (you know, because of reasons) -
@dave_c said in KVM in Production - Build it yourself:
Yes, I know. I am reading documentation and some of the source to determine if it is viable or not. So far it seems like some of the features it implements have been some how included in libvirt. Perhaps it would be a good challenge to bring this up to date.
The most important question is: Does anyone know a complete open source backup system for KVM? Like xen-orchestra for XS
I know about convirture and vprotect; seems like SEP has something but I am looking for open source (you know, because of reasons)Backups should be super easy. Take a snapshot -> copy the original disk image file -> remove snapshot, done.
-
@travisdh1 said in KVM in Production - Build it yourself:
@dave_c said in KVM in Production - Build it yourself:
Yes, I know. I am reading documentation and some of the source to determine if it is viable or not. So far it seems like some of the features it implements have been some how included in libvirt. Perhaps it would be a good challenge to bring this up to date.
The most important question is: Does anyone know a complete open source backup system for KVM? Like xen-orchestra for XS
I know about convirture and vprotect; seems like SEP has something but I am looking for open source (you know, because of reasons)Backups should be super easy. Take a snapshot -> copy the original disk image file -> remove snapshot, done.
Backups are totally not that that easy.
Sure they are is you have unlimited storage and unlimited bandwidth on 1 Gbps+ connections.
But in the real world, we need incrementals and/or differentials of those snaphots.
-
But in the real world, we need incrementals and/or differentials of those snaphots.
Exactly. And please, a super easy and fast way to recover, either a VM or a file.
I know that Veeam/Xen Orchestra and others make backups look really easy and I would like something like that for KVM.
So far, https://github.com/dguerri/LibVirtKvm-scripts looks great on paper. -
Has anyone ever used ShadowProtect SPX with KVM?
-
@black3dynamite
Interesting question. Since Veeam added agents I completely forgot about ShadowProtect.
in my case, I am looking for Hypervisor based backups; otherwise Veeam and even Acronis work well for agent based backups. -
@dave_c said in KVM in Production - Build it yourself:
@black3dynamite
Interesting question. Since Veeam added agents I completely forgot about ShadowProtect.
in my case, I am looking for Hypervisor based backups; otherwise Veeam and even Acronis work well for agent based backups.I am in this boat. I want hypervisor backups. I have application backups and shit. But those are rarely as easy as restoring the entire hypervisor.
State controlled systems need not apply. This is regarding real world SMB workloads.
-
Building a template VM with your backup agent installed might work (urbackup etc) as they'd register to your server.
Buy individual agents is again more than anyone really wants to manage.
-
@dustinb3403 said in KVM in Production - Build it yourself:
Building a template VM with your backup agent installed might work (urbackup etc) as they'd register to your server.
Buy individual agents is again more than anyone really wants to manage.
I think we have two KVM backup discussions going on, so I'll post a link to my other reply here:
