KVM Snapshot/Backup Script



  • So I posted a snapshot export script on here a long time ago, and it wasn't really correct. It created an overlay disk of a backing store and exported the backing store. Red Hat doesn't include the most up to date version of QEMU, so they can push RHEV. RHEL/CentOS 7 only comes with QEMU 1.5, and for live snapshots you need at least 2.0. So if you ad the oVirt repos you can get the newest version with the package qemu-kvm-rhev, or use Fedora server. Here's a script I wrote to take a live external snapshot and compress it to a location. I used my normal template with the interactive function. vmLocation is the location of your VMs. I usually keep mine in the same area. I should add full paths for everything, but I'm lazy this evening. I'll add them later. I also need to add some error handling but this is a rough start if anyone needs to use it.

    #!/bin/bash
    
    
    vmLocation="/data/VMs"
    
    today=$(date +%m-%d-%Y)
    
    
    
    #Script functions
    function script_help () {
      echo "
          Usage: $(basename $0) [options] dom-name snap-name save-location disk-name
    
              -i   Ineractive mode
    
              -h   this help text
    
              dom-name        Name of domain to take snapshot of
    
              snap-name       Name of snapshot
    
              save-location   location to copy snapshot to
    
              disk-name       Name of disk in VM
    
          Example:
            $(basename $0) bind-server bind-snap01 /export/snaps/ vda"
    
      exit ${1:-0}
    }
    
    function interactive_snap () {
    
      echo "Domain name"
      read name
    
      echo "Snapshot name"
      read snap
    
      echo "Location to save snap"
      read location
    
      echo "Disk name"
      read disk
    
      virsh dumpxml $name > $location/$name.xml
    
      diskPath=$(virsh domblklist $name | grep -i $disk | awk '{ print $2 }')
    
      virsh snapshot-create-as --domain $name $snap --diskspec $disk,file=$vmLocation/$snap.qcow2 --disk-only --atomic
    
      tar -czvf $location/$snap-$today.tar.gz $diskPath
    
      virsh blockcommit $name $disk --active --verbose --pivot
    
      rm -f $vmLocation/$snap.qcow2
    
      virsh snapshot-delete --metadata $name $snap
    
      exit ${1:-0}
    
    }
    
    function argument_snap () {
    
      virsh dumpxml $name > $location/$name.xml
    
      diskPath=$(virsh domblklist $name | grep -i $disk | awk '{ print $2 }')
    
      virsh snapshot-create-as --domain $name $snap --diskspec $disk,file=$vmLocation/$snap.qcow2 --disk-only --atomic
    
      tar -czvf $location/$snap-$today.tar.gz $diskPath
    
      virsh blockcommit $name $disk --active --verbose --pivot
    
      rm -f $vmLocation/$snap.qcow2
    
      virsh snapshot-delete --metadata $name $snap
    
      exit ${1:-0}
    
    
    
    }
    
    #Show help if no arguments or options are passed
    [[ ! "$*" ]] && script_help 1
    OPTIND=1
    
    
    #Read command line options
    while getopts "ih" opt; do
        case "$opt" in
          i) interactive_snap ;;
          h) script_help ;;
          \?) script_help 1 ;;
        esac
    done
    shift $(($OPTIND-1));
    
    #Run argument function
    name=$1
    snap=$2
    location=$3
    disk=$4
    argument_snap
    




  • @stacksofplates this basically is a super fast clone of the original vm using snapshots?



  • I haven't used external snapshots for anything, but now that I am reading about them I should be using them more.



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates this basically is a super fast clone of the original vm using snapshots?

    It takes a snapshot, which directs writes to the new file. Then tars and gzips the backing store (original disk image) to wherever you put in for the location, and then merges the snapshot back into the backing store.



  • @Romo said in KVM Snapshot/Backup Script:

    I haven't used external snapshots for anything, but now that I am reading about them I should be using them more.

    I still use internal for things like updates and large changes, but the external ones are nice.



  • @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates this basically is a super fast clone of the original vm using snapshots?

    It takes a snapshot, which directs writes to the new file. Then tars and gzips the backing store (original disk image) to wherever you put in for the location, and then merges the snapshot back into the backing store.

    This is too keep the system live during the backup?



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates this basically is a super fast clone of the original vm using snapshots?

    It takes a snapshot, which directs writes to the new file. Then tars and gzips the backing store (original disk image) to wherever you put in for the location, and then merges the snapshot back into the backing store.

    This is too keep the system live during the backup?

    Right. If you don't do it this way, you have to either shut the VM down or suspend it.



  • Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?



  • @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that. However, it does make a nice VDI tool. If you have an image and do qemu-img create -b and use it as the backing file, the overlays spin up really quickly. Like less than a second. But you can't write to the backing file as long as there are overlay files reading from it.



  • I create the template, and run virt-sysprep on it. Then I can update the disk with virt-sysprep --update. It automatically spins up a temp VM that updates all of the packages in the disk. But if you do this, you need to run virt-sysprep --selinux-relabel so it relabels the disk on the next clone. If not, labels can get screwed up;



  • @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?

    No, qcow2 is thin by default. But all of my templates are RHEL systems. So the OS doesn't use hardly any space. Are you cloning Windows machines?



  • @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?

    No, qcow2 is thin by default. But all of my templates are RHEL systems. So the OS doesn't use hardly any space. Are you cloning Windows machines?

    No, no windows



  • @stacksofplates said in KVM Snapshot/Backup Script:

    I create the template, and run virt-sysprep on it. Then I can update the disk with virt-sysprep --update. It automatically spins up a temp VM that updates all of the packages in the disk. But if you do this, you need to run virt-sysprep --selinux-relabel so it relabels the disk on the next clone. If not, labels can get screwed up;

    Is this is only available for RHEL guests or can it be used with other distros?



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?

    No, qcow2 is thin by default. But all of my templates are RHEL systems. So the OS doesn't use hardly any space. Are you cloning Windows machines?

    No, no windows

    Hmm, I don't have anything special. Some 300G 10K SAS drives in RAID 10.

    Here's a video I did for Dash:
    Youtube Video

    And another of a script I wrote that names the VM and spins up how many instances you tell it:
    Youtube Video



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    I create the template, and run virt-sysprep on it. Then I can update the disk with virt-sysprep --update. It automatically spins up a temp VM that updates all of the packages in the disk. But if you do this, you need to run virt-sysprep --selinux-relabel so it relabels the disk on the next clone. If not, labels can get screwed up;

    Is this is only available for RHEL guests or can it be used with other distros?

    You should be able to sysprep Ubuntu also. Not sure about any others.



  • @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?

    No, qcow2 is thin by default. But all of my templates are RHEL systems. So the OS doesn't use hardly any space. Are you cloning Windows machines?

    No, no windows

    Hmm, I don't have anything special. Some 300G 10K SAS drives in RAID 10.

    Here's a video I did for Dash:
    Youtube Video

    And another of a script I wrote that names the VM and spins up how many instances you tell it:
    Youtube Video

    I want that speed!! I am on 4 500GB 7200 SATA in RAID 10



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?

    No, qcow2 is thin by default. But all of my templates are RHEL systems. So the OS doesn't use hardly any space. Are you cloning Windows machines?

    No, no windows

    Hmm, I don't have anything special. Some 300G 10K SAS drives in RAID 10.

    Here's a video I did for Dash:
    Youtube Video

    And another of a script I wrote that names the VM and spins up how many instances you tell it:
    Youtube Video

    I want that speed!! I am on 4 500GB 7200 SATA in RAID 10

    How big is your template?

    2.0G -rw-------. 1 root root  16G Feb 13 03:37 template.qcow2
    

    That's what I have.



  • Also it's not virt-sysprep --update it's virt-customize --update. I didn't think that was right, so I just went back and looked.

    I have a daily cron job that runs this

    /bin/virt-customize --update --selinux-relabel -a /data/VMs/template.qcow2


  • @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?

    No, qcow2 is thin by default. But all of my templates are RHEL systems. So the OS doesn't use hardly any space. Are you cloning Windows machines?

    No, no windows

    Hmm, I don't have anything special. Some 300G 10K SAS drives in RAID 10.

    Here's a video I did for Dash:
    Youtube Video

    And another of a script I wrote that names the VM and spins up how many instances you tell it:
    Youtube Video

    I want that speed!! I am on 4 500GB 7200 SATA in RAID 10

    How big is your template?

    2.0G -rw-------. 1 root root  16G Feb 13 03:37 template.qcow2
    

    That's what I have.

    Way bigger, apparently its not thing provisioned at all

    1 root root 2.9G Oct 26 17:39 centos7-clone.qcow2
    1 root root  26G Feb  8 15:35 centos-7.qcow2
    


  • I use this to create my image and the use virt-manager to finish the install

    qemu-img create -f qcow2 -o preallocation=metadata centos-7.qcow2 25G
    


  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    Are you using external snapshots to thin provision any vms? Is there a performance hit on doing this?

    No. I have a template that uses a qcow2 disk. It's only a 15GB disk, but since it's thin provisioned it's only around 1.5GB. I can clone it in about 1-2 seconds so I haven't bothered with doing externals for that.

    My clones take 30-40 seconds, how do you thin provision? Using virt-sparsify on an image?

    No, qcow2 is thin by default. But all of my templates are RHEL systems. So the OS doesn't use hardly any space. Are you cloning Windows machines?

    No, no windows

    Hmm, I don't have anything special. Some 300G 10K SAS drives in RAID 10.

    Here's a video I did for Dash:
    Youtube Video

    And another of a script I wrote that names the VM and spins up how many instances you tell it:
    Youtube Video

    I want that speed!! I am on 4 500GB 7200 SATA in RAID 10

    How big is your template?

    2.0G -rw-------. 1 root root  16G Feb 13 03:37 template.qcow2
    

    That's what I have.

    Way bigger, apparently its not thing provisioned at all

    1 root root 2.9G Oct 26 17:39 centos7-clone.qcow2
    1 root root  26G Feb  8 15:35 centos-7.qcow2
    

    was that with ls -lsh? It should give you the actual size on the left before the permissions.

    I use a minimal image by default, then just add what I need after the clone.



  • @stacksofplates I only did a ls -lh

    output of ls -lsh

    2.9G -rw-r--r--. 1 root root 2.9G Oct 26 17:39 centos7-clone.qcow2
    1.1G -rw-r--r--. 1 root root  26G Feb  8 15:35 centos-7.qcow2
    


  • @Romo said in KVM Snapshot/Backup Script:

    I use this to create my image and the use virt-manager to finish the install

    qemu-img create -f qcow2 -o preallocation=metadata centos-7.qcow2 25G
    

    I preallocated the original template, and then when I clone with Virt-Manager or cli I don't usually change it after. I did some tests and didn't see any difference between running the preallocation on the clone and not. I'm not sure if it copies the preallocation flag when you clone, but like I said, I haven't seen much of a read/write difference.



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates I only did a ls -lh

    output of ls -lsh

    2.9G -rw-r--r--. 1 root root 2.9G Oct 26 17:39 centos7-clone.qcow2
    1.1G -rw-r--r--. 1 root root  26G Feb  8 15:35 centos-7.qcow2
    

    Ya so it's thin provisioned. I wonder why it's taking so long. I don't think the disk speeds would make that much of a difference.



  • What's your host specs? Mine is a DL380 G6. Dual 4 core Xeons and 96GB RAM. Don't think the RAM would have much to do with it. I had 24 originally and I'm pretty sure it cloned at the same speed.



  • @stacksofplates said in KVM Snapshot/Backup Script:

    What's your host specs? Mine is a DL380 G6. Dual 4 core Xeons and 96GB RAM. Don't think the RAM would have much to do with it. I had 24 originally and I'm pretty sure it cloned at the same speed.

    Its tiny 🙂

    ML110 G7 8GB RAM , Single 4 core Xeon



  • @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    What's your host specs? Mine is a DL380 G6. Dual 4 core Xeons and 96GB RAM. Don't think the RAM would have much to do with it. I had 24 originally and I'm pretty sure it cloned at the same speed.

    Its tiny 🙂

    ML110 G7 8GB RAM , Single 4 core Xeon

    Hmm. Do you have anything else running while you clone? You would think 4 cores would be enough as long as you're not way over provisioned.



  • @stacksofplates said in KVM Snapshot/Backup Script:

    @Romo said in KVM Snapshot/Backup Script:

    @stacksofplates said in KVM Snapshot/Backup Script:

    What's your host specs? Mine is a DL380 G6. Dual 4 core Xeons and 96GB RAM. Don't think the RAM would have much to do with it. I had 24 originally and I'm pretty sure it cloned at the same speed.

    Its tiny 🙂

    ML110 G7 8GB RAM , Single 4 core Xeon

    Hmm. Do you have anything else running while you clone? You would think 4 cores would be enough as long as you're not way over provisioned.

    3 vms

    virsh # list 
     Id    Name                           State
    ----------------------------------------------------
     111   FreePBX                        running
     144   rocket-chat                    running
     160   ubt-ans-ininja                 running
    

    This is a clone on the centos image.

    [[email protected] ~]# virt-clone -o centos-7 -n clone-test -f /vmrepo/clone-test.qcow2
    Allocating 'clone-test.qcow2'                                                  |  25 GB  00:00:33     
    
    Clone 'clone-test' created successfully.