XenServer Export Performance Seems Poor

Dashrender

The backup I'm making today is a point in time (if I don't have logging because of a fire, it's not the end of the world, as long as i have the data).

This drive contains PHI from another project we recently shutdown. So since there was plenty of space on it, I put a copy of this VM on there so the whole thing can go to the safety deposit box and be an offsite copy for now.

This isn't my main backup.

Also, It appears that my slow backup has more to do with GZip than USB 3.0.

I'll add another 1 TB drive to my machine and do a non compressed, non USB backup and we'll see what i get..

then I should do a non compressed backup to USB 3.0 and see what I get to see if the USB has any real baring here.. I doubt it does.

JaredBusch

@DustinB3403 said:

@JaredBusch And what should he restore from if the backup that he takes is corrupted and he has nothing else to restore from?

Say goodbye to this system which is critical? @JaredBusch did you even see how @Dashrender is making this backup? He's pulling it from XO's webconsole and saving it to a USB 3.0 disk.

This alone would take a long time. Even if the system is used only for historical purposes only, that's a huge amount of time to try and recover from the backup he's created today.

Assuming that there is nothing corrupted in the backup he's made. Or that nothing gets corrupted while he tries to restore it.

That it is writing to USB is not relevant here. That is not the bottleneck. A 700GB copy to a USB 3.0 is not that big of a deal.

The source of the problem has already been pointed out to be XS.

If @Dashrender was willing to kill the current job, he would get mush better speeds on his subsequent backup.

He is backing up using the designed tools in the designed method. Those tools are what is broke.

His backup media of choice is irrelevant unless it is actually the bottleneck (it isn't).

What if I backup to a NAS and the NAS shits during a restore? It is the same scenario.

Once he resolves the gzip issue, the limitation here will be wirespeed.

JaredBusch

@DustinB3403 said:

Assuming that there is nothing corrupted in the backup he's made. Or that nothing gets corrupted while he tries to restore it.

I wanted to highlight this point separately. You are using the wrong Hypervisor if you even have to think about this. Data does not just randomly corrupt when being accessed. If you regularly have this kind of experience, then you have been doing all kinds of shit wrong. Likely using cheap consumer hardware.

DustinB3403

@JaredBusch You've never had a large download become corrupted?

Dashrender

@JaredBusch said:

@DustinB3403 said:

Assuming that there is nothing corrupted in the backup he's made. Or that nothing gets corrupted while he tries to restore it.

I wanted to highlight this point separately. You are using the wrong Hypervisor if you even have to think about this. Data does not just randomly corrupt when being accessed. If you regularly have this kind of experience, then you have been doing all kinds of shit wrong. Likely using cheap consumer hardware.

yeah I was kinda thinking along the same lines. Though - corruption does happen, so having it on the table isn't a bad thing.

JaredBusch

@DustinB3403 said:

@JaredBusch You've never had a large download become corrupted?

Are you seriously comparing a download over the Internet to local LAN file copying? WTF?

JaredBusch

@Dashrender said:

@JaredBusch said:

@DustinB3403 said:

Assuming that there is nothing corrupted in the backup he's made. Or that nothing gets corrupted while he tries to restore it.

I wanted to highlight this point separately. You are using the wrong Hypervisor if you even have to think about this. Data does not just randomly corrupt when being accessed. If you regularly have this kind of experience, then you have been doing all kinds of shit wrong. Likely using cheap consumer hardware.

yeah I was kinda thinking along the same lines. Though - corruption does happen, so having it on the table isn't a bad thing.

Of course, anything can happen. That is the reason for the 3-2-1 mentality of backups. But you also have to address the likelihood of the scenario in question happening.

DustinB3403

@JaredBusch said:

@DustinB3403 said:

@JaredBusch You've never had a large download become corrupted?

Are you seriously comparing a download over the Internet to local LAN file copying? WTF?

No you're assuming that is what I'm comparing.

This is a large file to download and then upload, regardless of the LAN.

There could be any number of issues that occur, and the only way to ensure he has a good backup (the one he's created) is to upload it back into XenServer and see if it boots.

If it doesn't then there is an issue. Period.

But it has to be tested.

Dashrender

@DustinB3403 said:

There could be any number of issues that occur, and the only way to ensure he has a good backup (the one he's created) is to upload it back into XenServer and see if it boots.

If it doesn't then there is an issue. Period.

But it has to be tested.

Sadly, the problem with that is that I have no place to upload it to. all of my machines around here have 500 GB drives. I guess I'll have to get another 1 TB+ drive so I can do this.

scottalanmiller

Those are pretty cheap.

Dashrender

@scottalanmiller said:

Those are pretty cheap.

it's just the waiting...
and if not real waiting, well because there is a BB about 10 mins from here.. i don't want to go out in the snow.

Dashrender

Now I really want emoticons back

scottalanmiller

@Dashrender said:

@scottalanmiller said:

Those are pretty cheap.

it's just the waiting...
and if not real waiting, well because there is a BB about 10 mins from here.. i don't want to go out in the snow.

@Dashrender said:

@scottalanmiller said:

Those are pretty cheap.

it's just the waiting...
and if not real waiting, well because there is a BB about 10 mins from here.. i don't want to go out in the snow.

BB?