Cloning XenServer on USB or SD

BRRABill

It's a straight copy, block by block. It doesn't care what size the drive is. It just writes to it identically. It ignores that the device might be larger. It will clone to smaller too and just fails when it fills up too quickly.

So I did this today, but ran into a problem.

The 64GB USB stick is reporting as a 32GB stick, just like the original 32GB.

Like there isn't even 30GB of unused partitions, it thinks the 64GB stick is only 32GB.

I also tried the program @DustinB3403 mentioned in his writeup, but that program even warns this is going to be a problem.
"Warning: Due to the forensic nature of image duplication by ImageUSB, please ensure that you select UFDs with a storage size similar to the image you wish to duplicate. For example, if a 2GB image is copied to an 8GB USB Flash Drive, the drive will only be able to use two out of the eight gigabytes of storage space. In this scenario, users will need to reformat the UFD in order to access the rest of the storage space."

So is there really any way to do what I am trying to do?

coliver

http://gparted.org/livecd.php

This will allow you to extend partitions.

DustinB3403

You would still be able to make changes to the FS on the USB once its completed, but as I said, changing a live partition isn't a trivial task.

With XenServer the installation creates two partitions, of which you should be able to easily adjust the size of the non-active partition. But the boot partition, that is a whole other issue.

It's possible, but likely not worth the effort.

BRRABill

@DustinB3403 said

You would still be able to make changes to the FS on the USB once its completed, but as I said, changing a live partition isn't a trivial task.

With XenServer the installation creates two partitions, of which you should be able to easily adjust the size of the non-active partition. But the boot partition, that is a whole other issue.

It's possible, but likely not worth the effort.

So really the only way to go from the "old" partitioning scheme of 6.5 on my 32GB drive to the "new" scheme on a 64GB drive would be to move off all the VMs, and reinstall XS7 from scratch, then move them back. Is that what I am picking up here?

DustinB3403

You could do an inline upgrade, but the upgrade doesn't even change the partition sizes. Because it's far easier to break the system trying to resize partitions than it is to just reinstall.

So yes, a clean install is recommended.

BRRABill

@DustinB3403 said

So yes, a clean install is recommended.

Hear ya loud and clear.

Hopefully XS will allow me to transfer both directions for migrations, then none of this will be an issue.

Or, I'll maybe just buy a refurb server from xByte for this purpose.

RamblingBiped

This is the command I usually use to clone one drive to another identical drive.

 dd if=/dev/sdX of=/dev/sdY bs=512 conv=noerror,sync

It uses the default block size to make a direct copy and keeps any errors from interrupting the process.

I've had problems come from changing to a larger blocksize before, so I usually just stick with the default of 512. If you are going to be moving between disks/partitions of different sizes you'll just need to start with a smaller image and make sure you resize your partitions and fix the filesystems after cloning. If you have a standard image you are working from you can test this out and find the largest block size that is stable and works for your specific task.

I wrote a small program in BASH that did a quick disk test/verification and automated the process of writing a standard image onto sd-cards of varying sizes (2GB up to 32GB) using dd and fdisk. It ended up replacing some old piece of software that ran on Windows and could not properly detect the newer larger capacity sd cards that our client wanted to start using.

BRRABill

Strangely enough ImageUSB did it properly.

But I'm not sure it's worth the risk of trying what I want to do.

I'll get everything upgraded to XS7 and then start clean installing.

BRRABill

@scottalanmiller said

Assuming the devices refer to the two USB devices:
dd if=dev/sda1 of=/dev/sdb1

If you are doing the entire disk (such as with XS which creates multiple partitions) why wouldn't it be...

dd if=/dev/sda of=/dev/sdb

scottalanmiller

@BRRABill said in Cloning XenServer on USB or SD:

@scottalanmiller said
Assuming the devices refer to the two USB devices:
dd if=dev/sda1 of=/dev/sdb1
If you are doing the entire disk (such as with XS which creates multiple partitions) why wouldn't it be...
dd if=/dev/sda of=/dev/sdb

You have to copy what there is to where you want it. If the filesystem is on /dev/sda1 you don't want /dev/sda or you are copying the partitioning layer with it.

BRRABill

@scottalanmiller said

You have to copy what there is to where you want it. If the filesystem is on /dev/sda1 you don't want /dev/sda or you are copying the partitioning layer with it.

But if I want to make an exact clone of the USB for DR purposes, I'd want the partitioning layer, no?

scottalanmiller

@BRRABill said in Cloning XenServer on USB or SD:

@scottalanmiller said

You have to copy what there is to where you want it. If the filesystem is on /dev/sda1 you don't want /dev/sda or you are copying the partitioning layer with it.

But if I want to make an exact clone of the USB for DR purposes, I'd want the partitioning layer, no?

Never tried that. Normally we partition manually. Maybe someone has tried this to know about reliability.

BRRABill

@scottalanmiller said in Cloning XenServer on USB or SD:

@DustinB3403 said in Cloning XenServer on USB or SD:

@scottalanmiller said in Cloning XenServer on USB or SD:

The standard tool for cloning is the dd command. No third party tools needed.

Can you provide a sample step by step, as I've not used DD on linux before and the documentation I've seen has been "sloppy" IMO.

Assuming the devices refer to the two USB devices:
dd if=dev/sda1 of=/dev/sdb1

So the poster asked for an example to clone the USB boot device, and this is what was given.

Are you saying you'd dd each of the partitions individually?

I am now confused.

scottalanmiller

I assume that there is only one partition. You can check yours but there should not be multiple partitions.

BRRABill

@scottalanmiller said in Cloning XenServer on USB or SD:

I assume that there is only one partition. You can check yours but there should not be multiple partitions.

This is how XS7 sets it up, at least...

sdb                                                                 8:16   0  74.5G  0 disk
├─sdb1                                                              8:17   0    18G  0 part /
├─sdb2                                                              8:18   0    18G  0 part
├─sdb3                                                              8:19   0   512M  0 part
├─sdb5                                                              8:21   0     4G  0 part /var/log
└─sdb6                                                              8:22   0     1G  0 part [SWAP]