Today is the day from Hell!
-
I'm currently creating a Mint Live USB stick.
My plan is to boot a laptop off it and then look at the SD card to see if it's full - if so, find logs and kill them, then try to boot the server from it.
If not - then I guess I get to reinstall XS on a new SD card - and pry that I can import my SR and recover my VMs.
-
Sorry
-
I purchased two new 32 GB SD cards and am now booting from the XS install media.
Here are the instructions Bill found
-
So I popped in a new SD Card and installed XS 6.5 again. Went to do updates and decide to just start at the oldest and move my way upward. Well once I reached Sp1 suddenly there was about 35 more - no thanks, don't want to be here forever, already late enough.
Found this link: http://support.citrix.com/article/CTX138115
Updated June 14, 2016It shows that you can go Install > Sp1 > update 12 > update 24 > update 27 > update 29 > update 32 > update 33.
Many of the updates are cumulative, do you don't have to install them all from scratch.
Hope this saves someone some time.
-
OK another snag. Bill's link never mentioned the possibility that this isn't an LVM partition (which it's not, it's ext) so my re-introduce command is wrong.
ug - I wonder if my SR is destroyed?
*edit
I feel that it's important to get as much information to anyone reading this thread for troubleshooting reasons as early in the thread as possible as such, enjoy the following.
Step number 11 you are introducing the old storage repository to your system. The current instructions do not have you confirm the type of partition that you have.
During a typical install of XS 6.5 you are given two options (through a checkbox) to either full provision VMs or Thin Provision them. The default is a non checked box which means your VMs will be full provisioned and your SR will be lvm. But if you're like me, I checked the box because i wanted Thin Provisioning, my partition was ext.This is a long winded way to say, check your partition type first. If you have an ext partition type, change the type=lvm to type=ext in your xe sr-introduce command.
Also in step 14, the last line, which is a UUID, is bolded and appears to be part of the command you're suppose to execute. But really, this is the output from the command. You'll use this output in step 15.
-
This, I think, is beautiful!
https://i.imgur.com/mdAmIbC.png
I believe that this means that my SR still has all of my disks on it. Now to create the VMs and assign the disks to them.
-
Next major piece of success - Got XO running again. Now to build a Windows 2012 R2 VM and assign the VDIs to it.
-
OK, the best site I've seen, well, so far today.
https://i.imgur.com/gSN9vAJ.png
All of my servers are back up and running.
-
@Dashrender said in Today is the day from Hell!:
OK another snag. Bill's link never mentioned the possibility that this isn't an LVM partition (which it's not, it's ext) so my re-introduce command is wrong.
ug - I wonder if my SR is destroyed?
This bit missing from the Citrix documentation cost me at least 1 hour, but probably more like 2.
-
This thread was helpful in removing the SR and PBD so I could reattach them correctly.
-
and major props to
@Stefan Lissmats
For providing this excellent post
Quite old thread but I wanted to contribute with what I strongly believe is the answer.
I ran into the exact same issue today and also tried the renaming solution you tried.
But the solution is quit simple, or it was for me anyway.
The prefix XSLocalEXT is not a lvm volume but i ext based volume.
So in the end what i simply did was to run the sr-introduce with the type ext instead of lvm
So your command would be sr-introduce uuid=43d6cf1c-3b86-dd4f-8afc-9dcf2688082c type=ext >name-label="Local storage" content-type=userOver here
Here was my error
Command
xe pbd-plug uuid=9dba453d-db22-57e5-0504-55d2d1dbb6d5Error code: SR_BACKEND_FAILURE_47
Error parameters: , The SR is not available [opterr=no such volume group: VG_XenStorage-3c68d5fc-2e9f-5f5b-b832-dc1aacf920b4],Comparing the above output including VG_XenStorage which did not match my XSLocalEXT let me to that post.
-
@Dashrender said in Today is the day from Hell!:
@Dashrender said in Today is the day from Hell!:
OK another snag. Bill's link never mentioned the possibility that this isn't an LVM partition (which it's not, it's ext) so my re-introduce command is wrong.
ug - I wonder if my SR is destroyed?
This bit missing from the Citrix documentation cost me at least 1 hour, but probably more like 2.
Hope things are working again?
About the documentation: Contribute back, tell them about the missing part.
-
Just FYI, this is what I use for patching. I have it setup on a cron schedule.
-
@thwr said in Today is the day from Hell!:
@Dashrender said in Today is the day from Hell!:
@Dashrender said in Today is the day from Hell!:
OK another snag. Bill's link never mentioned the possibility that this isn't an LVM partition (which it's not, it's ext) so my re-introduce command is wrong.
ug - I wonder if my SR is destroyed?
This bit missing from the Citrix documentation cost me at least 1 hour, but probably more like 2.
Hope things are working again?
About the documentation: Contribute back, tell them about the missing part.
Done.
-
Glad that things are back working. Sorry that I was only on telegram for most of this.
-
@Dashrender said
Many of the updates are cumulative, do you don't have to install them all from scratch.
That was I was discussing in my thread from a while back, that there is a specific order to do them
The conclusion of that thread was that XO was smart enough to do it and should always be used, but if you don't have XO running, what you found is the best alternative I think.
-
I was a little confused about how exactly to recover from a XS boot failure. Obviously the way you did it was able to get the VMs back up and running. That is good to know.
There is also a methodology to backup the entire XS system, including the VM metadata. This process is described here...
http://techblog.danielpellarini.com/sysadmin/steps-to-take-to-restore-xenserver-from-backup/In another linked article, he explains what is backup up in the metadata as:
"And by VM metadata I mean what drives are connected to what VM, the name of the drive, the network interfaces etc."I guess if you had a lot of custom settings for your VMs (tags, etc.) or a lot of stuff listed as metadata, it might be good to do such a backup.
This is something I am still a bit unclear on.
-
@BRRABill said in Today is the day from Hell!:
I guess if you had a lot of custom settings for your VMs (tags, etc.) or a lot of stuff listed as metadata, it might be good to do such a backup.
This is something I am still a bit unclear on.
Or just have like 20+ VMs (hell I'd want that at more than 5 for sure). You don't want to be guessing what VDI (I just feel icky typing that) - I mean virtual disks - go with what VM. Plus, as you said, what adapters are there, etc.
-
@coliver said in Today is the day from Hell!:
Just FYI, this is what I use for patching. I have it setup on a cron schedule.
Thanks!
-
@Dashrender said in Today is the day from Hell!:
So I popped in a new SD Card and installed XS 6.5 again. Went to do updates and decide to just start at the oldest and move my way upward. Well once I reached Sp1 suddenly there was about 35 more - no thanks, don't want to be here forever, already late enough.
Found this link: http://support.citrix.com/article/CTX138115
Updated June 14, 2016It shows that you can go Install > Sp1 > update 12 > update 24 > update 27 > update 29 > update 32 > update 33.
Many of the updates are cumulative, do you don't have to install them all from scratch.
Hope this saves someone some time.
This is what i did when upgraded past sp1, just installed about 3 or 4 cumulative updates.
Also, in the future you could save a backup of your XS install with vm/sr info to the local SR, or anywhere else, just in case. It is an option from #xsconsole
I do this before upgrading usually, and keep it there afterwards.
edit: ah wait, i see someone mentioned the xsconsole backups.
