How to Stop XenServer from Mounting /var/log
-
Can I unmount it while the server is running?
AKA ... what should be the process for the whole job?
- Stop rsyslog
- Copy files from /var/log to the new location
- Unmount /var/log
- Delete /var/log folder
- Symlink /var/log to other storage
- Restart rsyslog
Does that sound about right?
-
Oooh, I think I am getting somewhere!
lrwxrwxrwx 1 root root 64 Jul 11 13:01 log -> /run/sr-mount/cbe8175f-43b3-3a64-cdac-3c1475aa2d01/xenserverlogs -
The one thing to note is that in my system, XS7 used my entire array for VM storage. I boot off USB and only have one presented drive (Raid 5 SSD) in the server.
After discussing with @scottalanmiller it was determined it was probably OK to create a directory alongside the .VHD files in SR-Mount and store the logs there.
So far, so good. Nothing has imploded.
-
Other options would be to use LVM to create two logical volumes on that RAID device or use the RAID device's LVM system to do the same.
-
Can you think of any reason why XS would object to creating a folder in sr-mount?
-
@BRRABill said in How to Stop XenServer from Mounting /var/log:
Can you think of any reason why XS would object to creating a folder in sr-mount?
Yes, if it is scanning that directory for folders believing that the existence of a folder in that location implies something, then yes.
-
@scottalanmiller said
Yes, if it is scanning that directory for folders believing that the existence of a folder in that location implies something, then yes.
Would it be safer, do you think, to go with your other method of shrinking the volume and adding another volume for the logging?
I mean, I'd like to figure this out not just for me, but for all the people in the future who ML recommends XS on USB to.
-
You've already done it, so I would just check and see how it is working.
-
@scottalanmiller said
You've already done it, so I would just check and see how it is working.
Seems fine.
I just like to, you know, check.
I figure someone here would know if it's an OK idea or say
NO FOR THE LOVE OF ALL THAT IS GOOD DO NOT DO THAT!!!!We have other XS users ... what are they doing?
-
@BRRABill said in How to Stop XenServer from Mounting /var/log:
@scottalanmiller said
You've already done it, so I would just check and see how it is working.
Seems fine.
I just like to, you know, check.
I figure someone here would know if it's an OK idea or say
NO FOR THE LOVE OF ALL THAT IS GOOD DO NOT DO THAT!!!!We have other XS users ... what are they doing?
Just letting it log to the usb drive for now. Yes, I've got a copy of the current config that gets taken every day should the usb stick croak.
-
@travisdh1 said
Just letting it log to the usb drive for now. Yes, I've got a copy of the current config that gets taken every day should the usb stick croak.
Out of curiosity, how are you copying the current config?
Cloning the USB? Or making backups through XC/CLI?
-
So, things did not go so well overnight. (Note this is a test XS not my production box.)
Couple things happened. Not sure if it is a coincidence, or something more sinister.
I could not connect to this XS from either XC or Putty. You could connect through the local console.
The first thing I did, of course, was the check to see if the logging was still working. /var/log was sitting there with big fat red text. Hmmm. So I figured I would check the LV on /run/sr-mount to see when the logs stopped. Except that was no longer mounted there. Uh oh.
I also noticed there seemed to be something wrong with the reporting of the devices.
So, I decided to reboot the server, and hope it came back up. After taking a looooooooooong time to shut down, it did indeed come back up, but the same problem existed. The LV was not attached, and it had no network connectivity.
The LV still appears to be there, but I wonder if something else went wacky with this XS install. I mean, could putting a subfolder in the VM storage directory really cause this much havoc?
-
I'm leaning towards something wacky, because even if the SR was detached for some reason, why would the networking not work as well?
-
Looks more like a crash than anything. So it is back up and working after the reboot?
-
@scottalanmiller said
Looks more like a crash than anything. So it is back up and working after the reboot?
The SR is not mounted, and the networking is not working.
So no, not working. Which is why I wonder if the entire thing just went south for some reason.
-
@BRRABill said in How to Stop XenServer from Mounting /var/log:
@scottalanmiller said
Looks more like a crash than anything. So it is back up and working after the reboot?
The SR is not mounted, and the networking is not working.
So no, not working. Which is why I wonder if the entire thing just went south for some reason.
Something is wrong with that first lsblk output. System has crashed/been corrupted by something.
-
Well that sucks.
-
@scottalanmiller said in How to Stop XenServer from Mounting /var/log:
Well that sucks.
The good news is that this is Xen. Reinstall, import the storage repository, and import the configs, shouldn't take more than an hour.
I'd be nervous about putting that into production till I figured out what caused the issue.
-
@travisdh1 said
Something is wrong with that first lsblk output. System has crashed/been corrupted by something.
It came back "normal" after a reboot, but obviously a lot of functionality was hosed.
I tried running a dd on this machine the other day, and it was taking forever.
After about 90 minutes I stopped it, and it had only done 15GB thus far. (Of a 64 GB stick.)
Possible maybe the USB stick was going bad, and finally gave up the ghost?
Is it possible cancelling (^C) dd would have done anything to the source USB stick?
-
dd only reads from the if device, it does not modify it. So no.
