Stopping XenServer From Writing To A USB Boot Drive



  • There has been discussion on various threads about this topic, so I thought it would be good to give it its own area.

    Basically, we have been directing a lot of people to install XS, and install it on USB. However, XS does a good deal of writing to the boot disk, and that cannot happen for the safety of the USB boot device.

    I think it would be prudent to figure out exactly WHAT it writes, and how to stop it.

    QUESTION 1:
    How do we determine what is being written? Is there a way to track what XS is writing?

    I have found these two articles, both of which talk about some of the files it does write. But how do we know these are the only files being written?

    http://xenserver.org/discuss-virtualization/virtualization-blog/entry/xenserver-root-disk-maintenance.html

    http://xenserver.org/discuss-virtualization/virtualization-blog/entry/log-rotation-and-syslog-forwarding.html



  • With remote syslog turned ON, these are still some of the files that appear to be writing to the disk.

    This is from /var/log from today.

    -rw------- 1 root root 552045 Jul 8 11:54 daemon.log
    -rw------- 1 root root 36569171 Jul 8 11:54 xensource.log
    -rw------- 1 root root 6993911 Jul 8 11:54 xenstored-access.log
    -rw------- 1 root root 3314999 Jul 8 11:53 secure
    -rw------- 1 root root 1651148 Jul 8 11:53 audit.log
    -rw-r--r-- 1 root root 38273316 Jul 8 11:53 lastlog
    -rw------- 1 root root 1092509 Jul 8 11:53 SMlog
    -rw-rw-r-- 1 root utmp 24960 Jul 8 11:53 wtmp
    -rw------- 1 root root 39140 Jul 8 11:52 xcp-rrdd-plugins.log
    -rw------- 1 root root 7910 Jul 8 11:50 cron
    -rw------- 1 root root 25237 Jul 8 11:49 user.log
    -rw------- 1 root root 2197 Jul 8 11:27 kern.log
    drwxr-xr-x 2 root root 4096 Jul 8 11:27 blktap
    -rw-r--r-- 1 root root 0 Jul 8 04:02 boot.log
    -rw------- 1 root root 0 Jul 8 04:02 crit.log
    drwxr-xr-x 2 root root 4096 Jul 8 04:02 xen
    -rw------- 1 root root 0 Jul 8 04:02 maillog
    -rw------- 1 root root 0 Jul 8 04:02 messages
    -rw------- 1 root root 0 Jul 8 04:02 spooler
    -rw------- 1 root root 25430 Jul 8 04:02 cron.1
    -rw------- 1 root root 1397607 Jul 8 04:02 daemon.log.1
    -rw-r--r-- 1 root root 0 Jul 8 04:02 interface-rename.log
    -rw------- 1 root root 21282 Jul 8 04:02 user.log.1
    -rw------- 1 root root 17914819 Jul 8 04:01 xenstored-access.log.1
    -rw------- 1 root root 2878021 Jul 8 04:01 audit.log.1
    -rw------- 1 root root 6119984 Jul 8 04:01 secure.1
    -rw------- 1 root root 1438123 Jul 8 04:01 SMlog.1
    -rw------- 1 root root 96116 Jul 8 03:57 xcp-rrdd-plugins.log.1
    -rw------- 1 root root 54918053 Jul 8 00:39 xensource.log.1
    drwxr-xr-x 2 root root 4096 Jul 8 00:00 sa


  • Service Provider

    You have to not just turn on remote, but stop it from writing locally.



  • @scottalanmiller said

    You have to not just turn on remote, but stop it from writing locally.

    Right. I am going to try the steps they suggested in that article. Perhaps it will work this time.

    Well, step one, already an issue...

    This stpe
    "Finally, select "OK" and the stand-alone XenServer (or pool) will update its Syslog configuration, or more specifically, /var/lib/syslog.conf. "

    /var/lib/syslog.conf is no longer there



  • I think that file has been replaced with /etc/rsyslog.d/xenserver.conf

    Which reads...

    # Suppress duplicate messages and report "Last line repeated n times"
    $RepeatedMsgReduction on
    
    # Don't rate-limit messages - this isn't the right way to go about 
    # reducing log size!
    $IMUXSockRateLimitInterval 0
    $SystemLogRateLimitInterval 0
    
    # Ensure critical and higher level errors are logged synchronously.
    *.crit;mail.none;authpriv.none;cron.none		/var/log/crit.log
    
    # Log by facility.
    kern.*							-/var/log/kern.log
    daemon.*						-/var/log/daemon.log
    user.*							-/var/log/user.log
    
    # The authpriv file has restricted access.
    authpriv.*						-/var/log/secure
    
    # Log all the mail messages in one place.
    mail.*							-/var/log/maillog
    
    # Log cron stuff
    cron.*							-/var/log/cron
    
    # Save boot messages also to boot.log
    local7.*						/var/log/boot.log
    
    # Xapi rbac audit log echoes to syslog local6
    local6.*						-/var/log/audit.log
    
    # Xapi, xenopsd echo to syslog local5
    local5.*						-/var/log/xensource.log
    
    # V6d echo to syslog local4
    local4.*						-/var/log/v6d.log
    
    # xenstore access to syslog local3
    local3.info						-/var/log/xenstored-access.log
    
    # Storage Manager to syslog local2
    local2.*						-/var/log/SMlog
    
    # xcp-rrdd-plugins (info and above) to local0
    local0.info						-/var/log/xcp-rrdd-plugins.log
    
    # ignore default rules
    
    *.* @10.0.4.26
    *.* ~
    


  • If you turn off remote logging, this is what you get at the bottom...

    # ignore default rules
    
    *.* ~




  • Another thing to ponder...

    With XS7, there is a separate 4GB log partition.

    Is there a way to forward that elsewhere?



  • @Danp said

    Found this: http://discussions.citrix.com/topic/378269-xenserver-7-varlogmessages/

    Yeah, I found that, too.

    But I am wondering...is that the file that tells all the possible logs, and where they write to?

    And can we tell if anything else is writing?



  • Does the link here not already address this issue?

    "I mentioned before that Syslog can forward messages to other hosts. Furthermore, it can forward Syslog messages to other hosts without writing a copy of the log to local disk. What this means is that a single XenServer or a pool of XenServers can send their log data to a "Syslog Aggregator". "





  • Or even this last guide on the very same page, All-In-One-Guide.



  • @DustinB3403 said

    Does the link here not already address this issue?

    "I mentioned before that Syslog can forward messages to other hosts. Furthermore, it can forward Syslog messages to other hosts without writing a copy of the log to local disk. What this means is that a single XenServer or a pool of XenServers can send their log data to a "Syslog Aggregator". "

    No, because further in the article, you will see...

    "Certain logs will still continue to record Syslog on the host, so it may be desirable to edit /var/lib/syslog.conf and add comments to lines where a "-/var/log/some_filename" is specified as lines with "@x.x.x.x" dictate to forward to the Syslog aggregator."

    And in XS7, that file has been totally replaced with the one I posted, which has different syntax.



  • @BRRABill Wouldn't the same essentially apply to the log file you have there?

    Your file in /etc/rsyslog.d/xenserver.conf:

    # Save boot messages also to boot.log
    local7.*	
    

    Would be changed to:

    # Save boot messages also to boot.log
    local7.*      @your-syslog-ip
    #local7.*


  • And in XS6.5 the syslog location is stored at

    /etc/syslog.conf
    

    Which still has the same flags.



  • @DustinB3403

    Well, in 6.5, the syntax was as follows...

    # Save boot messages also to boot.log
    local7.*             @10.0.0.1
    # local7.*         /var/log/boot.log
    

    In this new version, it just puts the IP at the bottom.

    @scottalanmiller mentioned, as I think you did as well, that I should just redirect /var/log somewhere else.

    Being a little fresh in Linux partitioning, I most post a thread for help with that.



  • @BRRABill Why are you not commenting out the records as needed and adding them under each field in order?

    You can edit the file (on XS7) with:

    sudo nano /etc/rsyslog.d/xenserver.conf
    

    Or on XS 6.5

    sudo nano /etc/syslog.conf


  • I'll follow the setup on my lab tonight that is posted in the first link, and make all of the changes on XS6.5.

    Then I'll see how it performs. from there I'll tear down my installation and repeat for XS7.



  • Here's one to watch from the Citrix forums: http://discussions.citrix.com/topic/379454-booting-xenserver-off-usb-safe/

    P.S. Which one of you guys wrote this? 😉



  • @Danp said in Stopping XenServer From Writing To A USB Boot Drive:

    Here's one to watch from the Citrix forums: http://discussions.citrix.com/topic/379454-booting-xenserver-off-usb-safe/

    P.S. Which one of you guys wrote this? 😉

    That was me.

    Those guys know the nuts and bolts of XS, though they aren't real responsive to threads all the time.

    On ML that would have have 500 posts already and been forked 6 times. 🙂


Log in to reply
 

Looks like your connection to MangoLassi was lost, please wait while we try to reconnect.