Xen Orchestra - Community Edition - Installing with Yarn

DustinB3403

@noelweston Start by providing the output of journalctl -u xo-server -f -n 50

noelweston

-- Logs begin at Mon 2018-06-25 13:11:23 BST. --
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:11:31 xoa systemd[1]: Stopped XO Server.
Jun 25 13:11:31 xoa systemd[1]: Started XO Server.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:11:31 xoa systemd[1]: Stopped XO Server.
Jun 25 13:11:31 xoa systemd[1]: Started XO Server.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:11:32 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:11:32 xoa systemd[1]: Stopped XO Server.
Jun 25 13:11:32 xoa systemd[1]: xo-server.service: Start request repeated too quickly.
Jun 25 13:11:32 xoa systemd[1]: Failed to start XO Server.
Jun 25 13:16:23 xoa systemd[1]: Stopped XO Server.
Jun 25 13:16:23 xoa systemd[1]: Started XO Server.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:16:23 xoa systemd[1]: Stopped XO Server.
Jun 25 13:16:23 xoa systemd[1]: Started XO Server.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:16:23 xoa systemd[1]: Stopped XO Server.
Jun 25 13:16:23 xoa systemd[1]: Started XO Server.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:16:23 xoa systemd[1]: Stopped XO Server.
Jun 25 13:16:23 xoa systemd[1]: Started XO Server.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:16:23 xoa systemd[1]: Stopped XO Server.
Jun 25 13:16:23 xoa systemd[1]: Started XO Server.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:16:23 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 25 13:16:24 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 25 13:16:24 xoa systemd[1]: Stopped XO Server.
Jun 25 13:16:24 xoa systemd[1]: xo-server.service: Start request repeated too quickly.
Jun 25 13:16:24 xoa systemd[1]: Failed to start XO Server.

DustinB3403

Another great place to start with is the troubleshooting template: https://github.com/Jarli01/xenorchestra_installer/blob/master/Troubleshooting.md

DustinB3403

Post that last 200 lines, as that error is just repeating. (change 50 to 200)

DustinB3403

As another option, you could just restore your snapshop (assuming you made one) and then download the config file and build a new instance and then import the config file.

noelweston

Hi Dustin -

That's all that journalctl gives me - I assume this is since the last reboot?

The troubleshooting info is pretty much what I've already got - service status is above, I did the git configs after the first update attempt.

/var/log/syslog has this:

Jun 25 13:11:31 xoa systemd[1]: Started Hold until boot process finishes up.
Jun 25 13:11:31 xoa systemd[1141]: xo-server.service: Failed at step CHDIR spawning /usr/local/bin/node: No such file or directory
Jun 25 13:11:31 xoa systemd[1]: Started Terminate Plymouth Boot Screen.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.

Unfortunately no snapshot - I've never had an issue with the updates before, so didn't even think to do so.

DustinB3403

@noelweston said in Xen Orchestra - Community Edition - Installing with Yarn:

Hi Dustin -

That's all that journalctl gives me - I assume this is since the last reboot?

Yes, that is correct.

The troubleshooting info is pretty much what I've already got - service status is above, I did the git configs after the first update attempt.

/var/log/syslog has this:
Jun 25 13:11:31 xoa systemd[1]: Started Hold until boot process finishes up.
Jun 25 13:11:31 xoa systemd[1141]: xo-server.service: Failed at step CHDIR spawning /usr/local/bin/node: No such file or directory
Jun 25 13:11:31 xoa systemd[1]: Started Terminate Plymouth Boot Screen.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 25 13:11:31 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Unfortunately no snapshot - I've never had an issue with the updates before, so didn't even think to do so.

So your only options are to either restore from backup (assuming you were at least making backups) or to build a new instance.

@noelweston do you have your configuration backed up?

noelweston

Yes, I do have a backup... not that recent, but it will do.

Any idea what went wrong?

noelweston

And next dumb question - how do I restore a Xen Orchestra .xva backup into Xen without Xen Orchestra?

DustinB3403

@noelweston said in Xen Orchestra - Community Edition - Installing with Yarn:

Yes, I do have a backup... not that recent, but it will do.

Any idea what went wrong?

No way to know without the logs, did you identify yourself to git as that can cause issues (mostly you won't be able to update).

noelweston

Yes, as above - the first run I got the git "who are you" warning, so I ran the git configs and re-ran the updates.
Which logs do you mean - are there any others I can look at, or do they disappear on a reboot?

DustinB3403

@noelweston said in Xen Orchestra - Community Edition - Installing with Yarn:

Yes, as above - the first run I got the git "who are you" warning, so I ran the git configs and re-ran the updates.
Which logs do you mean - are there any others I can look at, or do they disappear on a reboot?

the journal logs provide the "best" information as to what went wrong. If you have the update logs (what went by on the screen as you updated) that could provide some insight as well.

noelweston

Ah, OK - no, not any more - again, they went in one of the reboots.
I'll restore the backup (figured that one out), take a snapshot and try again.

black3dynamite

@noelweston said in Xen Orchestra - Community Edition - Installing with Yarn:

And next dumb question - how do I restore a Xen Orchestra .xva backup into Xen without Xen Orchestra?

You can import it using XenCenter.
Or via command line.
https://david.pryke.us/2016/02/17/import-xen-vm-from-xva-file-via-cmd-line-or-gui/

noelweston

OK, with a backup restored and snapshotted, I re-did the git config and tried the updater again, and it's failed / borked again.

Relevant lines from the log:

Jun 26 10:26:48 xoa systemd[1]: Stopping XO Server...
Jun 26 10:26:48 xoa xo-server[1013]: Tue, 26 Jun 2018 09:26:48 GMT xo:main SIGTERM caught, closing…
Jun 26 10:26:48 xoa xo-server[1013]: Tue, 26 Jun 2018 09:26:48 GMT xo:main bye :-)
Jun 26 10:26:48 xoa systemd[1]: Stopped XO Server.
Jun 26 10:38:06 xoa systemd[1]: Started XO Server.
Jun 26 10:38:06 xoa systemd[1]: xo-server.service: Main process exited, code=exited, status=200/CHDIR
Jun 26 10:38:06 xoa systemd[1]: xo-server.service: Unit entered failed state.
Jun 26 10:38:06 xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Jun 26 10:38:06 xoa systemd[1]: xo-server.service: Service hold-off time over, scheduling restart.
Jun 26 10:38:06 xoa systemd[1]: Stopped XO Server.

Which is the same as yesterday.

I do have the full build process captured - it's nearly 1200 lines, so it didn't seem like a good plan to upload it straight away. There are no errors that I can see, and the only warning is back to node versions:

warning You are using Node "7.6.0" which is not supported and may encounter bugs or unexpected behavior. Yarn supports the following semver range: "^4.8.0 || ^5.7.0 || ^6.2.2 || >=8.0.0"

Any further thoughts?

Thanks for the help, by the way - much appreciated.

Noel

noelweston

Another update - reverted to the snapshot and ran the updater with -n stable, and got a different set of errors but still a non-starting service...

First error shown is
error /opt/xen-orchestra/node_modules/level-party/node_modules/leveldown: Command failed.

Followed by

/home/serverbackup/.node-gyp/10.4.1/include/node/node.h:88:42: note: in definition of macro ‘NODE_DEPRECATED’
     __attribute__((deprecated(message))) declarator
                                          ^
leveldown.target.mk:113: recipe for target 'Release/obj.target/leveldown/src/batch.o' failed
make: Leaving directory '/opt/xen-orchestra/node_modules/level-party/node_modules/leveldown/build'
make: *** [Release/obj.target/leveldown/src/batch.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/opt/xen-orchestra/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
gyp ERR! System Linux 4.4.0-128-generic
gyp ERR! command "/usr/local/bin/node" "/opt/xen-orchestra/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /opt/xen-orchestra/node_modules/level-party/node_modules/leveldown

Any further thoughts?

Noel

DustinB3403

Node version 7 is depreciated, v 6 is their LTS and 8 is current (8 wasn't working a while ago)

DustinB3403

@noelweston based on the logs, you have a bad version of node. Downgrade to their LTS release.

Danp

My memory could be faulty, but I seem to recall XO requiring a newer version of Node than 7.x. I'm currently at 8.11.1 and not having any issues.

Have you tried to follow the troubleshooting steps listed here?

DustinB3403

@danp the issue we had is that they depreciated a version as we were using version "8" and changed it to LTS. Which iirc they depreciated the number and made an older version their LTS.

Here is the GH issue.