Hi there,

You probably heard about XenServer 7.3 release and some of its features removed.

So, I started a GitHub organization called "xcp-ng", hosting all the repo needed to make a XenServer OS from the sources. It's 100% community backed.

I actually hired (part time but still) an former XAPI developer, so it will accelerate the pace soon. I think we'll be able to release something in Q1, without any feature restriction (think vGPU + everything) and turnkey as XenServer itself.

If it works well, the door for including directly extra stuff in the project is opened: our Gluster driver for example (and a Ceph driver too), better compression algorithm, CentOS packages available, community contribution etc.

Obviously, we'll start small (probably re-using bunch of existing RPMs), but then we'll try to build more and more from the sources. But the ultimate goal is to have just a RPM repo to install from a CentOS and then use yum upgrade to stay up-to-date!

All contributions are welcome, if you want to join the GitHub org, feel free to ask!

Hi there,

Just to let you know I recently made initial tests to put XenServer in another level. I know it seems really important for some of you here to have an hyperconverged solution for XenServer (@scottalanmiller )

I got initial successful test, in a very minimal scenario, so it's just a start.

Context

• a pool of only XenServer 2 nodes (nothing else, no NAS/SAN)
• only using local storage

Objectives

• a loss of one host won't affect storage for any running VM on the other host
• allow live migration between two host without disk transfer
• no split-brain scenario even in 2 nodes context
• no SPOF
• no XenServer system modification

Results

Worked as expected in first tests.

Limitations

• a bit of RAM and storage will be needed to provide this
• it relies on dedicated VMs (one per host) to expose it
• it uses a iSCSI SR in multipath

Ideal objective

XOA would be able to download and deploy all the needed stuff inside an existing infrastructure (depending if there is enough resources obviously). So a "one click to install my virtual SAN" would be totally possible.

Also, there is very interesting possibilities for more nodes scenario (not just replicating, but distributing the content AND duplicate it to allow correct performances).

I think it's better to regroup a little bit all the recent conversations around XO in this topic (if it's fine?). It will be better for me to have one place to follow here than checking for new topics about it

Documentation

The first place to go before asking questions! https://xen-orchestra.com/docs/

Quick links:

• General architecture
• Install
• Main features
• Contributing

Quick intro

XO is a kind of middle-ware between you and multiple XenServer hosts. It doesn't require any XenServer setup: it works out of the box.

With XO, you can manage, backup, delegate your XenServer infrastructure.

Business model

If you are a company and you want a turnkey solution with support: XOA which contains:

• a virtual appliance with XO installed (nothing to do)
• an automatic updater (one click to upgrade)
• corporate support and full priority to fix any issue
• QA is done on XOAs, code/dependencies are "frozen" between releases

If you are not interested by the turnkey solution, you can install it from the sources, you'll have ALL XO features, but no XOA features (the previous given list). If the solution begin to go in production in your company, we strongly suggest to get a XOA to avoid any source problem. It's also paid customers which help to continue to work full time on this

Things to know

• You can complain about the UI, it's already planned to rewrite it from scratch in the next months
• I'm the project leader (and CEO of the company selling it), feel free to ask me anything
• We like to be transparent about how it works to build a project like this. Sharing thoughts is always positive.
• I'm personally convince that more brains are working on a topic, better it is: tell me your needs or ideas, it's always a good thing!
• I've got no problem with critics, go ahead
• We are a very small company, 100% dedicated on XO. We don't have 100 engineer working on it (not even 10)

I'll keep to update this post.

Custom builds

Here, I'll post link to scripts/stuff from people here to create a functional XO installation quickly.

Keep in touch

We try to release every 15 days. I'll write at least one or two blog post per week here: https://xen-orchestra.com/blog/

I put the new articles on Twitter if you want to follow: https://twitter.com/xenorchestra

If you are a XO user, reading the blog is really helpful to catch last features and how to's

Hey there! I'm seriously thinking about hiring a tech for doing XO support, ideally based in the US (where most of our users are). We are reaching a "critical mass" of users, and I would like to delegate level 1 issues (at the start) to someone external (not my devs or myself).

Do you have any tips on where should I look? Obviously, it's remote work at 100%.

I said Q1 for the first release. On schedule. https://xcp-ng.github.io/news/2018/03/31/first-xcp-ng-release.html

First PoC is here: https://xcp-ng.github.io/news/2018/01/22/xcp-ng-on-tracks.html

Have 7 min to read something?

Here is some bits about Vates, the startup behind Xen Orchestra: https://medium.com/@olivier.lambert/building-a-tech-startup-in-france-cc369c0865db#.wpayxisd7

Sorry about my English in this article, but I wanted to share this story here

@scottalanmiller It's planned to release file level restore in next release (~15 days) if everything go smooth.

haha nice try @Francesco-Provino This is foolish to think that

First, this is not true. Then, the number of active Xen users is growing (a reasonable part is due to XCP-ng). And even the number of contributors (also thanks to us).

Xen, by design, is more secure than the other "big" Open Source alternative. The only downside is that's requiring more knowledge to move it forward.

The main issues was 1. Citrix acquiring it but not pushing it fast enough, because not being part of their core skills and 2. not having any Open Source knowledge.

As a true type 1, you can accomplish great things, and yes it requires some efforts. That's exactly the reason why we are partnering with bigger players to really show the true potential of Xen

Maybe you lack the understanding of scale. Xen dev was built and maintained by a relatively small number of people, and despite that, got it working better than most competitor. And you can indeed consider us as a small player right now, but we are roughly doubling each year. Just next year, we'll have more people working on Xen that Citrix itself.

I hope this tells a bit on Xen's future

edit: just few example of driving Xen innovation:

• https://xcp-ng.org/blog/2021/09/14/runx-next-generation-secured-containers/ (in partnership with Stefano from Xilinx)
• https://xcp-ng.org/blog/2021/07/12/dpus-and-the-future-of-virtualization/ (in partnership with Kalray, a CPU manufacturer)
• https://vates.fr/blog/vates-joins-riscv-international/ (porting Xen to RISC-V)
• https://vates.fr/blog/kalray-vates-and-scaleway-alliance/ (alliance with a large Cloud company, Scaleway)

It's a LOT of innovation for a dying project. But well, I'm used to hear that Xen is dead for the last 10 years, so you are not the first to be wrong on this

Hi lads!

I'm XO's project leader

• @scottalanmiller XO does provide DR features, see https://xen-orchestra.com/docs/disaster_recovery.html
• @johnhooks it's all Open Source (aGPLv3). If you don't want the turnkey solution (appliance and pro support), "use the sources Luke"
• about the price itself, please consider you'll have an appliance with support working out of the box (plus the web updater). And price is flat.
• complete features list is here: https://xen-orchestra.com/docs/features.html

Last thing, about using it inside or outside your infrastructure: because it's agent-less, you can use it everywhere. You can even imagine a small host with only XOA inside, the thing needed to "talk" to your others XenServer host is only a TCP connection to port 443. VPN connection to multiple datacenter is fairly possible (or other tunnels).

@Francesco-Provino by scale I meant the scale of Xen core developers (ie headcount). It's not that much especially compared to Xen adoption at such… scale

So if you did a lot of things with a relatively small team, this is a pretty nice clue on what could be done with more focus and people! (and yes, Citrix was completely unfocused on Xen, only few years after acquiring it).

But this is clearly changing (the situation, not Citrix ).

And now v1.18.6 is just out

haha nice try @Francesco-Provino This is foolish to think that

First, this is not true. Then, the number of active Xen users is growing (a reasonable part is due to XCP-ng). And even the number of contributors (also thanks to us).

Xen, by design, is more secure than the other "big" Open Source alternative. The only downside is that's requiring more knowledge to move it forward.

The main issues was 1. Citrix acquiring it but not pushing it fast enough, because not being part of their core skills and 2. not having any Open Source knowledge.

As a true type 1, you can accomplish great things, and yes it requires some efforts. That's exactly the reason why we are partnering with bigger players to really show the true potential of Xen

Maybe you lack the understanding of scale. Xen dev was built and maintained by a relatively small number of people, and despite that, got it working better than most competitor. And you can indeed consider us as a small player right now, but we are roughly doubling each year. Just next year, we'll have more people working on Xen that Citrix itself.

I hope this tells a bit on Xen's future

edit: just few example of driving Xen innovation:

• https://xcp-ng.org/blog/2021/09/14/runx-next-generation-secured-containers/ (in partnership with Stefano from Xilinx)
• https://xcp-ng.org/blog/2021/07/12/dpus-and-the-future-of-virtualization/ (in partnership with Kalray, a CPU manufacturer)
• https://vates.fr/blog/vates-joins-riscv-international/ (porting Xen to RISC-V)
• https://vates.fr/blog/kalray-vates-and-scaleway-alliance/ (alliance with a large Cloud company, Scaleway)

It's a LOT of innovation for a dying project. But well, I'm used to hear that Xen is dead for the last 10 years, so you are not the first to be wrong on this

@stacksofplates said in KVM or VMWare:

@olivier said in KVM or VMWare:

@stacksofplates said in KVM or VMWare:

@pete-s said in KVM or VMWare:

It isn't the ability to automate that is the problem. It's the availablility of easy to use tools that is the problem.

Thats the whole point I'm making.

KVM is hard to automate. Not that it's impossible, but the tooling doesn't exist to where you can easily automate like with VMware.

And that's a very good point. That's why here at Vates, we made various efforts in XCP-ng/Xen Orchestra, providing multiple solutions: Packer, Terraform and even Ansible integration. That's also why Xen Orchestra really makes sense as a "middleware", as a single central point to consume with its API. Like vCenter in fact.

This is a true way to create value on top of it. The other aspect is all about integration, like we did with Netbox for example (sync all VMs and hosts, with their IP address, config and such to Netbox).

Right VMware or Xen Orchestra. If the tool isn't built with an API first mindset, the work needed to automate it greatly increases.

I agree. And to be honest, we learnt it "by accident" (ie our API was made for our internal usage). But now we are working more on the direction of "API as first class citizen", thanks to the large feedback we got from our users. I'm happy we took the right "overall design" decisions at first, allowing us to rely on Xen Orchestra as a central point (vs one UI per host, which can be handy but doesn't scale)

@stacksofplates said in KVM or VMWare:

@pete-s said in KVM or VMWare:

It isn't the ability to automate that is the problem. It's the availablility of easy to use tools that is the problem.

Thats the whole point I'm making.

KVM is hard to automate. Not that it's impossible, but the tooling doesn't exist to where you can easily automate like with VMware.

And that's a very good point. That's why here at Vates, we made various efforts in XCP-ng/Xen Orchestra, providing multiple solutions: Packer, Terraform and even Ansible integration. That's also why Xen Orchestra really makes sense as a "middleware", as a single central point to consume with its API. Like vCenter in fact.

This is a true way to create value on top of it. The other aspect is all about integration, like we did with Netbox for example (sync all VMs and hosts, with their IP address, config and such to Netbox).

Automation is key.

Some links/resources:

• https://registry.terraform.io/providers/terra-farm/xenorchestra/latest/docs
• https://github.com/ansible-collections/community.general/pull/3344
• https://xen-orchestra.com/blog/netbox-sync-with-xen-orchestra/

I think the VM could still be migrated so long as I detach the passthrough disks first, move those disks to a new host, migrate the VM to the new host, and then re-attach/passthrough the disks on the new host.

You can indeed. Not very practical but no technical barrier.

I wonder - can you create an NFS mount point in XenServer or XCP-NG? then just share that via loopback?

I don't really see the point of doing that? I had in mind an NFS share mounted directly in the VM. Simple, efficient (if you already have a NAS obviously)

You can attach more than 7 disks when you have tools in the VM. In your case, you don't need a VM in the traditional way, ie something flexible that you can migrate etc. So you can indeed attach your disks directly, regardless the hypervisor you choose.

Another more flexible alternative would be to have a "normal" VM, but attach a NFS share on it to store your data. This way you keep the flexibility of the VM and the large storage you need. The extra requirement is any NFS capable machine (even a very cheap NAS)

Thanks for correcting the sentence @travisdh1 Indeed, SMAPIv1 is using VHD format everywhere. This format is limited at 2TiB by "design" [1] . This has nothing to do with XO or even XCP-ng because it's a fork of XenServer, ie a copy with new or improved code. So remember that regardless which filesystem you use, as long as you are using VHD format to store virtual disk, you are limited to 2TiB.

However, SMAPIv3 is using qcow2 format instead, "solving" this limitation. We (XCP-ng team) are currently working on improving SMAPIv3 to support disk import/export in qcow2 (which isn't even done by Citrix people themselves ). As soon we got that, the next step is to write drivers for ext4 for example, which is doable relatively easily.

One of main issue with SMAPIv3 (there's others) is the fact a part of the development is done privately by Citrix instead of collaborating (see this conversation on GitHub), so the goal is to catching up on our side to be able to get an upstream public faster and become the de facto upstream standard. We are working toward that but it's not something you solve in one week (you need to go deep in qemu-dp/xen blktap, see our efforts here etc.)

[1]: The VHD format has a built-in limitation of just under 2 TiB (2040 GiB) for the size of any dynamic or differencing VHDs. This is due to a sector offset table that only allows for the maximum of a 32-bit quantity. It is calculated by multiplying 232 by 512 bytes for each sector.

edit: also, as soon we got qcow2 import/export support in XCP-ng, we could use that format in XO to store backup. So far, there's only 2 options to get disk data from XS/XCP-ng: raw or vhd (that's why XO is storing VHD files, because… that's what we got from the hypervisor!)

Also we could achieve hyperconvergence "the other way" (unlike having a global shared filesystem like Gluster or Ceph) but use fine grained replication (per VM/VM disk). That's really interesting (data locality, tiering, thin pro etc.). Obviously, we'll collaborate to see how to integrate this in our stack

FYI we started very interesting discussions with Linbits guys (we could achieve something really powerful by integrating Linstore inside XCP-ng as a new hyperconverged solution). It means really decent perfs (almost same as local storage) and keep it robust and simple.