Running Docker Containers in Scale HC3 VMs ... on Linux or Windows

[scale]

Let me begin by stating, I'm no docker / containers expert but we've been getting an increasing number of questions about containers on HC3, as well as an increasing number of customers actually using containers in production so I wanted to gather up some information, try some things out myself and begin a discussion here.

For years, you have been able to run linux based containers (using docker and LXC) inside linux VM's running on HC3. Nothing really fancy and there are all sorts of guides on docker out there. But high level, on Centos7 for example - simply su "yum install docker" then su "docker run hello-world" to run your first container. So Linux based containers on Linux VMs running on HC3 - check!

However, Microsoft recently introduced the ability to run Windows based containers (windows binaries) using Windows Containers feature in Windows Server 2016 and Windows 10. We've had a few people ask about it or try it inside windows VM's running on HC3 and have heard mixed results generally installation or the believe that nested virtualization (VTx) inside the VM was required. In my initial testing, I myself also saw mixed results but I believe I've "cracked the code" to running docker for windows images, on Windows VM's running on HC3.

tl/dr: docker for windows needs the windows OS to have a virtual switch configured, which is a component of windows hyper-v role... if it's not installed it will try to install hyper-v ... appear to work but not really (and actually can pretty badly mess up windows so don't do this on production VMs! use snapshots, test, etc.) If you try to install hyper-v using the add roles / features wizard inside a HC3 VM - it will complain that the CPU isn't VM capable because we don't pass the VTx flags into the guest OS (by design). The workaround seems to be to install the Hyper-V role using DISM (which doesn't seem to check the CPU flags), then configure a virtual switch (using either powershell or Hyper-V manager GUI), THEN install docker for Windows (selecting the option prompted to use Windows Containers). I'll give some steps and screenshots below.

So step one would be to install Hyper-V role and tools needed to configure the virtual switch ... ( I expect there is a single step command to install both in one step)

https://us.v-cdn.net/6029942/uploads/editor/ss/n91k3dababmf.png

https://us.v-cdn.net/6029942/uploads/editor/dj/z6hholqkcmsv.png

Next step is to configure a virtual switch ... which I have done both using powershell and the Hyper-V manager

https://us.v-cdn.net/6029942/uploads/editor/el/lqyd0ivhcvk0.png

At some point you also need to enable the windows Containers feature as well but it doesn't seem to matter when or how. I've done it using the gui roles / feature wizard, you could do it via powershell, if you skip it and install docker for windows, at some point it will ask you to install it as well. The powershell command would be: Enable-WindowsOptionalFeature -Online -FeatureName containers -All

I don't know if it was required but I specifically selected to download and install from the Docker Edge Channel to get the latest features as of March 2018. At a point during the install I was asked whether I wanted to switch to use the built in Windows Container Support and I responded yes

https://us.v-cdn.net/6029942/uploads/editor/x2/xbeysqfnb9uj.png

After the install I was able to run the windows version of hello world and have also run the full microsoft/windowsservercore container with "powershell" command. I've also tried other windows based containers including SQL server 2016 ("docker search microsoft" is a good place to star

https://us.v-cdn.net/6029942/uploads/editor/7v/8n2387qyv73k.png

One capability available to windows containers on physical machines is instead of sharing the same base windows kernel, to launch a new kernel inside a hyper-v VM for greater isolation (also known as hyper-v containers.) Attempting to start a container with the --isolation=hyperv flag fails because that "level 2" VM can't be created using hyper-v.

C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: container 0b2c3ccb877d0f250cb2a03c00a909838f998f01d65b03a031255927a9faa6d6 encountered an error during CreateContainer: failure in a Windows system call: No hypervisor is present on this system

Trying to run Linux based docker containers on Windows also fails with various messages as expected.

As always - would love to hear from HC3 users about their thoughts / plans / use or questions around containers in general (hint: there are at least a few different possible future features I see relating to running on HC3 I can see here that we will be monitoring the demand for from our customer base)