So, it was Fedora Virtualization Test Day last Tuesday and I actually went down and took the occasion for some good testing of Xen on the next Fedora release (Fedora 20, codename Heisenbug). Fedora is going to ship Xen 4.3 (and there are not many other mainstream distribution doing that), so it is very important to try to make sure it will be as good as possible for Fedora users!
A lot of information on how to (well, how you should have… but it’ll be for next time ;-P) participate  to such event are available on our Wiki. What I am up to, here, is reporting how some of the tests I did that day went. Hopefully, this would give an idea of where we stand, regarding the integration of Xen in Fedora, as well as how well Xen itself works with Fedora’s default virtualization toolstack, i.e., libvirt.
Setting up the testing environment
Well, you at least need a Fedora 20 installation, in order to test Xen on Fedora 20. For the details, have a look at the already mentioned wiki page. Here I’m only going to say that I decided to go for a PXE-boot based install, which I did by downloading the following files:
$ wget http://dl.fedoraproject.org/pub/alt/stage/20-Beta-TC2/Fedora/x86_64/os/images/pxeboot/vmlinuz $ wget http://dl.fedoraproject.org/pub/alt/stage/20-Beta-TC2/Fedora/x86_64/os/images/pxeboot/initrd.img
and by preparing an appropriate entry in my PXE server configuration (usually a file called pxeconfig.cfg):
label fedora-20btc1-amd64-s
KERNEL fedora/20/x86_64/Beta-TC2/vmlinuz
APPEND initrd=fedora/20/x86_64/Beta-TC2/initrd.img repo=http://dl.fedoraproject.org/pub/alt/stage/20-Beta-TC2/Fedora/x86_64/os/ console=ttyS0,115200n8 text serial
Mind the console=ttyS0,115200n8 text serial in case you want to run the install on a serial console, like I’m doing in this case.
On a second test box, I did a proper graphical install (still via PXE). No big difference, really, just follow the guided procedure, then grab a coffe` and wait for this screen (on the right) to happear.
Installing Xen and rebooting into Dom0
After finishing installing the host, we need to install Xen, libvirt and some libvirt related tools. It’s all described in this other Wiki page, so let’s skip it here…. Just follow that instructions and reboot into the following:
Fedora release 20 (Heisenbug) Kernel 3.11.2-301.fc20.x86_64 on an x86_64 (hvc0) odyn login: root Password: # cat /etc/fedora-release Fedora release 20 (Heisenbug) # sudo uname -a Linux odyn.uk.xensource.com 3.11.3-301.fc20.x86_64 #1 SMP Thu Oct 3 00:57:21 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux # virt-what xen xen-dom0
Well, I guess we can make a note of the first important fact of the Test Day:
- Fedora 20 works quite nicely and straightforwardly as Xen Dom0!
Creating guests
Ok, let’s move forward to creating some guest. In Fedora, when you want to create and install a guest, especially a Fedora guest, you do it with virt-install. Period. So, let’s do that, a Fedora 20 PV guest, on a Fedora 20 Dom0, with virt-install, installed from the serial console too. It’s actually more easily done than said:
# lvcreate -nf20_64 -L10G /dev/fedora_odyn # virt-install --paravirt --name f20_64 --ram 2048 --vcpus 4 -f /dev/fedora_odyn/f20_64 --network bridge=virbr0 --location http://dl.fedoraproject.org/pub/alt/stage/20-Beta-TC2/Fedora/x86_64/os/ --graphics none
=====================================================================
Installation
1) [x] Installation source 2) [!] Timezone settings
(http://dl.fedoraproject.org/pu (Timezone is
b/alt/stage/20-Beta-TC1/Fedora/ not set.)
x86_64/os/) 4) [!] Set root password
(Password is not set.)
3) [!] Install Destination 6) [!] Software selection
(No disks selected) (GNOME Desktop)
5) [!] Create user
(No user will be created)
7) [x] Network settings
(Wired (eth0) connected)
Please make your choice from above ['q' to quit | 'c' to continue |
'r' to refresh]:
[anaconda] 1:main* 2:shell 3:log 4:storage-log 5:program-log
Let’s now head to my second test box, and do something similar, which lead us where this screenshot shows:
I also created another PV guest and an HVM guest there, with similar procedures. From all this, we can reasonably assess the following:
- Fedora 20 works fine both as a PV and HVM Xen guest.
Playing with the guests with virsh
Now, what about, seeing what we have running:
# virsh list Id Name State ---------------------------------------------------- 39 F20-HVM running 40 fedora20 running
Pausing and resuming both the PV and HVM guests:
# virsh suspend F20-HVM Domain F20-HVM suspended # virsh suspend fedora20 Domain fedora20 suspended # virsh list Id Name State ---------------------------------------------------- 39 F20-HVM paused 40 fedora20 paused # virsh resume F20-HVM Domain F20-HVM resumed # virsh resume fedora20 Domain fedora20 resumed # virsh list Id Name State ---------------------------------------------------- 39 F20-HVM running 40 fedora20 running
And, finally, saving-&-restoring one of them
# virsh save fedora20 /tmp/savefile Domain fedora20 saved to /tmp/savefile # virsh list Id Name State ---------------------------------------------------- 39 F20-HVM running # virsh restore /tmp/savefile Domain restored from /tmp/savefile # virsh list Id Name State ---------------------------------------------------- 39 F20-HVM running 41 fedora20 running
I also tried importing and cloning a VM, as described here and here, and it all worked.
Any issues, then? There indeed was one. Basically, it looks like reaching the guest’s PV console via virsh does not work, while it is fine with xl console:
# virsh console fedora20 Connected to domain fedora20 Escape character is ^] error: internal error: cannot find character device (null) # xl console fedora20 Fedora release 20 (Heisenbug) Kernel 3.11.2-301.fc20.x86_64 on an x86_64 (hvc0) fedora20 login: root Password: [root@fedora20 ~]#
And I will of course report that to the appropriate mailing list/bugzilla.
What’s there, what’s missing
The previous section reveals that not only Xen is straightforward to install and works quite well on Fedora 20 as a Dom0, and that Fedora 20 works quite well as a Xen PV or HVM guest. It also shows how the basic VM lifecycle of a Xen guest, in Fedora 20, can be handled nicely enough with libvirt and the related tools (virt-install, virt-manager, virt-viewer, etc.). That of course does not exclude the possibility of using Xen’s default command line toolstack (xl).
The only two relevant missing features, at the time of writing, in the libvirt libxl driver are:
- PCI Passthrough
- live migration
Yes, big ones, I know. However, consider the following:
- that does not mean that PCI Passthrough and migration does not work on Xen on Fedora at all. They do work via the xl toolstack, they are just not available via libvirt;
- this is going to be solved soon, as the libvirt libxl driver maintainer Jim Fehling reported recently on xen-devel. In fact, this is the patch series for PCI Passthrough, and this is the patch series implementing live migration, and there are pretty good chances that both these patches make it in libvirt before Xen 4.4 release time (so, not in time for Fedora 20, but still not bad at all).
So, stay tuned since, as Jim says, “Slowly, with each libvirt release, the libxl driver is improving”.
Conclusions
Fedora really does a great job with these Test Days. All of it: the planning, the managing, the reporting… An example that many other project should look at and try to follow (and actually, Xen Project is trying, as we also started having Xen Test Days).
Participating to the latest Fedora Virtualization Test Day has been really nice, although we need to do a better job in convincing more Xen folks to be there and do some Xen specific testing. Anyway, I am really glad to have had the chance to verify how well Xen 4.3 will work on Fedora 20.
It is actually quite important that we get a good Xen on Fedora test coverage, at least as far as running the next release of Fedora as a Xen DomU is concerned. In fact, being a functional Xen guest is one of the release blockers for a Fedora release, as in, if release X does not work as a Xen guest, it can’t be released!
Since testing Xen on Fedora is, for the most part, testing Xen integration with libvirt, what about producing some libvirt test-cases for OSSTest? That would be very cool, and we are already working on it. Another interesting thing would be to also have OSSTest could try to build and run Xen on various distro (as host), instead than using only Debian, as it is doing right now. This is a bit more tricky than the above, but we are thinking at how to do that too (standalone mode could, perhaps, help).