From time to time we face requests to correctly enable support for various Xen features on PC Engines apu2 platform. Doing that requires firmware modification, which 3mdeb is responsible for.

Xen have very interesting requirements from firmware development perspective. Modern x86 have a bunch of features that support virtualization in hardware. Those features were described in Xen FAQ.

We used to receive many requests about IOMMU and SR-IOV. First, give the ability to dedicate PCI device to given VM and second enables so-called Virtual Functions, what means on a physical device (e.g. Ethernet NIC) can be represented by many PCI devices. Connecting IOMMU with SR-IOV give the ability for hardware-assisted sharing of one device between many VMs.

All those features are very nice and there is work spread on various forums, which didn’t get its way to mainline yet. Starting with this blog post we want to change that.

To start any work in that area we need a reliable setup. I had a plan to build something pretty simple using our automated testing infrastructure. Unfortunately, this has to wait a little bit since when I started this work I had to play with a different configuration.

If you don’t have PXE, DHCP (if needed) and NFS set up I recommend to read above blog post or just use pxe-server and dhcp-server.

Xen installation in Debian stable

I assume you have PXE+NFS boot of our Debian stable. To netboot simply enter iPXE:

And choose Debian stable netboot 4.14.y:

After boot, you can log in with presented credentials [root:debian]:

Xen installation in Debian is quite easy and there is community website describing the process, but to quickly dive in:

xencall error

I took a break from Xen debugging and found that after upgrading kernel and rootfs I’m getting below error message:

I’m not a Xen developer and it looked pretty cryptic to me. It happens that xen.service also fails to run:

It happen that during upgrading of my rootfs I forget to install all required packages to Xen rootfs directory. So, now you should not face this problem when using pxe-server, but if you see something similar please make sure you have all modules correctly loaded or compiled in. You can check my working kernel config

Xen boot log

Below boot log analysis was performed on v4.6.9 release candidate.

The thing that we are concerned about and want to fix is

There are some patches pending to enable IOMMU. Of course, enabling this features open new universe with various advanced virtualization features which we hope to discuss in further blog posts.

Trying Xen boot params

I tried to use iommu=on amd_iommu=on which doesn’t change anything with firmware not-IOMMU capable.


In further posts, I would like to get through IOMMU enabling by leveraging great community work from Kyosti and Timothy. Also, I would like to exercise and prove various virtualization features of PC Engines apu2. If you are interested in commercial enablement of advanced SoC features feel free to let us know at Also feel free to contribute to pxe-server mini-project as well as comment below.