At some point I stuck in the forest with WiFi connection and no physical access to router to create nice networking for my coreboot development needs. Recently I switched my laptop to Qubes OS what give interesting flexibility, but also additional problems.

My key requirement is to boot system over PXE, so I can easily do kernel development and play with Xen. Because only available connection for my apu2 platform was directly to my laptop I had to provide configured DHCP server and PXE server on it. Qubes OS networking is quite complex and to get to VM you have to pass-through at least sys-net VMs. Those VMs requires iptables configuration to correctly pass traffic or some tricks as I presented below.

I don’t think much people will face so weird configuration, but I need following notes for myself and there is some chance that someone will face similar issues.

To summarize my target configuration was like that:

qubes-apu2-setup

My initial idea was to have servers on AppVMs, but I didn’t have enough time to get through Qubes OS iptables rules. That led to discover interesting alternative with proxychains, which I will describe later in this article.

Qubes OS network configuration

Let’s start with putting together DHCP server:

Please change your network interface in start.sh it doesn’t match. Currently set is eno1 what may be good for Ubuntu users.

The only port that we have to forward for DHCP is 67.

sys-net setup

My routing table look like that:

wls6 is my wireless interface. apu2 is connected over Ethernet cable using ens5 interface. Let’s assign static IP to it:

Routing was added automatically:

After trying to correctly setup iptables in Qubes OS to forward traffic to vm where DHCP and PXE/NFS containers were started I decided to give up. It would be much easier to correctly setup sys-net for my development needs then spending hours on figuring out what is wrong with my IP tables.

Adjust your dhcp.conf and start.sh to network configuration. In my case it was modified like below:

and

On apu2 I booted to iPXE. I’m using v4.6.3.

Please note that MAC of my apu2 was already added to dhcp.conf.

From that place I can go to run my PXE and NFS container.

PXE and NFS server

On iPXE side:

This gives couple options during boot:

Probably more will be available overtime.

Qubes OS ssh reverse tunnel and port forwarding

I had to resolve that problem just because of my lack of deep understanding of iptables and ability to reconfigure Qubes OS sys-net routing to handle that case. On the other hand below exercise was very engaging and for sure this solution can be used in some situations in future.

Problem is that my apu2 192.168.42.101 cannot access outside world. This is because its only connection is to my laptop Ethernet port which is managed by sys-net VM and bunch of iptables rules. Flushing whole iptables configuration was not a solution, so I figured out how to create reverse ssh tunnel and use it to proxy whole traffic from apu2.

The solution came with this stackoverflow answer. What we doing here is setting up SOCKS proxy and reverse SSH tunnel for apu2 traffic. On sys-net I did:

Then on apu2:

Please note that if, for some reason connection on sys-net will break then you will have problem resolving DNS. To fix that you have to remove incorrect default gateway. This have to be automated somehow on sys-net:

What we can do now ?

You can use that configuration for many purposes, but my idea was to have Xen dom0 booting over PXE and NFS. I will describe that in other blog post.

Summary

I’m huge fan of Qubes OS and its approach to security. Unfortunately security typically came with less convenience, what can be problem in some situations. Nevertheless if you face some problems with Qubes OS, you need configuration or enabling support or you are interested in freeing your hardware setup, please do not hesitate to contact us.

If you know how to reliably setup iptables in above situation we would be glad to test it.