Isolation is the gift.

My latest KVM trick was to build a completely isolated KVM virtual machine. I can't reach it from my LAN, and it can't reach my LAN, but it can access the outside world. I had to get my DD-WRT router involved with this as all my attempts to isolate the machine with just iptables rules on the KVM host failed utterly. (So there's a game I lost.)

Update: The DD-WRT info below is retained for hysterical raisins, but I've gotten a new Buffalo WZR-600DHP, and the hardware it uses can't do tagging on individual ports, so I had to invent a new scheme.

The DD-WRT router allows me to define VLANs (virtual LANs), and I can set a port on the router (I chose port 1) to get VLAN tagged packets on the traffic on that port. So in my case, I can have traffic for two different subnets traveling along the same wire. To take advantage of this, the KVM host machine needs to be connected directly to port 1 on the router, and I need to define the virtual LANs on the host machine network setup so it will understand what to do with them. Doing all that on the host machine (running fedora 16 at the moment), gives me these network setup files:

/etc/sysconfig/network-scripts/ifcfg-em1

DEVICE="em1"
ONBOOT=yes
NM_CONTROLLED="no"

/etc/sysconfig/network-scripts/ifcfg-em1.1

DEVICE="em1.1"
ONBOOT=yes
NM_CONTROLLED="no"
HWADDR=74:d0:2b:2b:7a:b4
TYPE=Ethernet
DEFROUTE=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
USERCTL=no
BRIDGE=br0
PEERDNS=no
NAME="em1.1"
VLAN=yes

/etc/sysconfig/network-scripts/ifcfg-em1.3

DEVICE="em1.3"
ONBOOT=yes
NM_CONTROLLED="no"
HWADDR=74:d0:2b:2b:7a:b4
TYPE=Ethernet
DEFROUTE=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
USERCTL=no
BRIDGE=bifrost
PEERDNS=no
NAME="em1.3"
VLAN=yes

/etc/sysconfig/network-scripts/ifcfg-br0

# Networking Interface
DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
GATEWAY=192.168.1.1
IPADDR=192.168.1.106
IPV6INIT=no
NETMASK=255.255.255.0
ONBOOT=yes
PEERDNS=no
USERCTL=no
NM_CONTROLLED=no

/etc/sysconfig/network-scripts/ifcfg-bifrost

# Networking Interface
DEVICE=bifrost
TYPE=Bridge
BOOTPROTO=static
GATEWAY=192.168.2.1
IPADDR=192.168.2.99
IPV6INIT=no
NETMASK=255.255.255.0
ONBOOT=yes
PEERDNS=no
USERCTL=no
NM_CONTROLLED=no

Definitely complicated, but it has grown over time. The em1 name is the newfangled name fedora 16 gave my NIC interface. The br0 name is the primary bridge I use for most of my virtual machines and was just copied from examples. The bifrost name is the name I decided to give to my new bridge that has the em1.3 VLAN attached to it.

To go with all this, I needed to setup the VLAN definitions in the router.

On the DD-WRT router, on the Setup/VLANs page, I clicked the Tagged checkbox for the column for Port 1. Then I clicked the VLAN 3 row's checkbox also for the column for port 1. That winds up looking like this.

On the Setup/Networking page in the Port Setup section, I clicked the Unbridged button for the Network Configuration vlan3 line. That then exposes the network setup parameters where I set IP Address to 192.168.2.1 and Subnet Mask to 255.255.255.0. (I left the other settings with their default values: MTU 1500, Multicast forwarding disabled, Masquerade / NAT disabled).

Then down to the next section on the page, DHCPD, I clicked the Add button to add a new dhcp server, set it to work on vlan3, and left the other parameters alone (start 100, max 50, leasetime 3600). The resulting config screen looks like this.

Rebooting my KVM host, I get both bridges on the two different VLAN subnets, and they all seem to work. I can now connect the virtual machine I want to isolate to the bifrost bridge, and it gets assigned an address on the 192.168.2.0/24 net by the DHCP server on the router.

Unfortunately, the machine isn't actually isolated yet. I need one more gimmick on the router. In the Administration/Commands page, I need to type these lines in the command area and click on Save Firewall:

iptables -I FORWARD -s 192.168.2.0/24 -d 192.168.1.0/24 -j DROP
iptables -I FORWARD -s 192.168.1.0/24 -d 192.168.2.0/24 -j DROP

Now I have firewall rules in the router that prevent any traffic from leaking back and forth between my two subnets, yet both subnets can still access the internet at large. My isolation is complete!

I also run the virtual machine with a qcow2 image file that has a backing file that is a read only copy of the uncorrupted machine. So if any unsafe browsing or installing should wind up corrupting the machine, a simple qemu-img command will create a fresh image from the pure backing file, and everything is back to normal.

New Scheme

I had to get rid of the vlans, em1.1, and em1.3 and go back to plain old em1 attached to br0, with the bifrost bridge attached to nothing. That allows my host to boot and talk to the internet OK, but now the isolated KVM has no network at all.

I then proceeded to build a virtual router. This seems like too much work to me, but I still can't understand how to do what I want with just iptables on the host machine. What I've done now is install a very small fedora 20 virtual machine with two virtio NICs. One is connected to br0, the other to bifrost.

Inside that machine I configured it as a NAT router and added the same iptables rules above to block the local LAN from access to the bifrost LAN (and vice-versa).

Now I can bring up the firewall KVM first, then bring up the isolated KVM and once again it can access the internet at large, but not my local LAN.