Blog

Simple web and email hosting

Jail networking on a VPS


The easiest way to provide networking to jails is by providing it an IP address via ip4.addr. This however requires you to own those additional IP addresses, which is often not the case when using a virtual private server (VPS). The below example illustrates this easy way of working:

microblog {
   ip4.addr = "em0|203.0.113.80";
}

The method I will be using on my VPS instances I got from martin on the FreeBSD forums. It consists of the following steps:

  1. Create a unique loopback interface for the jails to use

  2. Allocate a private IP network range to be used by all the jails on this loopback interface

  3. Create a firewall rule to perform network address translation (NAT) on all traffic from this private IP range to the outside world.

  4. Next you can create jails and assign them IP addresses from this IP range.

The actual commands

Next we will cover these different topics in more depth with examples.

1. Unique loopback address

Edit /etc/rc.conf and add the following lines:

cloned_interfaces="lo100"
ifconfig_lo100_alias0="inet 10.1.0.1 netmask 255.255.255.255"

Next create the interface manually (will be done automatically at startup):

# ifconfig lo100 create

2. Allocate an IP range

This is easy, just pick something which is not already used on your network, e.g. 10.1.0.0/24.

3. Create the needed firewall rules

Define a micro containing the previously allocated IP range and add a NAT rule:

intf="em0"
net_jail="10.1.0.0/24"
set skip on { lo0 lo100 }
nat pass on $intf from $net_jail to any -> ($intf)

It is important here to put the macros at the beginning of your configuration file and the NAT rule after the queueing rules and before the packet filtering rules, see the man page.

Also note that for testing purposes (that's what they all say, no?) I have excluded the lo100 interface from the firewall rules. I really should tighten the security here.

4. Create and start a jail

To create a new jail I use the following command (yes, I should really update my base jail).

# zfs send -R zroot/var/jails/_base@11.2-base | zfs receive zroot/var/jails/nattest

Create a configuration file for the jail in /etc/jail.conf:

nattest {
   ip4.addr = "10.1.0.50";
   allow.raw_sockets;
}

Again ignoring security by enabling raw sockets for testing purposes.

# service jail start nattest
# jexec nattest /bin/tcsh
nattest:/ # ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
     options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
     ether 28:f1:0e:0c:1a:6a
     hwaddr 28:f1:0e:0c:1a:6a
     media: Ethernet autoselect
     status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
     options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
     groups: lo
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
     ether e4:a4:71:30:95:0d
     hwaddr e4:a4:71:30:95:0d
     media: IEEE 802.11 Wireless Ethernet OFDM/54Mbps mode 11g
     status: associated
ifconfig: unknown/undefined channel number 6 flags 0x0nattest:/ #
nattest:/ # netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
10.1.0.50          link#6             UH        lo100

The output from these two commands is quite weird. The IP address is not seen in the output of ifconfig although it might be related to me using a wireless network card and the jail does not know anything about that configuration (which also gives an error message in the output). The jail also does not have a default gateway configured. This is a testament to it being a jail and not a full virtual machine.

Conclusion

With the method described here I can host jailed services on a VPS. Well, the basic proof-of-concept is here. I will still need to verify inbound traffic works. I will also need to look into inter-jail communications. This might be needed when I want a Postfix jail delivering the email to a Dovecot jail or a Roundcube jail fetching the emails from the Dovecot jail. I will keep those investigations for another blog post though so you have something to look forward to.