SmartOS as a Home Router
Posted by Dave Eddy on May 31 2015 - tags: techFor the last couple of years I’ve used PFSense as my home router. It’s been great - it’s easy to manage with the web interface, and really easy to add features like an OpenVPN server, bandwidth monitoring, etc.
But, I like to manage all of my servers and zones at home using Chef, and my router has always been left out as it required being configured manually through the web interface. So now, I’ve replaced PFSense with 2 SmartOS zones: 1 for NAT and the other for DHCP - both managed by Chef, and both monitored with Nagios. This also had the added effect of reducing two physical servers at home down to one, for a cheaper power bill.
There are 3 steps required to configure a SmartOS server as a home router replacement.
- Add the External Interface
- Create the NAT Zone
- Create the DHCP Zone
1. Add the External Interface
The first thing is to ensure that you have 2 (or more) NICs on the SmartOS server - one for the external network (The Internet) and one for the internal network that will be behind NAT. To list the interfaces run
root - datadyne sunos ~ # dladm show-phys -m
LINK SLOT ADDRESS INUSE CLIENT
rge0 primary f4:6d:4:X:XX:XX yes rge0
e1000g0 primary 0:4:23:XX:XX:XX yes e1000g0
Cross-referencing this information with output from ifconfig(1M) reveals that
the e1000g0 interface is not currently in use, and can be used as the external
interface.
root - datadyne sunos ~ # ifconfig e1000g0
ifconfig: status: SIOCGLIFFLAGS: e1000g0: no such interface
Add the following line to /usbkey/config to put this interface on a tag called
external - modify the mac address to match the output from the dladm command
above
external_nic=0:4:23:XX:XX:XX
At this point I rebooted the hypervisor for the change to take effect, but I believe running the following will do the same thing without rebooting the machine.
sysinfo -u
You can verify this worked by running
root - datadyne sunos ~ # nictagadm list
NAME MACADDRESS LINK TYPE
external 00:04:23:XX:XX:XX e1000g0 normal
admin f4:6d:04:XX:XX:XX rge0 normal
You should see an external NIC tag with the same interface name and mac address
if everything went right.
Now that the NIC has been added and a tag is ready, a new zone must be created
with VNICs on both the external and the internal networks (in my case, the
external and admin networks)
2. Create the NAT Zone
To create the NAT zone you can use this JSON
{
"brand": "joyent",
"image_uuid": "c02a2044-c1bd-11e4-bd8c-dfc1db8b0182",
"autoboot": true,
"alias": "nat",
"hostname": "nat.rapture.com",
"dns_domain": "rapture.com",
"resolvers": [
"208.67.222.222",
"208.67.222.220"
],
"max_physical_memory": 512,
"nics": [
{
"nic_tag": "admin",
"ip": "10.0.1.1",
"gateway": "10.0.1.1",
"netmask": "255.255.255.0",
"allow_ip_spoofing": true,
"primary": true
},
{
"nic_tag": "external",
"ip": "dhcp",
"allow_ip_spoofing": true
}
]
}
And run
vmadm create -f nat.json
In the nics section of the config there are 2 VNICs defined, one on the
internal (admin) network with a static IP set, and the other on the external
network with DHCP set - as my ISP provides a dynamic IP. Both VNICs require
allow_ip_spoofing to be enabled for NAT to work properly.
vmadm(1M) creates the VNICs in the order they are given, so they will be seen
on the zone as
net0: the internal interfacenet1: the external interface (The Internet)
Login to the zone
zlogin "$(vmadm list -o uuid -H alias=nat)"
Verify that the interfaces are setup correctly
dave - nat sunos ~ $ ifconfig net0
net0: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 2
inet 10.0.1.1 netmask ffffff00 broadcast 10.0.1.255
dave - nat sunos ~ $ ifconfig net1
net1: flags=1104843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,ROUTER,IPv4> mtu 1500 index 3
inet 72.231.XXX.XXX netmask fffffe00 broadcast 72.231.XXX.XXX
net0 should have the static IP set from the JSON config, and net1 should be
configured by DHCP from your ISP.
Create a config for ipnat(1M)
cat <<-EOF > /etc/ipf/ipnat.conf
# NAT
map net1 10.0.1.0/24 -> 0/32 proxy port ftp ftp/tcp
map net1 10.0.1.0/24 -> 0/32 portmap tcp/udp auto
map net1 10.0.1.0/24 -> 0/32
# Port Forwards
# rdr net1 0/0 port 80 -> 10.0.1.5 port 8080 tcp
EOF
The first 3 lines of the config will setup ipnat(1M) to NAT from the internal
interface to the external one. The last line (commented out) will create a
port forward from the external interface on port 80 over TCP, to the internal
host 10.0.1.5 on port 8080 as an example.
To enable the service, run
routeadm -u -e ipv4-forwarding
svcadm enable ipfilter
At this point, any machine on your internal network with this zones IP set as its default gateway will be able to reach the internet.
3. Create the DHCP Zone
To create the DHCP zone you can use this JSON
{
"brand": "joyent",
"image_uuid": "c02a2044-c1bd-11e4-bd8c-dfc1db8b0182",
"autoboot": true,
"alias": "dhcp",
"hostname": "dhcp.rapture.com",
"dns_domain": "rapture.com",
"resolvers": [
"208.67.222.222",
"208.67.222.220"
],
"max_physical_memory": 512,
"nics": [
{
"nic_tag": "admin",
"ip": "10.0.1.4",
"allow_dhcp_spoofing": true,
"netmask": "255.255.255.0",
"gateway": "10.0.1.1",
"primary": true
}
]
}
And run
vmadm create -f dhcp.json
In the nics section of the config there is 1 VNIC defined on the
internal (admin) network with a static IP set. The setting
allow_dhcp_spoofing must be enabled in order for the DHCP server
to work as expected.
Login to the zone
zlogin "$(vmadm list -o uuid -H alias=dhcp)"
Install the isc-dhcpd package
pkgin up
pkgin in isc-dhcpd
Create a config for dhcpd, modifying it as necessary to fit your environment
cat <<-EOF > /opt/local/etc/dhcp/dhcpd.conf
default-lease-time 600;
max-lease-time 7200;
option subnet-mask 255.255.255.0;
option broadcast-address 10.0.1.255;
option routers 10.0.1.1;
option domain-name-servers 10.0.1.2, 10.0.1.3;
option domain-name "rapture.com";
subnet 10.0.1.0 netmask 255.255.255.0 {
range 10.0.1.200 10.0.1.250;
}
EOF
Finally, start the service
svcadm enable isc-dhcpd
Conclusion
With this all setup, any host on your network that requests DHCP will
be answered by the dhcp zone, which will tell the host to use the nat
zone as the default gateway.
In other words, with just 2 low-powered zones, any existing SmartOS server can replace a dedicated home/NAT router on a network.
Notes / Tips
A lot of this blog post was based around this incredibly helpful SmartOS Wiki page
- https://wiki.smartos.org/display/DOC/NAT+using+Etherstubs
You can view statistics for NAT by running
ipnat -s
ipnat -l
If you modify /etc/ipf/ipnat.conf and want the changes to take effect without
restarting the service you can run
ipnat -FC -f /etc/ipf/ipnat.conf
On the dhcp zone, you can look at the current leases by running
cat /var/db/isc-dhcp/dhcpd.leases
Alternatively, check out the DHCPD Dashboard
I wrote to create a web interface that shows all of the current DHCP leases on
the dhcp zone
- https://github.com/bahamas10/node-dhcpd-dashboard
Monitoring
I’ve created a Nagios style script to check the current number of NAT mappings on the nat
zone to alert if the percentage is over a certain threshold
$ ./check_ipnat_mappings
ok: 347 / 30000 in use - 1% total|inuse=347;max=30000;perc=1
#!/usr/bin/env bash
#
# nagios check for ipnat mappings inuse
#
# Author: Dave Eddy <dave@daveeddy.com>
# Date: May 19, 2015
# License: MIT
warning=80
critical=90
while getopts 'w:c:' option; do
case "$option" in
w) warning=$OPTARG;;
c) critical=$OPTARG;;
esac
done
ipnats=$(ipnat -s)
if (($? != 0)); then
echo "unknown: failed to call ipnat(1M)"
exit 3
fi
inuse=$(echo "$ipnats" | awk '/^inuse/ { print $2 }')
max=$(ipf -T ipf_nattable_max | awk '{ print $NF }')
perc=$((inuse * 100 / max))
if [[ -z $perc ]]; then
echo 'unknown: error retrieving data'
exit 3
elif ((perc >= critical)); then
msg='critical'
ret=2
elif ((perc >= warning)); then
msg='warning'
ret=1
else
msg='ok'
ret=0
fi
echo "$msg: $inuse / $max in use - $perc% total|inuse=$inuse max=$max perc=$perc%"
exit "$ret"
