When testing new protocols on my test-bed routers, I often need to generate traffic through those routers. Since routers are meant to route, and not to generate traffic, I use my laptop to push traffic. To make it even more convenient, I like to receive the routed traffic on the very same laptop (e.g. when using iperf or similar). But when the traffic is sent to the local host, how does one make it leave the system? The answer iptables...
There are a two problems that must be solved:
- Outgoing packets with a local IP address as destination are sent on the loopback interface.
- Incoming packets with a local IP address as source are dropped.
To solve these issues, we "invent" two remote addresses and use iptables to put these addresses in the IP header. The process is something like this:
- When sending to a fake address, change the source address to the fake address.
- When receiving from a fake address, change the destination address to a local address.
Let me illustrate this:
The concept might seem a bit hard to grasp, and it certainly is difficult to explain, but let me try: iptables is used to change the source address before the packet leaves the computer, and when it arrives back again (using hard-coded neighbor entries), we use iptables to change the fake destination address to a local one. Should the receiver choose to reply, it will read the source of the packet (which we changed to the other fake address) and the whole process is repeated with the two fake addresses exchanged.
But enough talk! Let's look at how all this is really done. As always, I cooked a script to do the work, so lets have a look:
#!/bin/bash
unload=$1
iface_one=eth1
iface_two=eth2
local_one=10.0.0.1
local_two=10.0.0.2
remote_one=10.0.0.101
remote_two=10.0.0.102
mac_one=$(cat /sys/class/net/$iface_one/address)
mac_two=$(cat /sys/class/net/$iface_two/address)
# iptables rules
ipt_src_one="POSTROUTING -s $local_one/32 -d $remote_two/32 -o $iface_one -j SNAT --to-source $remote_one"
ipt_src_two="POSTROUTING -s $local_two/32 -d $remote_one/32 -o $iface_two -j SNAT --to-source $remote_two"
ipt_dst_one="PREROUTING -s $remote_two/32 -d $remote_one/32 -i $iface_one -j DNAT --to-destination $local_one"
ipt_dst_two="PREROUTING -s $remote_one/32 -d $remote_two/32 -i $iface_two -j DNAT --to-destination $local_two"
# routes
route_one="$remote_two dev $iface_one"
route_two="$remote_one dev $iface_two"
# neighbors
neigh_one="$remote_one dev $iface_two lladdr $mac_one"
neigh_two="$remote_two dev $iface_one lladdr $mac_two"
# remove old iptables rules
ipt=$(sudo iptables-save)
if echo $ipt | grep -q "$ipt_src_one"; then
echo "delete iptables $ipt_src_one"
sudo iptables -t nat -D $ipt_src_one
fi
if echo $ipt | grep -q "$ipt_src_two"; then
echo "delete iptables $ipt_src_two"
sudo iptables -t nat -D $ipt_src_two
fi
if echo $ipt | grep -q "$ipt_dst_one"; then
echo "delete iptables $ipt_dst_one"
sudo iptables -t nat -D $ipt_dst_one
fi
if echo $ipt | grep -q "$ipt_dst_two"; then
echo "delete iptables $ipt_dst_two"
sudo iptables -t nat -D $ipt_dst_two
fi
# remove old routes
routes=$(sudo ip route)
if echo $routes | grep -q "$route_one"; then
echo "delete route $route_one"
sudo ip route del $route_one
fi
if echo $routes | grep -q "$route_two"; then
echo "delete route $route_two"
sudo ip route del $route_two
fi
# remove old neighbors
neighs=$(sudo ip neigh)
if echo $neighs | grep -q "$neigh_one"; then
echo "delete neigh $neigh_one"
sudo ip neigh del $neigh_one
fi
neighs=$(sudo ip neigh)
if echo $neighs | grep -q "$neigh_two"; then
echo "delete neigh $neigh_two"
sudo ip neigh del $neigh_two
fi
neighs=$(sudo ip neigh)
if echo $neighs | grep -q "$remote_two dev $iface_one"; then
echo "flush neighbor $remote_two dev $iface_one"
sudo ip link set dev $iface_one arp off
sudo ip link set dev $iface_one arp on
fi
neighs=$(sudo ip neigh)
if echo $neighs | grep -q "$remote_one dev $iface_two"; then
echo "flush neighbor $remote_one dev $iface_two"
sudo ip link set dev $iface_two arp off
sudo ip link set dev $iface_two arp on
fi
if [ "$unload" != "" ]; then
exit 0
fi
# set routes
echo "add route $route_one"
sudo ip route add $route_one || exit 1
echo "add route $route_two"
sudo ip route add $route_two || exit 1
# add neighbor entries
echo "add neigh $neigh_one"
sudo ip neigh add $neigh_one || exit 1
echo "add neigh $neigh_two"
sudo ip neigh add $neigh_two || exit 1
# fake source addresses of outgoing packets
echo "add iptables $ipt_src_one"
sudo iptables -t nat -A $ipt_src_one || exit 1
echo "add iptables $ipt_src_two"
sudo iptables -t nat -A $ipt_src_two || exit 1
# replace incoming destination address to match local address
echo "add iptables $ipt_dst_one"
sudo iptables -t nat -A $ipt_dst_one || exit 1
echo "add iptables $ipt_dst_two"
sudo iptables -t nat -A $ipt_dst_two || exit 1
echo "done"
The script is pretty long, but this is mostly because I failed to count the real way (like: 0, 1, many)... The script works like this:
After the configuration, we declare the rules for iptables, because we need these to first remove any existing rules and to create the rules afterwards. The same goes for the routing entries and the neighbor lines.
Then we check for existing rules in iptables and removes these if needed. Again, we repeat for routes and neighbors. I had to toggle the arp-switch to actually remove existing neighbor entries. Don't ask me why...
With everything cleared, we can start adding stuff. First we tell the system to use eth1 to reach 10.0.0.102 and eth2 to reach 10.0.0.101:
sudo ip route add 10.0.0.101 dev eth2
sudo ip route add 10.0.0.102 dev eth1
But the system cannot send packets to either10.0.0.101 or 10.0.0.102 without knowing the corresponding MAC addresses. Usually we would let ARP resolve these, but since our fake addresses doesn't exist, we much tell these manually:
sudo ip neigh add 10.0.0.101 dev eth2 lladdr $mac_one
sudo ip neigh add 10.0.0.102 dev eth1 lladdr $mac_two
With this in place, we are all set to handle packets with fake addresses, so next up is to actually use these fake addresses:
sudo iptables -t nat -A POSTROUTING -s 10.0.0.1/32 -d 10.0.0.102/32 \
-o eth1 -j SNAT --to-source 10.0.0.101
sudo iptables -t nat -A POSTROUTING -s 10.0.0.2/32 -d 10.0.0.101/32 \
-o eth2 -j SNAT --to-source 10.0.0.102
These two rules works like this: after the routing subsystem has consulted the routing tables, we check for a source address matching our local IP, and a destination address matching our fake IP, and make sure we are the packet is sent on the correct interface. If these three conditions are met, we override the source address with the correct fake address.
Finally, we have to make sure that, when the mangled packet is received again, it will not be dropped This is achieved with the following rules:
sudo iptables -t nat -A PREROUTING -s 10.0.0.102/32 -d 10.0.0.101/32 \
-i eth1 -j DNAT --to-destination 10.0.0.1
sudo iptables -t nat -A PREROUTING -s 10.0.0.101/32 -d 10.0.0.102/32 \
-i eth2 -j DNAT --to-destination 10.0.0.2
Done! To use the script, plug two ethernet cables into the computer plug the cables into a switch. Then configure the interfaces with the right addresses (eth1: 10.0.0.1 and eth2: 10.0.0.2). The run the script, and test it all by pinging one of the fake addresses:
ping 10.0.0.102
If you run tcpdump on eth1, you should see ICMP requests with 10.0.0.101 as source and 10.0.0.102 as destination, and ICMP replies with the inverse.
Happy routing!