To test a multipath application that I am currently working on, I need to setup my laptop with two active network interfaces. As always, I cooked a script to do the work, which uses NetworkManager to read the needed information and iproute2 to setup the routing table(s):
#!/bin/bash
function oct2cidr () {
local mask bit cidr i
mask=$1
if grep -qv '\.' <<<$mask; then
echo $mask
return
fi
for i in 1 2 3 4; do
bit=${bit}$(printf "%08d" \
$(echo 'ibase=10;obase=2;'$(cut -d '.' -f $i <<<$mask) |bc))
done
cidr=$(echo -n ${bit%%0*} |wc -m)
echo $cidr
}
function read_info() {
nminfo=$(nmcli device list iface $1)
iface=$(echo -e "$nminfo" | grep "GENERAL.IP-IFACE" | cut -f2 -d:)
addr=$(echo -e "$nminfo" | grep ip_address | cut -f2 -d=)
gw=$(echo -n "$nminfo" | grep " routers " | cut -f2 -d=)
net=$(echo -n "$nminfo" | grep network_number | cut -f2 -d=)
mask=$(echo -n "$nminfo" | grep " subnet_mask " | cut -f2 -d=)
cidr=$(oct2cidr $mask)
echo $iface
echo $addr
echo $gw
echo $net/$cidr
}
function setup_table() {
table=$1
sudo ip route flush table $table
sudo ip route del $net/$cidr
sudo ip route add $net/$cidr dev $iface src $addr
sudo ip route add $net/$cidr dev $iface src $addr table $table
sudo ip route add default via $gw dev $iface table $table
sudo ip rule add from $addr table $table prio $prio
sudo ip route add 127.0.0.0/8 dev lo table $table
prio=$(echo $prio - 1 | bc)
}
function setup_balance() {
nminfo=$(nmcli device list iface $1)
gw1=$(echo -n "$nminfo" | grep " routers " | cut -f2 -d=)
if1=$(echo -e "$nminfo" | grep "GENERAL.IP-IFACE" | cut -f2 -d:)
nminfo=$(nmcli device list iface $2)
gw2=$(echo -n "$nminfo" | grep " routers " | cut -f2 -d=)
if2=$(echo -e "$nminfo" | grep "GENERAL.IP-IFACE" | cut -f2 -d:)
sudo ip route del default
sudo ip route add default scope global \
nexthop via $gw1 dev $if1 weight $3 \
nexthop via $gw2 dev $if2 weight $4
}
function flush_rule() {
prio=32767
sudo ip rule flush
sudo ip rule add from all lookup default prio $prio
prio=$(echo $prio - 1 | bc)
sudo ip rule add from all lookup main prio $prio
prio=$(echo $prio - 1 | bc)
}
flush_rule
dev1=$1
dev2=$2
read_info $dev1
setup_table t1
echo ""
read_info $dev2
setup_table t2
setup_balance $dev1 $dev2 50 50
Let me explain how it works...
Since we are going to work with multiple routing tables, we need to tell the kernel when to use which table. This is done by setting up rules, but before doing this, the script calls the function flush_rule(). Here, the list of rules is flushed with ip rule flush and then two "standard" rules are added:
ip rule add from all lookup default prio $prio
ip rule add from all lookup main prio $prio
The $prio variable is initialized to the very high value 32767 and decreased by one every time we add a rule. This ensures that our later rules with get higher priority than the standard rules.
Next, the script reads two arguments from the caller: $dev1 and $dev2, which specifies which interfaces to use. For each interface, the script calls the function read_info() to collect the needed information about the network interface. The function uses nmcli to get information from NetworkManager, and then some grepping to read out the needed parts:
nminfo=$(nmcli device list iface $1)
iface=$(echo -e "$nminfo" | grep "GENERAL.IP-IFACE" | cut -f2 -d:)
addr=$(echo -e "$nminfo" | grep ip_address | cut -f2 -d=)
gw=$(echo -n "$nminfo" | grep " routers " | cut -f2 -d=)
net=$(echo -n "$nminfo" | grep network_number | cut -f2 -d=)
mask=$(echo -n "$nminfo" | grep " subnet_mask " | cut -f2 -d=)
cidr=$(oct2cidr $mask)
The second line might seem a bit weird, but NetworkManager uses a different name than the kernel for my internal 3g-modem (ttyACM0 vs. wwan0). This line tells me the kernel name of the interface. I found this approach the easiest way to collect the gateway address received by DHCP. Another way would have been to tell NetworkManager to use dhcpcd instead of dhclient (put dhcp=dhcpcd in the [main] section of /etc/NetworkManager/NetworkManager.conf), and then parse the lease-file from dhcpcd.
With the needed information available, we can setup the routing tables for the two interfaces. Before using the script, you must create these tables in the iproute2 configuration by adding these to lines to /etc/iproute2/rt_tables:
2 t1
3 t2
The two integers are not line numbers. With these in place, the script can call the function setup_table() to configure routes for the interface that we have just received information about with NetworkManager:
table=$1
ip route flush table $table
ip route del $net/$cidr
ip route add $net/$cidr dev $iface src $addr
ip route add $net/$cidr dev $iface src $addr table $table
ip route add default via $gw dev $iface table $table
ip rule add from $addr table $table prio $prio
ip route add 127.0.0.0/8 dev lo table $table
The function first flushes the table in case the script was previously used, and then add subnet routing entries for the interface, a default route to the new table and a rule to tell linux to use this table.
When both interface has been setup by first reading the information and then creating the table, the script can call the function setup_balance to start using the two interface:
ip route del default
ip route add default scope global \
nexthop via $gw1 dev $if1 weight $3 \
nexthop via $gw2 dev $if2 weight $4
The two $gw variables are created before this in the function to know the kernel names of the interfaces. The first call to ip deletes the current default route, which is most likely from one the two interfaces. Then add a new default route is added. This specifies two "default" routes via the two interfaces and how they should be balanced. In some cases you just wanna use the two routes by calling bind (2) on the socket before connection, and in this case you don't need this special default route.
So, with all this in place, we can run the script like this:
./multi_homed.sh wlan0 usb0
This makes your box use both your wifi and your phone (if using USB tethering in Android). In case you also have an internal 3g-modem, you can retrieve its NetworkManager name by calling nmcli:
nmcli device
The next thing I need to do is to setup a box with MultiPath TCP (MPTCP), so that I can compare my multipath application against MPTCP.
Happy networking :)