← cdp by tcpdump | network | bridge on bond →

Policy-based routing using multiple routing tables

Making a System accessible over two IPs (Interfaces) is a bit of a challenge. Say you have connected the Machine via

eth0 192.168.0.10/24 GW: 192.168.0.1 (old)
eth1 192.168.1.10/24 GW: 192.168.1.1 (new)

Normally all traffic to remote IPs would be routed to the default gateway. Further say your default gateway is 192.168.0.1, so it's reachable via eth0. IP packages coming in via the IP address of eth1 would be answered via the default gateway through eth0 and therefore be invalid. The Solution to this problem is called Policy-based routing (by cisco) the implementation for Linux is based on multiple routing tables. Those routing tables are registered in /etc/iproute2/rt_tables:

#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
200   oldnet
201   newnet

In this case the oldnet and newnet tables are custom, here we want to migrate the Server from one Network (oldnet) to the other (newnet) and the Policy-based routing allows us to do this seamless on the server side. To use your defined tables you can simply use the ip command:

ip route add 192.168.0.0/24 dev eth0 table oldnet
ip route add default via 192.168.0.1 table oldnet

To actually use your new routing table you have to issue an ip rule command, which makes the kernel use the table according to the matched rule:

ip rule add from 192.168.0.0/24 table oldnet

Now to actually make a difference we also have to define the routing for the newnet so:

ip route add 192.168.1.0/24 dev eth1 table newnet
ip route add default via 192.168.1.1 table newnet
ip rule add from 192.168.1.0/24 table newnet

Now we have our server use the table oldnet whenever he sends out a package from the 192.168.0.0/24 subnet and use the table oldnet for packages from 192.168.1.0/24. It's important to understand the direction of the packages, it's NOT about packages comming in, they will reach the host anyhow it's all about the (answering) packages that need to go the same way back as they came from, so the from statement is the one to use here.

Now the commands ip rule list will show you the rules actually used and the ip route show table oldnet resp. ip route show table newnet will show you the actual tables. Btw. a ip route show table all will give you quite an intimate look at the routing behind the scenes, just in case you didn't know.

To make things happen at boot time there are different ways things are handle, to name the major ones:

Debian

Quite obviously you need to touch the almighty /etc/network/interfaces like this:

iface eth0 inet static
address 192.168.0.1
netmask 255.255.255.0
gateway 192.168.0.1
post-up ip route add 192.168.0.0/24 dev eth0 table oldnet
post-up ip route add default via 192.168.0.1 table oldnet
post-up ip rule add from 192.168.0.0/24 table oldnet
post-down ip rule del from 192.168.0.0/24 table oldnet


iface eth1 inet static
address 192.168.1.1
netmask 255.255.255.0
gateway 192.168.1.1
post-up ip route add 192.168.1.0/24 dev eth1 table newnet
post-up ip route add default via 192.168.1.1 table newnet
post-up ip rule add from 192.168.1.0/24 table newnet
post-down ip rule del from 192.168.1.0/24 table newnet

Redhat"ish" (Scientific, Centos, Fedora and the like)

You need to edit (or create) the files /etc/sysconfig/network-scripts/route-eth{0,1} to contain the following:

/etc/sysconfig/network-scripts/route-eth0

192.168.0.0/24 dev eth0 table oldnet
default via 192.168.0.1 dev eth0 table oldnet

/etc/sysconfig/network-scripts/route-eth1
```
192.168.1.0/24 dev eth1 table newnet
default via 192.168.1.1 dev eth0 table newnet
```
The rules are configured in /etc/sysconfig/network-scripts/rule-eth{0,1}:
/etc/sysconfig/network-scripts/rule-eth0
```
from 192.168.0.0/24 lookup oldnet
```
/etc/sysconfig/network-scripts/rule-eth0
```
from 192.168.1.0/24 lookup newnet
```

openSUSE 11.1

Sadly I was not able to find a "simple" way to configure it for openSUSE 11.1. As its only a transitional Situation I live with the routing not beeing reboot-safe. *fingers crossed*