Multiple WAN Load Balancing Using iptables Aug 27th 2021 Words: 765

Background

I have a multi-wan set up in my office on an OpenWrt router, and the load balancing over the 6 WAN interfaces is required.

I have tried the mwan3 but it is incompatible with miniupnpd and seems to had problem with IPv6 as well, so nope. I think iptables is powerfull enough to achieve load balancing / policy routing etc, so I palyed around a little with it.

Steps

Set routing table alias (Optional)

Edit /etc/iproute2/rt_tables:

1
2
3
4
5
6
10 WAN_A
20 WAN_B
30 WAN_C
40 WAN_D
50 WAN_E
60 WAN_F

This is to set an alias for the routing tables later will be created. If the alias is not set, then only number value can be used to name a routing table. Setting aliases is recommended if you have a bad memory.

Create the routing table

Create a separate table for each interface with a default route, then link a fwmark to the table so all packets with a mark would be forward to the specified table and use the defined interface. 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# clear old and create routing tables
ip rule del table 100
ip rule del table 110
ip rule del table 120
ip rule del table 130
ip rule del table 140
ip rule del table 150
ip route add default dev pppoe-wan_a table 100
ip route add default dev pppoe-wan_b table 110
ip route add default dev pppoe-wan_c table 120
ip route add default dev pppoe-wan_d table 130
ip route add default dev pppoe-wan_e table 140
ip route add default dev pppoe-wan_f table 150
# for dhcp protocol use device name eg. 'eth0.100'

# link mark with routing table and set priority
ip rule add from all to 10.0.0.0/8 table main prio 1000
ip rule add from all to 192.168.0.0/16 table main prio 1001
ip rule add from all to 172.16.0.0/12 table main prio 1002
# the above rules are critical for routing packets back to clients
ip rule add fwmark 100 table 100 prio 1003
ip rule add fwmark 110 table 110 prio 1004
ip rule add fwmark 120 table 120 prio 1005
ip rule add fwmark 130 table 130 prio 1006
ip rule add fwmark 140 table 140 prio 1007
ip rule add fwmark 150 table 150 prio 1008
ip rule add fwmark 160 table 160 prio 1009
ip route flush cache
  • You can use the aliases set in the previous step instead of the numeric table name.
  • Make sure the priority number is smaller than main table ( check with ip rule)
  • Make sure main table has a default route (check use default gateway in the interface advanced settings or add option defaultroute '1' in the /etc/config/network)

Verify with ip rule:

1
2
3
4
5
6
7
8
9
10
11
12
13
0:      from all lookup local 
1000:   from all to 10.0.0.0/8 lookup main 
1001:   from all to 192.168.0.0/16 lookup main 
1002:   from all to 172.16.0.0/12 lookup main 
1003:   from all fwmark 0x64 lookup 100 
1004:   from all fwmark 0x6e lookup 110 
1005:   from all fwmark 0x78 lookup 120 
1006:   from all fwmark 0x82 lookup 130 
1007:   from all fwmark 0x8c lookup 140 
1008:   from all fwmark 0x96 lookup 150 
1009:   from all fwmark 0xa0 lookup 160 
32766:  from all lookup main 
32767:  from all lookup default

Create iptables rules

For OpenWrt the following packets are required:

  • iptables-mod-ipopt
  • iptables-mod-conntrack-extra
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# load balancing over 6 WAN
iptables -t mangle -N wan_load_balancing
iptables -t mangle -A wan_load_balancing -d 10.0.0.0/8 -j ACCEPT
iptables -t mangle -A wan_load_balancing -d 172.16.0.0/12 -j ACCEPT
iptables -t mangle -A wan_load_balancing -d 192.168.0.0/16 -j ACCEPT
iptables -t mangle -A wan_load_balancing -j CONNMARK --restore-mark
iptables -t mangle -A wan_load_balancing -m mark ! --mark 0 -j ACCEPT
iptables -t mangle -A wan_load_balancing -m conntrack --ctstate NEW -m statistic --mode nth --every 6 --packet 0 -j MARK --set-mark 10
iptables -t mangle -A wan_load_balancing -m conntrack --ctstate NEW -m statistic --mode nth --every 6 --packet 1 -j MARK --set-mark 20
iptables -t mangle -A wan_load_balancing -m conntrack --ctstate NEW -m statistic --mode nth --every 6 --packet 2 -j MARK --set-mark 30
iptables -t mangle -A wan_load_balancing -m conntrack --ctstate NEW -m statistic --mode nth --every 6 --packet 3 -j MARK --set-mark 40
iptables -t mangle -A wan_load_balancing -m conntrack --ctstate NEW -m statistic --mode nth --every 6 --packet 4 -j MARK --set-mark 50
iptables -t mangle -A wan_load_balancing -m conntrack --ctstate NEW -m statistic --mode nth --every 6 --packet 5 -j MARK --set-mark 60
iptables -t mangle -A wan_load_balancing -j CONNMARK --save-mark
iptables -t mangle -A PREROUTING -j wan_load_balancing

Persist config

Put all the above scripts in /etc/firewall.user