ByteME

This confused me for a while so I have made a (public) note of it as a reference for me and anybody else who is intrested.

Background

IPSEC is a mechanism that enables encrypted secure communications to take place between computers. One use of it is to allow the creation of Wide Area Networks WAN over the public internet so that the Local Area Networks (LAN's) that are joined together are joined in a secure and encrypted way. Other uses are to protect traffic on LAN's from evesdroppers and to allow "road warriors" i.e. people with laptops who wish to access a company/private network to do so in a secure way. In a nutshell IPSEC is a specific implementation of Virtual Private Networks (VPN), but there are many other ways to do VPN I.M.H.O. IPSEC is the most secure. IPSEC can run on linux, windows, mac, embedded systems and you can even get dedicated routers with built in IPSEC abilities.

There are many implementations of IPSEC on linux now, the most famous are the "swans" freeswan (no longer in development) and the children of freeswan, openswan and strongswan(both very much in development). The 2.6.X linux kernel now features a native IPSEC stack but is still very convenient to use the *swan userspace tools to facilitate the implementation

The problem

On 2.4.X Linux kernels the swans had a kernel patch that added IPSEC support to the kernel via what is known as the KLIPS stack. This boils down to that once set up as well as eth0,ppp0 etc interfaces you get ipsec0,ipsec1 etc interfaces. This makes firewalling trivial. Lets assume that the internet is on interface ppp0 and the local network is on eth0.
**UPDATE** the 2.6 Native IPSEC stack has been back ported to some of the newer 2.4 kernels, this was certainly true when I installed Debian testing and got kernel 2.4.27. So the instructions for the 2.6.X kernels may also apply for the most recent 2.4 series kernels.

ALLOW UDP to port 500 in and out of ppp0
ALLOW ESP (Protocol 50) in and out of ppp0
BLOCK everything else (apart from any other services you want, that's up to you) in and out of ppp0
ALLOW Everyhing in and out of ipsec0,ipsec1 etc

More restrictive rules can be placed on ipsecN interfaces if required but you know that all traffic from ipsecN is secure traffic and not public internet traffic. On 2.6.X kernels it is typical to use the built in IPSEC stack (although the final release of freeswan has KLIPS for 2.6.X). This causes firewall problems, there is NO ipsecN interface. What happens is that a secure packet (ESP) enters ppp0 is decrypted by the kernel then pumped back out of ppp0!. Nightmare the old way of firewalling is now VERY broken as with the previous simple rules it is impossible to distinguish between public internet traffic and decrypted secure traffic.

The solution

It took me ages to find it but the solution is quite simple. Use the power of the mangle table on IPTABLES and the traffic shaping rules. What needs to be done is that when a secure (ESP) packet enters ppp0 it is marked, this mark is internal to the kernel and does not exists outside of the kernel. The mark is also persistant across decryption so that any marked packet when it is still an ESP packet is STILL marked when it is decrypted and becomes a TCP/UDP or what ever packet. I use the shorewall filewall system which just sets up the iptables for you nicely. To enable packet marking add the following to /etc/shorewall/tcrules

#MARK           SOURCE          DEST            PROTO   PORT(S) CLIENT  USER
1:P             ppp0            0.0.0.0/0       ESP
1:F             ppp0            0.0.0.0/0       ESP
1               $FW             0.0.0.0/0       ALL

To allow shorewall to process these rules you must also change the TC_ENABLED parameter in the shorewall.conf (/etc/shorewall/shorewall.conf) file. Eg :-
TC_ENABLED=Yes
(Thanks to Clemens Ender for pointing out my omission).

Then you need to allow marked packets through the firewall, shorewall does not directly support this so add the following to /etc/shorewall/start

iptables -I all2all -i eth0 -d 0.0.0.0/0 -m mark --mark 1 -j ACCEPT
iptables -I all2all -o eth0 -d 0.0.0.0/0 -m mark --mark 1 -j ACCEPT

** UPDATE ** NAT-T if you are using the Nat Traversal system (a patch to freeswan and native to openswan and strongswan) then you do not see ESP packets at your firewall, instead you see UDP-4500 packets which must be given similar treatment to the ESP packets from a firewalls point of view