Using iptables and ipsets to replace TCP Wrappers

Is it possible, is it practical?

This page addresses the subject of a possible iptables representation of TCP wrappers and its configuration file hosts.allow in the context of the openSUSE distribution of GNU/Linux. However the iptables techniques used are applicable across a wide range of distributions.

The reader is assumed to be an experienced administrator of GNU/Linux systems and have a working knowledge of iptables.

Health and safety warning: The techniques described by this page require root access to the system, which can be dangerous. You have been warned. In addition, use of the custom extensions to the SUSE firewall SuSEfirewall2 is not supported by SUSE.

1 — Introduction

TCP wrappers have been removed from openssh at release 6.7. See the Linux Weekly News article. This change reached openSUSE in release 13.2. Since that release users of TCP wrappers must now look for other means of securing services such as openssh. A common response is to say Use the iptables firewall, but iptables runs at network level whereas TCP wrappers is an application level mechanism. The notion of a layered defence in depth is weakened and it is questionable that iptables can replace TCP wrappers.

In an effort to address this question, a Bash script proposed by this page attempts to convert hosts.allow, the TCP wrappers configuration file, to a list of IP numbers for use in an ipset collection, and a set of iptables rules to be placed in a hook function to the SUSE firewall SuSEfirewall2. The script also explores some of the extensions that become possible once such a conversion process is available.

If you are just interested in my conclusions, and not the technical details, then go straight to the Conclusions. For those who enjoy the gory details, read on.

The Bash script takes the TCP Wrappers configuration file, often /etc/hosts.allow and extracts each rule in turn. For the purposes of the Bash script, the rules have the simplified form daemon_list : client_list : script : action and for each rule the Bash script builds ipsets and iptables rules as shown in the following table:

Figure 1. Overview of what the Bash script hosts.allow.ctl does.
  Configuration
  file
  hosts.allow.ctl action
daemon_list d1,d2 : client_list : script : action is treated as
d1 : client_list : script : action followed by
d2 : client_list : script : action, and for
each di an iptables rule is created in which the di
becomes a --dport value. Each iptables rule is labelled to
show it's origin. The rule is placed by default in filter:INPUT .
but there is an option to use filter:OUTPUT or
filter:FORWARD . See figure 1.
client_list The client list is converted to a hash:net ipset with
comment and counters. Each element in the set is
labelled to show it's origin and the counters show
the traffic volume subject to that element.
script The optional shell scripts and other options in the original
TCP Wrappers configuration are not supported.
iptables_options are supported as an extension. The TRACE
option places rules in raw:PREROUTING and raw:OUTPUT .
action The actions become -j targets in the iptables rules.
ALLOW is supported as -j ALLOW,
DENY is implement as -j DROP with an option
to use -j REJECT.

1.1 — Change log

2 — TCP Wrappers configuration file

The configuration file used by TCP wrappers is usually found in /etc/hosts.allow   In this file, comment lines start with a # and are ignored. Blank lines are ignored. Rules are separated by newlines. Within a rule, the separators are space and tab, and in lists the comma is also used as a separator.

Rules may be folded over several lines using the \ before newline convention.

Figure 2. Structure of a TCP Wrappers configuration file
1 hosts.allow ::= line
2 | line hosts.allow
1 line ::= comment Starts with #
2 | rule
3 | Empty lines ignored
1 rule ::= daemon_list : client_list : action
1 daemon_list ::= d
2 | d daemon_list E.g. upsd sshd
3 | d, daemon_list E.g. upsd, sshd   Note the comma
4 d ::= daemon name E.g. upsd
5 | ALL TCP and UDP
6 | ALL[tcp-only] TCP only. Experimental addition
7 | ALL[udp-only] UDP only. Experimental addition
8 | EXCEPT Keyword not supported
9 | daemon name[tcp-only] E.g. ssh, tcp only. Experimental addition
10 | daemon name[udp-only] E.g. openvpn, udp only. Experimental addition
11 | port number E.g. 1194 Experimental addition
12 | port number[tcp-only] E.g. 873, tcp only. Experimental addition
13 | port number[udp-only] E.g. 1194, udp only. Experimental addition
1 client_list ::= client
2 | client client_list
3 | client, client_list Note the optional comma
1 client ::= localhost 127.0.0.1 and ::1
2 | ALL Specify 0/1 128/1 or /directory/file
3 | LOCAL Not supported
4 | UNKNOWN Not supported
5 | KNOWN Not supported
6 | PARANOID Not supported
7 | EXCEPT Not supported
8 | host_name As in /etc/hosts
9 | IPv4_dotted_quad E.g. 212.27.48.10
10 | IPv4_dotted_quad/CIDR E.g. 212.27.48.0/24
11 | IPv4_dotted_quad/net_mask E.g. 212.27.48.0/255.255.255.0
12 | incomplete_IPv4 E.g. 212. 212.27. 212.27.48. Trailing dot
13 | partial_IPv4/CIDR E.g. 212.27/16 No trailing dot
14 | dotted_quad-dotted_quad E.g. 10.0.0.16-10.0.0.31 range
15 | dotted_quad- E.g. 10.0.0.16- single host range
16 | FQDN E.g. www.kernel.org
17 | user@FQDN E.g. root@nsa.gov   Not supported
18 | ccTLD E.g. .cn   Support for country code TLD's
19 | incomplete_domain_name E.g. .kernel.org   Not supported
20 | [IPv6_address] E.g. [2a01:e0c:1::1]   Brackets required
21 | [IPv6_address/CIDR] E.g. [2a01:e0c:1::1/124]   Brackets required
22 | /directory/file File holds a client_list
1 action ::= ALLOW iptables ALLOW
2 | DENY iptables DROP
3 | script : ALLOW iptables ALLOW
4 | script : DENY iptables DROP
1 script ::= spawn shell_command Not supported. See man 5 hosts_options
2 | twist shell_command Not supported. See man 5 hosts_options
3 | keepalive Not supported. See man 5 hosts_options
4 | linger s Not supported. See man 5 hosts_options
5 | rfc931 s Not supported. See man 5 hosts_options
6 | banners /some/directory Not supported. See man 5 hosts_options
7 | nice n Not supported. See man 5 hosts_options
8 | setenv name value Not supported. See man 5 hosts_options
9 | umask 022 Not supported. See man 5 hosts_options
10 | user nobody Not supported. See man 5 hosts_options
11 | user nobody.kmem Not supported. See man 5 hosts_options
12 | iptables_options Experimental addition
1 iptables_options ::= Experimental addition
2 | option Experimental addition
3 | option iptables_options Experimental addition
1 option ::= LOG Experimental addition
2 | --log-prefix "text" --log-prefix "hosts.allow[7.4.ssh]". Experimental addition
3 | --log-level "level" --log-level "warning". Experimental addition
4 | LOG-WITH-LIMIT Experimental addition
5 | LIMIT Experimental addition
6 | --rate "rate" --limit "3/minute". Experimental addition
7 | TRACE Experimental addition
8 | filter:INPUT Experimental addition
9 | filter:OUTPUT Experimental addition
10 | filter:FORWARD Experimental addition
 

Notes on the configuration specification

  1. rule: The rule is to be written on a single line. For convenience, the line may be folded by escaping the new-line. A TCP wrapper rule translates into one or more iptables rules.
  2. d: In the original TCP wrappers, the only daemons allowed were those for which TCP wrapper support was compiled in. The daemons were specified with the name of the binary, typically the service name with a final letter d, e.g. sshd. The proposed hosts.allow.ctl script accepts all services specified in /etc/services and allows omission of the additional letter d. The script knows about the services quotad, vid, rpasswd, maitrd, raid-cd, blackboard and epmd which end in a letter d. The service may also be specified by it's associated port number.
  3. d: The Bash script hosts.allow.ctl assumes that the protocols for each service are TCP and UDP unless specified otherwise.
  4. d: The suffixes [tcp-only] and [udp-only] are case insensitive, but I write them in lower case. They are directly attached to keyword ALL, to the daemon name and to the port number and limit the rule for that service to the specified protocol. There is no intervening white space.
  5. action: Keywords such as ALL, ALLOW, DENY are case insensitive, although I always write them in upper case.
  6. script: The grammar shown for those options which are not supported is a simplification.

2.1 — client_list summary

The client list translates into an IPv4 and an IPv6 ipset collection. The client list may contain sub lists, and the elements in the collection are labelled to show their origin. This helps to identify in an ipset listing including multiple sources, exactly which elements contributed to the action.

Basic client list elements

Here is a summary of those elements of the original TCP wrappers client_list which are wholly or partially supported by the proposed script.

localhost
The domain name localhost is reserved by RFC 6761 for the IP loopback address, for example 127.0.0.1 and ::1. The proposed script resolves localhost as 127.0.0.1 in IPv4 and ::1 in IPv6.
ALL
The client ALL is not supported directly, but the effect can be easily achieved for IPv4 by defining a set of subnets which cover the entire IPv4 address space. The simplest is 0/1 128/1 but a more detailed analysis of the traffic is possible if 256 /8 subnets are used. These may be placed in file /etc/fw_custom/ALL with the Bash invocation:
 cd /etc/fw_custom ; : > ALL ; for i in {0..255}; do echo $i/8 >> ALL; done

or, if you like a lot of detail, try 65536 /16 subnets

 cd /etc/fw_custom ; : > ALL 
 for i in {0..255}; do for j in {0..255}; do echo $i.$j/16 >> ALL; done; done
host_name
The IPv4 and IPv6 values are taken from /etc/hosts . For example if the file /etc/hosts contained the line
 10.218.0.13  maria
then the host name maria would be resolved to 10.218.0.13.
FQDN, Fully Qualified Domain Name
E.g. www.kernel.org   Note that a FQDN may correspond to more than one IPv4 or IPv6 address, and that those addresses may depend on your geographic position, and may also change with time. The proposed Bash script generates ipsets of IPv4 and IPv6 numbers from the values discovered by dig when the script is run. To update the numbers, you have to re-run the script to rebuild the ipsets.
Note also that a business may be sending you spam traffic from sources not specified by A or AAAA records, for example MailChimp.
ccTLD   country code top level domain
This is the only support provided by the proposed Bash script for the TCP wrapper's incomplete domain specification. The script makes use of a public service of IP address block lists provided by IPverse IP address block lists. For those countries supported by IPverse IP address block lists you have only to specify the top level domain of the country, preceeded by a dot, to include the entire IPverse IP address block lists block list. E.g. .cn to block traffic from IP addresses in the PRC.
Note that this is not exactly the same as TCP wrappers' .cn . TCP wrappers perform a reverse DNS lookup when the packet arrives. If the tld is available, and is .cn, then the packet would be detected, no matter which country it had come from. The proposed Bash script does not perform reverse DNS lookups, and is strictly geographically based.
Block lists for those countries which are not supported by IPverse IP address block lists for example .uk , can be found at Country IP blocks. Select the format Netmask, click on Create ACL make a local copy of the resulting file. Place the address of this file in the client_list. For the UK it is convenient to use file /etc/fw_custom/.uk .
IPv4 dotted quad
This is a direct specification of an IPv4 host address, e.g. 212.27.48.10
IPv4 dotted quad/CIDR
This is a direct specification of an IPv4 network, e.g. 212.27.48.0/24 . Note that 0/0 is not allowed.
Partial IPv4 dotted quad/CIDR
This is an abbreviation of the IPv4 dotted quad/CIDR element in which trailing numbers on the quad are omitted. Their values are taken as 0. For example 129.63/16 is an abbreviation of 129.63.0.0/16. Another example is 10/8 for an entire private network.
IPv4 dotted quad/net mask
This is a direct specification of an IPv4 network, e.g. 212.27.48.0/255.255.255.0 = 212.27.48/24
Incomplete IPv4 dotted quad
This specification of an IPv4 class A, B or C network is specific to TCP wrappers. For example 212. represents 212/8, 212.27. represents 212.27/16 and 212.27.48. represents 212.27.48/24. Note that the incomplete dotted quad notation always ends in a dot.
[IPv6_address]
This is a direct specification of an IPv6 host address, e.g. [2a01:e0c:1::1]. The brackets are required.
[IPv6_address/CIDR]
This is a direct specification of an IPv6 network, e.g. [2a01:e0c:1::1/124]. The brackets are required.
/directory/file
It is sometimes convenient to place a specific white list or blocking list in a separate file and refer to that file where needed. If the list is a list of IPv4 subnetworks, then they are assumed to have been sanitized — i.e. there are no overlaps or duplicates. If you want the optimizer to clean up the list, then specify the hosts and networks in the list using the range notation.

Additional client list elements

Here is a summary of the additional element notations which have been added to the client_list by the proposed script. These additional client list elements are also translated into IPv4 and IPv6 ipset collections. Note that only host ranges are subject to subnet optimization. Subnets specified using the basic client list elements are assumed to have been sanitized already.

IPv4 host ranges — A.B.C.D-W.X.Y.Z
It is sometimes convenient to express a group of hosts as a range of IPv4 numbers, e.g. 10.218.0.10-10.218.0.99   . The proposed bash script will express this range as a set of IPv4 subnets. This set of subnets, together with any sets produced by other ranges specified in the client list will be optimized to the least number of subnets. The optimization process is highly recursive and is slow. You will get a warning or even an error message if you over-use this facility. For example, the range 129.67.0.1-129.67.255.255 produces 65535 subnets to be optimized. Bash is inefficient and such a job would take a couple of hours. You can reduce the load considerably by either
Incomplete IPv4 host range — a single host
It is possible to specify a single host in such a way that it is included in the optimization. This is done by placing a hyphen after the dotted quad address, e.g. 129.67.7.91-   .

2.2 — TCP wrappers example

This example shows some of the interesting things that could be done with the original TCP wrappers.

 # hosts.allow for modest home server
 upsd :         localhost                              :ALLOW
 sshd, rsyncd : .math.bigU.edu, 10.8.0/24, localhost :\
                 spawn (echo "Accepted access to %d from %c" | \
                 /bin/mail -r hosts.allow@modest-server.tld\
                           -s '%s (host %h/maria) accepts access to %d from %c'\
                           admin@domain.tld) &         :ALLOW       
 ALL :          ALL                                    :DENY

In this example we see 3 rules:

  1. The NUT (UPS management) daemon is accessible from the local host only. This rule can be reproduced using iptables.
  2. ssh and rsync are available on this server for all the hosts in the mathematics department at Big University, for the VPN users and the local host users. Whenever an ssh or rsync client is accepted, a mail is sent to the sysadmin. The %d %c %s and %h parameters are defined in man 5 hosts_access. This rule cannot be reproduced using iptables.
  3. Nothing else is to be accepted for those applications using TCP wrappers.
Figure 3. Simplified netfilter packet flow showing INPUT, OUTPUT, FORWARD, PREROUTING and POSTROUTING chains. This chart does not distinguish the work done at different network layers. The numbers in brown italics attached to the tables correspond to the steps described in Chapter 6 of the iptables Tutorial: traversing of chains and tables.

The hosts.allow.ctl script places iptables rules in filter:INPUT , filter:OUTPUT , filter:FORWARD , raw:PREROUTING and raw:OUTPUT .
iptables innards

3 — Firewalls based on iptables

There are many firewalls based on iptables running on GNU/Linux and the BSD's. I address here only those with which the proposed hosts.allow.ctl Bash script has been tested.

For an introduction to iptables, see the iptables Tutorial 1.2.2. The iptables packet flow is summarised in a simplified form in Figure 3, adapted from the figure by xkr47. The numbers in brown italics attached to the tables correspond to the steps described in Chapter 6 of the iptables Tutorial: traversing of chains and tables. Figure 3 does not take into account the work done at the different network layers. Here is a more complex figure by Jan Engelhardt which does.

3.1 — SUSE firewall 2

The SUSE firewall SuSEfirewall2, currently version 2, is integrated into YaST and relies on a Bash script /sbin/SUSEfirewall2 which rebuilds the iptables rules as a batch job.

The SUSE script sets up the iptables rules by defining a sequence of BASH functions which build up the set of rules. The SUSE functions populate the chains INPUT, OUTPUT, FORWARD, PREROUTING and POSTROUTING , plus additional SUSE-specific chains input_int, input_ext, input_dmz, forward_int, forward_ext, forward_dmz, reject_func and conditional chains of the form target_if_direction_zone.

The SUSE firewall offers the user three levels of configuration of the firewall:

  1. The basic configuration is presented through a GUI which is part of YaST: YaST => Security and Users => Firewall.
  2. Non-basic configuration such as port forwarding is defined by declaring SUSE-defined variables in file /etc/sysconfig/SuSEfirewall2 .
  3. There are five hook functions available for further customisation of the SUSE firewall. These functions after_chain_creation, before_port_handling, before_masq, before_denyall and after_finished normally have do-nothing definitions. A custom modification might consist in specifying additional iptables rules in one of these hook functions. To activate the custom modification, place the address of the file containing the modified hook function in variable FW_CUSTOMRULES in file /etc/sysconfig/SuSEfirewall2 and the modified function will be run as part of the batch job which sets up the SUSE firewall. Note that using these hook functions is at the responsibility of the system administrator. SUSE do not support such extensions.

TCP Wrappers support with SuSE firewall

The proposed Bash script hosts.allow.ctl generates IP sets representing the original client_lists, and these are connected to the firewall's iptables rules with additional iptables rules placed in the hook function after_chain_creation . The other hook functions before_port_handling , before_masq , before_denyall  and after_finished  retain their do-nothing definitions provided by SUSE.

The hosts.allow.ctl script places the hook function after_chain_creation  in file /etc/fw_custom/fw_hosts.allow. The user must activate the hook function by setting variable FW_CUSTOMRULES in file /etc/sysconfig/SuSEfirewall2 The default is to set

 FW_CUSTOMRULES="/etc/fw_custom/fw_hosts.allow"

but the directory can be changed with the scripts's -o option.

Self documentation for SUSE firewall

A lot of thought has gone into the SUSE firewall, but there are few comments in the code to help the reader. However the names of the functions in the script are helpful, and it is instructive to add these names to the rules they create. I suggest modifying the file /sbin/SuSEfirewall2 to add -m comment --comment "${FUNCNAME}[${LINENO}]" to all those iptables commmands which create firewall rules. For example

 function allow_basic_established()
 {  # needed for dhcp and dns replies
    local iptables
    for iptables in "$IPTABLES" "$IP6TABLES"; do 
        $LAA $iptables -A INPUT ${LOG}"-IN-ACC-EST " -m conntrack --ctstate ESTABLISHED -m comment --comment "${FUNCNAME}[${LINENO}]"
        $iptables -A INPUT -j "$ACCEPT" -m conntrack --ctstate ESTABLISHED -m comment --comment "${FUNCNAME}[${LINENO}]"
...
    done
    # need to accept icmp RELATED packets (bnc#382004)
    $LAA $IPTABLES -A INPUT ${LOG}"-IN-ACC-REL " -p icmp -m conntrack --ctstate RELATED -m comment --comment "${FUNCNAME}[${LINENO}]"
    $IPTABLES -A INPUT -j "$ACCEPT" -p icmp -m conntrack --ctstate RELATED -m comment --comment "${FUNCNAME}[${LINENO}]"
    $LAA $IP6TABLES -A INPUT ${LOG}"-IN-ACC-REL " -p icmpv6 -m conntrack --ctstate RELATED -m comment --comment "${FUNCNAME}[${LINENO}]"
    $IP6TABLES -A INPUT -j "$ACCEPT" -p icmpv6 -m conntrack --ctstate RELATED -m comment --comment "${FUNCNAME}[${LINENO}]"
 }

Be patient, there are 159 lines to update in SUSE 13.2, but a true l33t with vi and sed would have no problem. Here is an example of the commented output of command iptables -n --line-numbers -t filter -L INPUT .

4 — The Bash script

4.1 — hosts.allow.ctl options

Usage hosts.allow.ctl options < configuration-file

Where the options are

--dry-run or -dry or -d
This is a dry run, not a production run. Give the ipsets names with the suffix -dry , and display the hook function, but do not create it. One or the other of the options --dry-run and --production must be specified. This option can only be used by root since it requires access to ipsets. The default output directory is /etc/fw_custom . The user root should create this directory and make it readable and writable by root only.
--help or -h
Read about the options.
--list
Display a listing of the iptables and ipset configuration and then exit. In the ipset listings, subnets for which no traffic is recorded are omitted.
--ipv6 or -IPv6
Create and list rules for IPv6. By default rules for IPv6 are not generated. IPv6 rule creation has not been tested.
--out-dir or -o
Use the specified output directory rather than the default /etc/fw_custom. This directory will be used to restore ipsets and for the hook function file.
--production or -prod or -p
This is a production run. Give the ipsets their correct names, and generate the file which contains the hook function. One or the other of the options --production and --dry-run must be specified. This option can only be used by root, and cannot be used while the firewall is running, since it modifies ipsets in use by the kernel. The default output directory is /etc/fw_custom . The user root should create this directoty, readable and writable by root only.
--quiet or -q
Do not display the hook function during dry runs. The default is to display the hook function during dry runs.
--list
Display a listing of the iptables, ip6tables and the ipset configuration and then exit. In the ipset listings, subnets for which no traffic is recorded are omitted. To include the display of IPv6 tables, place the option --IPv6 before the option --list,
--REJECT
Use target -j REJECT rather than -j DROP for unwanted datagrams and packets. With this option a matching rule returns reject-with-icmp-port-unreachable. The default is -j DROP.
--SUSE-REJECT
Use SUSE specific chain reject_func rather than -j DROP for unwanted datagrams and packets. The chain provides TCP, UDP and other specific return values. See iptables -t filter -n -L reject_func for details.
--UA
User agent identification for wget. The default is hosts.allow.ctl v1.2 ipset builder. http://rogerprice.org/hosts.allow.
--verbose or -v
Display a copious console trace of the internals of hosts.allow.ctl. There are approximately 10 lines of trace per ipset rule. You were warned. By default there is no trace.
--version
Display script name and version, e.g. hosts.allow.ctl v1.2 .

configuration-file

The traditional TCP wrappers assume that there is only one configuration file, /etc/hosts.allow, and this assumption is carried over into the hosts.allow.ctl script. At each creation of an ipset from a client_list, the previous ipsets are destroyed. The dry run ipsets are separate from the production ipsets.

4.2 — iptables options

The iptables rules generated by hosts.allow.ctl can be modified for each rule by placing optional specifications in the TCP Wrappers shell script field. The options are:

LOG log_options
The iptables rule generated by this hosts.allow rule will be accompanied (preceeded) by a second iptables rule which logs the traffic subject to the ALLOW or DENY. Only the traffic subject to the ALLOW or DENY is logged, and only the datagrams and packets in state NEW are logged. If the traffic does not satisfy the match in an ALLOW, it is not logged. If the traffic does not satisfy the match in a DENY, it is not logged. The logging is not subject to rate limiting.
LOG-WITH-LIMIT log_options limit_option
As LOG, but in addition the logging is subject to the rate limiting used by default in SuSEfirewall2. In addition to the log_options the limit_option is available without having to specify LIMIT .

log_options

--log-prefix prefix
The logging of traffic which matches the hosts.allow rule is watermarked with a specific prefix to help identify it, and to ease the debugging of the rules. The text supplied for the log prefix shall not contain white space, the backslash \ or the double quote character ", e.g. --log-prefix "not from \"them\"" wrongly contains both spaces and ". The default prefix is hosts.allow[L.R.S] where L is the line number of the rule in the configuration file, R is the rule number, and S is the service name as shown in /etc/services . I have never had to specify this option, since the default value has always suited my needs.
--log-level level
The traffic which matches the hosts.allow rule is logged at a specific priority. The default is warning which may be specified as 4 . For the other levels, see RFC 5427 clause 3 SyslogSeverity. The levels may be referenced by number or name.
LIMIT limit option
The traffic which matches the rule is subject to rate limiting, as specified by limit option .

limit_option

--rate number/period
The traffic which matches the hosts.allow rule is rate limited at the specified rate, where number is an integer, and period is one of second  , minute or hour . The default value is 3/minute .
TRACE
Log all those firewall rules which participate in the processing of the datagrams or packets identified by the TCP Wrapper rule. This option can quickly fill up your disk. You have been warned. There is at present no rate limiting, since this option is intended for debugging only, not for regular use. This option is helpful in showing that the iptables rules generated by hosts.allow do indeed perform the required filtering. The proposed Bash script will warn you if you try to use the options TRACE and ALLOW with the same rule.
filter:INPUT
By default, the iptables rules generated by hosts.allow.ctl are placed in filter:INPUT . It is not normally necessary to specify this option.
filter:OUTPUT
The iptables rules generated by hosts.allow.ctl are placed in filter:OUTPUT . See Figure 3. In this case, the daemon_list becomes a list of services on the remote machine(s), and the client_list becomes a ipset specifying a set of remote machines. This option may for example be used to allow employees to visit only carefully selected management approved sites, or to forbid access to sites with no business interest, such as Facebook. Remember that such sites have many IP addresses.
filter:FORWARD
The iptables rules generated by hosts.allow.ctl are placed in filter:FORWARD . See Figure 3. In this case, the daemon_list becomes a list of services on the remote machine(s), and the client_list becomes a ipset specifying a set of remote machines. This option may for example be used to prevent Windows 10 boxes on a subnet reporting all their activity to Microsoft as part of Microsoft's Telemetry.

4.3 — Exit codes

Figure 4. Bash script exit codes
  Code   Meaning
0 Script completed, no errors or warnings
10 Script completed but warning(s) were issued
11 Invalid script option
12 Invalid element or syntax in Bash source file
13 Bash script internal error
14 dig A error or timeout
15 dig AAAA error or timeout
16 getent hosts error
17 getent services or protocol number error
18 Required utility program not available
19 Valid hosts.allow.ctl option not supported by this script
20 wget error
21 ipset error
22 iptables error
23 Invalid IPv4 value or range

4.4 — Bash programming notes

  1. Little attempt is made to adopt a Bash l33t coder style such as the use of LDC=":" in SUSE's /sbin/SuSEfirewall2 . For example, all the if...then...else...fi are written out in full.
  2. Parameter expansion is always simple. There is no case modification, no substring removal, no alternate values. There are a few default values.
  3. Folks who think it is modern to use ++ to add 1 will be disappointed. It fails in opensuse.
  4. Flag variables use integer values 0 and 1. The default Bash notions of true and false are not used.
  5. The function read-clients which reads the client_list is recursive, calling itself to read sublists, including those sublists created by utility program dig. The function IP_TREE_insert used to handle IPv4 ranges, and it's subfunctions are particularly recursive and therefore slooow in Bash.
  6. This script uses the Bash regular expresion operator =~ introduced with Bash version 4. See also the IT World article.
  7. The script contains several cat file | while read... loops which Bash executes in subshells. To overcome the invisibility of subshell variables in the outer shell, the script uses temporary files for variables. These temporary files are cleared away by an exit trap.
  8. The input file is piped in to avoid having to use yet another cat file | while read... construction which would place the main working code in a subshell whose variables do not escape.
  9. The trace option --verbose which activates the function msg-dbg is helpful for debugging, but is very verbose, typically 10 lines per ipset element. You have been warned.
  10. No attempt has been made to improve performance of the Bash script with parallelisation. On a Dell Precision T7500 with 6 cores and 48 Gbyte memory, the script will generate more than 30000 ipset elements per minute using one core at a time. For the efficiency of ipsets see the article by daemonkeeper. The script has been tested with network/CIDR files containing 155000 networks.
  11. The proposed script has been run through ShellCheck and some obvious errors corrected, but the style suggested by that Haskell program seems unhelpful, and in some cases just wrong.

5 — Using the Bash script

The following examples use the SUSE distribution and the SUSE firewall SuSEfirewall2. The firewall has been connected to the extensions defined by the Bash script by setting variable FW_CUSTOMRULES in file /etc/sysconfig/SuSEfirewall2 to:

 FW_CUSTOMRULES="/etc/fw_custom/fw_hosts.allow"

Remember that the file fw_hosts.allow contains hook function after_chain_creation which sets the required iptables rules.

5.1 — Example: Limiting access to sshd

Overview

We wish to allow access to the ssh service to a limited number of clients. All other connections are to be refused. The original TCP Wrappers rules were:

sshd : .BigU.edu 10.218.0., localhost : shell script : ALLOW
sshd : ANY                                           : DENY

In the first rule, the very useful TCP wrapper incomplete domain specification .BigU.edu which was resolved at run time is not available with SuSEfirewall2. All we have is an approximation such as 129.63.   . The shell script cannot be used with iptables; the nearest we can get is to log the successful connections.

The effect of the second rule is obtained by closing the SUSE firewall to ssh traffic. Since the iptables translation of the first rule will be executed before the final traffic rejection, we obtain a generally closed ssh service with limited access by a few specified clients.

The ALLOW rule will be replaced by the rule:

sshd[tcp-only] : 129.63. 10.218.0., localhost : LOG-WITH-LIMIT : ALLOW

and the DENY rule will be satisfied by removing all access to the service in the YaST SuSEfirewall2 administration.

We test the setup with with a temporary TRACE rule before specifying the LOG_WITH_LIMIT option. Normally you should not use TRACE and ALLOW options in the same rule – it will flood your disk.

Details

The first version of the TCP wrapper demonstration configuration demonstration.conf uses the TRACE option. Note that it is risky to use TRACE on an ALLOW rule, sine the trace will be voluminous and will flood journald. However we do it here to show the effect. The floods of journald records have been swept away.

1    # Limited access to ssh service
2    sshd[tcp-only] : 129.63., 10.218.0., localhost : TRACE : ALLOW

First we try a dry run with command hosts.allow.ctl --no-ipv6 --dry-run < demonstration.conf and see (an edited version of) the proposed hook function containing the iptables rules:

 function fw_custom_after_chain_creation { true 
  # Restoring ipsets when firewall (re)starts
  ipset destroy; ipset restore < /etc/fw_custom/hosts.allow.ipsets
  iptables -t raw -A PREROUTING -p tcp --dport 22 -m set --match-set hosts.allow-rule-1-inet src.dst -j TRACE -m comment --comment "[2.1.ssh]"
  iptables -A INPUT -p tcp --dport 22 -m set --match-set hosts.allow-rule-1-inet src.dst -j ACCEPT -m comment --comment "[2.1.ssh]"
 }

where the first iptables rule is the translation of the TRACE option and the second represents the configuration file sshd rule, as confirmed by the comments. The client_list has been translated to a dry run ipset which we display using command ipset list hosts.allow-rule-1-inet-dry :

 Name: hosts.allow-rule-1-inet-dry
 Type: hash:net
 Revision: 6
 Header: family inet hashsize 64 maxelem 65536 comment
 Size in memory: 1688
 References: 0
 Members:
 10.218.0.0/24 comment "[2.1]"
 129.63.0.0/16 comment "[2.1]"
 127.0.0.1 comment "[2.1]"

We are now ready to install the SuSEfirewall2 modification. We stop the firewall with the command systemctl stop SuSEfirewall2.service , create the working ipsets with the command hosts.allow.ctl --no-ipv6 --production < configuration.conf , and restart the firewall with the command systemctl start SuSEfirewall2.service .

For the moment we leave the ssh service available in the SUSE firewall, and attempt an ssh connection from client kananga 10.218.0.16 to server pinta 10.218.0.14. Once the connection is established the server pinta the command hosts.allow-journal + ssh --since 13:02 reports:

         Current boot
 Dec 14 13:02:36 pinta SuSEfirewall2[20560]: Firewall custom rules loaded from /etc/fw_custom/fw_hosts.allow
 Dec 14 13:02:36 pinta hosts.allow.ctl[20581]: Hook file /etc/fw_custom/fw_hosts.allow: function fw_custom_after_chain_creation
 Dec 14 13:02:36 pinta hosts.allow.ctl[20582]: restores ipsets from /etc/fw_custom/hosts.allow.ipsets
 Dec 14 13:26:53 pinta kernel: TRACE: raw:PREROUTING:policy:3 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 Dec 14 13:26:53 pinta kernel: TRACE: filter:INPUT:rule:4 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 Dec 14 13:26:53 pinta kernel: TRACE: raw:PREROUTING:policy:3 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 Dec 14 13:26:53 pinta kernel: TRACE: filter:INPUT:rule:2 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 ...

and command iptables -n --line-numbers -t filter -L INPUT reports:

 Chain INPUT (policy DROP)
 num target     prot opt source       destination         
 1   ACCEPT     all  --  0.0.0.0/0    0.0.0.0/0   /* set_basic_rules[768] */
 2   ACCEPT     all  --  0.0.0.0/0    0.0.0.0/0   ctstate ESTABLISHED /* allow_basic_established[685] */
 3   ACCEPT     icmp --  0.0.0.0/0    0.0.0.0/0   ctstate RELATED /* allow_basic_established[699] */
 4   ACCEPT     tcp  --  0.0.0.0/0    0.0.0.0/0   tcp dpt:22 match-set hosts.allow-rule-1-inet src /* [2.1.ssh] */
 5   input_ext  all  --  0.0.0.0/0    0.0.0.0/0   /* fork_to_chains[1488] */
 6   LOG        all  --  0.0.0.0/0    0.0.0.0/0   limit: avg 3/min burst 5 /* finish_chains[1507] */ LOG flags 6 level 4 prefix "SFW2-IN-ILL-TARGET "
 7   DROP       all  --  0.0.0.0/0    0.0.0.0/0   /* finish_chains[1508] */

This shows that the connection was indeed accepted when in state NEW by filter:INPUT:rule:4 which carries the comment [2.1.ssh]. Thereafter the connection had status ESTABLISHED and was accepted by the SUSE rule allow_basic_established[685] .

When we are sure that iptables is doing its job, we use YaST to turn off the ssh service in the SuSEfirewall2, and replace the hosts.allow rule by

 1    # Limited access to ssh service
 2    sshd[tcp-only] : 129.63., 10.218.0., localhost : LOG_WITH_LIMIT : ALLOW

The DENY of all others is expressed by turning off the ssh service in the SuSEfirewall2.

5.2 — Example: Blocking undesirables from httpd

I run a modest http server so that I can test things before putting them on my public site. There is very little real traffic, but the file /var/log/apache2/access log fills up as secret government agencies, organised criminels, content thieves and many others try to take as much as possible. Lists are available of known undesirables, and these can be used with TCP wrappers to filter the traffic getting through to the http server.

I use the IPverse IP address block lists which provide the IPv4 and IPv6 addresses for many countries. Since these change regularly, it will be necessay to rerun the hosts.allow.ctl script regularly to update the ipsets. Just as an example, I have chosen some random undesirable countries. I live in one of them – this will help testing. Your list may well be very very different.

Some country IP lists are not available at IPverse IP address block lists, for example .uk . You can find a list for the UK at Country IP blocks. Select the format Netmask, click on Create ACL make a local copy of the resulting file. Place the address of this file in the client_list.

 1    # Keep "undesirables" away from web server
 2    httpd[tcp-only] : .be .ca .ch .fi .fr .li .lu .mc .va /etc/fw_custom/.uk\
                      : LOG_WITH_LIMIT : DENY

After restarting the firewall and waiting a few minutes, the simplest way of checking that this is working is with the command

 maria:~ # iptables --verbose -n --line-numbers -t filter -L INPUT | grep dpt:80
 8  27 1520 DROP  tcp  --  * *  0.0.0.0/0  0.0.0.0/0  tcp dpt:80 match-set hosts.allow-rule-1-inet src /* [2.1.http] */
Figure 5. The efficiency of ipsets vs iptables rules. Source: daemonkeeper.net. Please read the full article.
opsets vs iptables

where

It is clear from this example that placing the client_list in an IP set is a far more elegant solution than generating thousands of iptables rules. Read a very interesting article showing how much more efficient ipsets are.

5.3— Example: Blocking Windows Telemetry

Microsoft provides a Telemetry service for Windows 7, 8 and 10. This service sends a lot of user data to Microsoft so that Microsoft can improve the quality of Microsoft services. Many people understand this to mean Sell your data and take offence, considering it to be an invasion of their privacy. There are articles explaining how to turn Telemetry off.

However system administrators in businesses in which employees and guests bring their devices to work, may well find themselves thinking of blocking Microsoft Telemetry in a firewall. Others have thought of this and propose black lists of Telemetry servers. The example I use here is provided by BlockWindows, but I do not caution this list as in any way accurate.

To use the blocking list you have a choice:

  1. Copy the contents of file hostslist provided by BlockWindows into a local file, say /etc/fw_custom/BlockWindows, and make those contents one single line, with spaces between the entries, not newlines. Add the local file name /etc/fw_custom/BlockWindows to the client_list. hosts.allow.ctl will place the IP addresses corresponding to the FQDN's in the hostslist into an ipset.
     1    # Block access from local machines to MSFT telemetry
     2    all : /etc/fw_custom/BlockWindows : filter:OUTPUT  LOG-WITH-LIMIT : DENY
     3    all : /etc/fw_custom/BlockWindows : filter:FORWARD LOG-WITH-LIMIT : DENY
  2. Use the (highly) experimental TCP Wrappers extension MSFT-TELEMETRY to the client_list.
     1    # Block access from local machines to MSFT telemetry
     2    all : MSFT-TELEMETRY : filter:OUTPUT  LOG-WITH-LIMIT : DENY
     3    all : MSFT-TELEMETRY : filter:FORWARD LOG-WITH-LIMIT : DENY

In both cases, it would probably be better after a while to remove the LOG-WITH-LIMIT, or at least set a low rate, in order to avoid filling the logs with useless data.

For the moment I have not tested this, since I am not a Windows user.

6 — Helper script to display the journal

The helper script hosts.allow-journal offers a simple way of picking out the records of interest to the firewall in the syslog journal. The script begins by picking out only those records created since the most recent boot and then applies further filtering.

The helper script first tries to use journalctl, but if this is not available, it will try to use the file /var/log/messages. When using file /var/log/messages only the records for the current day are displayed, and the --since option is not available.

Usage hosts.allow-journal options

Where the options are

--include RE
+ RE
Display only the journal records which satisfy the regular expression RE. The option may be used repreatedly and the effect is additive, it adds further expressions to the initial default RE which is SuSEfirewall|SFW2|hosts.allow|TRACE
--exclude RE
- RE
Exclude from the display any records which match the regular expression RE . This option may be used more than once, the effect is cumulative. The default value is <nil> .
--since hh:mm
-T hh:mm
Display only those journal records dated since this time today. For example --since 18:05 . By default there is no temporal filtering.

Example  In general you will want to add at least the service name used in the rule you are testing, for example:

 hosts.allow-journal + ssh

If your output is overwhelmed by the results of successive tests, note the time of your most recent test and use the --since option:

 hosts.allow-journal + ssh --since 03:19

7 — Conclusion

After some experiments using the SUSE GNU/Linux distribution, and the proposed hosts.allow.ctl Bash script, here is my reply to the question in the title, and some of my conclusions.

  1. Can iptables and ipsets replace TCP Wrappers?:  The answer is partially. It is inaccurate and unhelpful to say You don't need TCP wrappers, use the firewall instead. TCP Wrappers operate at application level whereas the SUSE firewall which is based on iptables operates mainly at network level. The best practice of defense in depth is weakened, and the full power of TCP wrappers cannot be reproduced. The required custom modifications to the SUSE firewall are not supported by SUSE. The user is on his own in a difficult area since iptables is no easier to read or hack than sendmail.
  2. TCP Wrappers are an example of declarative programming in which the programmer says what is to be done, without saying how. On the contrary, iptables is a prescriptive or imperative programming language in which the programmer says how the job is to be done. Many programmers, particularly those in the functional programming and logic programming areas believe that the declarative paradigm leads to simpler, clearer programs with fewer bugs. Migrating from TCP Wrappers to iptables removes this advantage, especially the simplicity of the configuration file.
  3. It is possible to recover the declarative programming paradigm, and reproduce a useful subset of the facilities offered by TCP Wrappers by using a script to convert the hosts.allow configuration file to ipsets, accompanied with iptables rules to use them, and a hook function to tie the additional rules to the SUSE firewall.
  4. The existence of such a script, with extensions to the hosts.allow format opens up the possibility of using TCP Wrapper style rules to manage some useful network filtering beyond that attempted by TCP Wrappers. For example:
    1. UDP filtering and filtering of any service, especially those which never included TCP wrappers, such as openvpn and the http server Apache.
    2. Updating white lists and block lists from Internet sources without changing the iptables rules.
  5. The use of IP sets is an elegant and simple way of expressing the TCP Wrapper client_list, and preferable to adding yet more iptables rules.
  6. In the longer term, possible extensions include filtering of outgoing and forwarded traffic, for example to support blocking of Windows telemetry.

8 — Documentation

9 — Downloads

Either click to download or use command   wget -c http://rogerprice.org/hosts.allow/file_name

Figure 6. Stuff to download
  Stuff   file_name   Where I put it
Bash script hosts.allow.ctl /usr/local/sbin/
journalctl helper script hosts.allow-journal /usr/local/sbin/
Script: Is firewall running? is-firewall-running /usr/local/sbin/
Forbidden man 3 hosts_access hosts_access.3.gz /usr/share/man/man3/
Forbidden man 5 hosts_access hosts_access.5.gz /usr/share/man/man5/
Forbidden man 5 hosts_options hosts_options.5.gz /usr/share/man/man5/
Forbidden man tcpd tcpd.8.gz /usr/share/man/man8/
Forbidden man tcpdchk tcpdchk.8.gz /usr/share/man/man8/
Forbidden man tcpmatch tcpdmatch.8.gz /usr/share/man/man8/
The md5 sums of the downloadable files,
Use command md5sum -c MD5SUMS to
check the downloaded files.
MD5SUMS  

10 — Legal

The web page that you are reading is published under the Creative Commons Attribution-NoDerivatives 4.0 International (CC BY-ND 4.0) license.

The man pages discussing TCP Wrappers were written by Wietse Venema.

GPL v3 logo The Bash script hosts.allow.ctl is free as in freedom software: you can copy it, use it, redistribute it, and modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

The script is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this page. If not, see http://www.gnu.org/licenses/.

Comments, bugs, whatever: SUSE-specific? I read the opensuse@opensuse.org mailing list. They have some mailing list rules and mailing list archives. An iptables problem? There is a netfilter user mailing list. Read their rules carefully. Don't send them HTML. If you do, they will ignore you.


© Copyright 2015, 2016 Roger Price < hosts.allow at rogerprice dot org >
In order to facilitate access from all browsers, now and in the future, these pages conform to the International Standard ISO/IEC 15445 and the corresponding W3C Recommendations. Last change: sam. déc. 24 18:33:02 CET 2016

Creative Commons License Valid HTML 4.01! Valid CSS! Level A conformance icon,            W3C-WAI Web Content Accessibility Guidelines 1.0 Valid ISO 15445!
http://rogerprice.org/hosts.allow