• 1 — Introduction
  • 1.1 — Change log
• 2 — TCP Wrappers
  configuration file
  • 2.1 — client_list summary
  • 2.2 — Example
• 3 — Firewalls based on
  iptables
  • 3.1 — SuSEfirewall2
  • 3.2 — Running SuSEfirewall2
    on Debian
• 4 — The Bash script
  • 4.1 — hosts.allow.ctl options
  • 4.2 — iptables options
  • 4.3 — Exit codes
  • 4.4 — Bash programming
    notes
• 5 — Using the Bash script
  • 5.1 — Limited access
    to sshd
  • 5.2 — Blocking
    undesirables from httd
  • 5.3 — Blocking Windows
    Telemetry
• 6 — Helper script to display
  journal
• 7 — Conclusion
• 8 — Documentation
• 9 — Downloads
• 10 — Legal

Using iptables and ipsets to replace TCP Wrappers

Is it possible, is it practical?

This page addresses the question of a possible iptables representation of TCP wrappers and its configuration file hosts.allow using the openSUSE and Debian distributions of GNU/Linux. The techniques are based on the SUSE stateful packet filter SuSEfirewall2 which is a Bash script. However the iptables techniques used are applicable across a wide range of distributions.

The reader is assumed to be an experienced administrator of GNU/Linux systems and have a working knowledge of iptables.

Health and safety warning: The techniques described by this page require root access to the system, which can be dangerous. You have been warned. In addition, use of the custom extensions to the SUSE firewall SuSEfirewall2 is not supported by SUSE.

1 — Introduction

TCP wrappers have been removed from openssh at release 6.7. See the Linux Weekly News article. This change reached openSUSE in release 13.2. Since that release users of TCP wrappers must now look for other means of securing services such as openssh. A common response is to say Use the iptables firewall, but iptables runs at network level whereas TCP wrappers is an application level mechanism. The notion of a layered defence in depth is weakened and it is questionable that iptables can replace TCP wrappers.

In an effort to address this question, a Bash script proposed by this page attempts to convert hosts.allow, the TCP wrappers configuration file, to a list of IP numbers for use in an ipset collection, and a set of iptables rules to be placed in a hook function to the SUSE firewall SuSEfirewall2. The proposed script also explores some of the extensions that become possible once such a conversion process is available.

If you are just interested in my conclusions, and not the technical details, then go straight to the Conclusions. For those who enjoy the gory details, read on.

The proposed Bash script takes the TCP Wrappers configuration file, often /etc/hosts.allow and extracts each rule in turn. For the purposes of the Bash script, the rules have the simplified form daemon_list : client_list : script : action and for each rule the Bash script builds ipsets and iptables rules as shown in the following table:

1.1 — Change log

• 2015-12-31 Version 1.0.
• 2016-01-28 Version 1.1. Added IPv4 ranges and subnet optimizer.
• 2016-04-06 Version 1.2. Fixed missing maxelem. Use files instead of text variables. Approx 10 times faster. Test for Bash interpreter.
• 2017-09-20 Version 1.3. Improved --list makes clear the relationship between iptables rules and ipsets. Added --input.
• 2018-01-06 Version 1.4. Support for Debian 9, IPv4 only.
• 2018-02-25 Version 1.5. Support for MAC addresses.

2 — TCP Wrappers configuration file

The configuration file used by TCP wrappers is usually found in /etc/hosts.allow   In this file, comment lines start with a # and are ignored. Blank lines are ignored. Rules are separated by newlines. Within a rule, the separators are space and tab, and in lists the comma is also used as a separator.

Rules may be folded over several lines using the \ before newline convention.

Notes on the configuration specification

1. rule: The rule is to be written on a single line. For convenience, the line may be folded by escaping the new-line. A TCP wrapper rule translates into one or more iptables rules.
2. d: In the original TCP wrappers, the only daemons allowed were those for which TCP wrapper support was compiled in. The daemons were specified with the name of the binary, typically the service name with a final letter d, e.g. sshd. The proposed hosts.allow.ctl script accepts all services specified in /etc/services and allows omission of the additional letter d. The script knows about the services quotad, vid, rpasswd, maitrd, raid-cd, blackboard and epmd which end in a letter d. The service may also be specified by it's associated port number.
3. d: The Bash script hosts.allow.ctl assumes that the protocols for each service are TCP and UDP unless specified otherwise.
4. d: The suffixes [tcp-only] and [udp-only] are case insensitive, but I write them in lower case. They are directly attached to keyword ALL, to the daemon name and to the port number and limit the rule for that service to the specified protocol. There is no intervening white space.
5. action: Keywords such as ALL, ALLOW, DENY are case insensitive, although I always write them in upper case.
6. script: The grammar shown for those options which are not supported is a simplification.

2.1 — client_list summary

The client list translates into an IPv4 and an IPv6 ipset collection. The client list may contain sub lists, and the elements in the collection are labelled to show their origin. This helps to identify in an ipset listing including multiple sources, exactly which elements contributed to the action.

Basic client list elements

Here is a summary of those elements of the original TCP wrappers client_list which are wholly or partially supported by the proposed script.

localhost

The domain name localhost is reserved by RFC 6761 for the IP loopback address, for example 127.0.0.1 and ::1. The proposed script resolves localhost as 127.0.0.1 in IPv4 and ::1 in IPv6.

ALL

The client ALL is an alias for 0.0.0.0-255.255.255.255 which covers the entire IPv4 address space and which is represented by subnets 0/1 and 128/1, but a more detailed analysis of the traffic is possible if 256 /8 subnets are used. These may be placed in file /etc/fw_custom/ALL with the Bash invocation:

 cd /etc/fw_custom ; : > ALL ; for i in {0..255}; do echo $i/8 >> ALL; done

or, if you like a lot of detail, try 65536 /16 subnets

 cd /etc/fw_custom ; : > ALL 
 for i in {0..255}; do for j in {0..255}; do echo $i.$j/16 >> ALL; done; done

/directory/file

It is sometimes convenient to place a specific white list or blocking list in a separate file and refer to that file where needed. If the list is a list of IPv4 subnetworks, then they are assumed to have been sanitized — i.e. there are no overlaps or duplicates. If you want the optimizer to clean up the list, then specify the hosts and networks in the list using the range notation.

MAC address

For greater precision, it is possible to specify the MAC address of equipment which is to be allowed or denied. For example, this is especially useful for mobile equipment which requires access to a server from a country which is normally blocked. The MAC addresses may be expressed as de:ad:be:ef:01:12 or DE-AD-BE-EF-01-12 . I.e. the format is case-insensitive and the separator may be a colon : or a hyphen - . MAC addresses may be mixed in with the other clients such as IPv4 and IPv6 addresses. The hosts.allow.ctl script will place the MAC addresses in ipsets of type hash:mac  and the rest in ipsets of type hash:net . The MAC address 00:00:00:00:00:00 is not allowed by program ipset .

host_name

The IPv4 and IPv6 values are taken from /etc/hosts . For example if the file /etc/hosts contained the line

 10.218.0.13  maria

then the host name maria would be resolved to 10.218.0.13.

FQDN, Fully Qualified Domain Name

E.g. www.kernel.org   Note that a FQDN may correspond to more than one IPv4 or IPv6 address, and that those addresses may depend on your geographic position, and may also change with time. The proposed Bash script generates ipsets of IPv4 and IPv6 numbers from the values discovered by dig when the script is run. To update the numbers, you have to re-run the script to rebuild the ipsets.

Note also that a business may be sending you spam traffic from sources not specified by A or AAAA records, for example MailChimp.

ccTLD   country code top level domain

This is the only support provided by the proposed Bash script for the TCP wrapper's incomplete domain specification. The script makes use of a public service of IP address block lists provided by IPverse IP address block lists. For those countries supported by IPverse IP address block lists you have only to specify the top level domain of the country, preceeded by a dot, to include the entire IPverse IP address block lists block list. E.g. .cn to block traffic from IP addresses in the PRC.

Note that this is not exactly the same as TCP wrappers' .cn . TCP wrappers perform a reverse DNS lookup when the packet arrives. If the tld is available, and is .cn, then the packet would be detected, no matter which country it had come from. The proposed Bash script does not perform reverse DNS lookups, and is strictly geographically based.

Block lists for those countries which are not supported by IPverse IP address block lists for example .uk , can be found at Country IP blocks. Select the format Netmask, click on Create ACL make a local copy of the resulting file. Place the address of this file in the client_list. For the UK it is convenient to use file /etc/fw_custom/.uk .

IPv4 dotted quad

This is a direct specification of an IPv4 host address, e.g. 212.27.48.10

IPv4 dotted quad/CIDR

This is a direct specification of an IPv4 network, e.g. 212.27.48.0/24 . Note that 0/0 is not allowed.

Partial IPv4 dotted quad/CIDR

This is an abbreviation of the IPv4 dotted quad/CIDR element in which trailing numbers on the quad are omitted. Their values are taken as 0. For example 129.63/16 is an abbreviation of 129.63.0.0/16. Another example is 10/8 for an entire private network.

IPv4 dotted quad/net mask

This is a direct specification of an IPv4 network, e.g. 212.27.48.0/255.255.255.0 = 212.27.48/24

Incomplete IPv4 dotted quad

This specification of an IPv4 class A, B or C network is specific to TCP wrappers. For example 212. represents 212/8, 212.27. represents 212.27/16 and 212.27.48. represents 212.27.48/24. Note that the incomplete dotted quad notation always ends in a dot.

[IPv6_address]

This is a direct specification of an IPv6 host address, e.g. [2a01:e0c:1::1]. The brackets are required.

[IPv6_address/CIDR]

This is a direct specification of an IPv6 network, e.g. [2a01:e0c:1::1/124]. The brackets are required.

Additional client list elements

Here is a summary of the additional element notations which have been added to the client_list by the proposed script. These additional client list elements are also translated into IPv4 and IPv6 ipset collections. Note that only host ranges are subject to subnet optimization. Subnets specified using the basic client list elements are assumed to have been sanitized already.

IPv4 host ranges — A.B.C.D-W.X.Y.Z

It is sometimes convenient to express a group of hosts as a range of IPv4 numbers, e.g. 10.218.0.10-10.218.0.99   . The proposed bash script will express this range as a set of IPv4 subnets. This set of subnets, together with any sets produced by other ranges specified in the client list will be optimized to the least number of subnets. The optimization process is highly recursive and is slow. You will get a warning or even an error message if you over-use this facility. For example, the range 129.67.0.1-129.67.255.255 produces 65535 subnets to be optimized. Bash is inefficient and such a job would take a couple of hours. You can reduce the load considerably by either

• Creating a second rule to handle the specific case of the host 129.67.0.0, and then the range 129.67.0.0-129.67.255.255 will be seen immediately to be the equivalent of 129.67/16   .
• Splitting the range into 129.67.0.1-129.67.0.255 and 129.67.1.0-129.67.255.255, which corresponds to 510 subnets and which the optimizer will reduce to 16 in less than 90 seconds (Dell Pecision 690, 4 cores, 32 Gbyte) : 129.67.0.1 129.67.0.2/31 129.67.0.4/30 129.67.0.8/29 129.67.0.16/28 129.67.0.32/27 129.67.0.64/26 129.67.0.128/25 129.67.1.0/24 129.67.2.0/23 129.67.4.0/22 129.67.8.0/21 129.67.16.0/20 129.67.32.0/19 129.67.64.0/18 129.67.128.0/17   .

Incomplete IPv4 host range — a single host

It is possible to specify a single host in such a way that it is included in the optimization. This is done by placing a hyphen after the dotted quad address, e.g. 129.67.7.91-   .

2.2 — TCP wrappers example

This example shows some of the interesting things that could be done with the original TCP wrappers.

 # hosts.allow for modest home server
 upsd :         localhost                              :ALLOW
 sshd, rsyncd : .math.bigU.edu, 10.8.0/24, localhost :\
                 spawn (echo "Accepted access to %d from %c" | \
                 /bin/mail -r hosts.allow@modest-server.tld\
                           -s '%s (host %h/maria) accepts access to %d from %c'\
                           admin@domain.tld) &         :ALLOW       
 ALL :          ALL                                    :DENY

In this example we see 3 rules:

1. The NUT (UPS management) daemon is accessible from the local host only. This rule can be reproduced using iptables.
2. ssh and rsync are available on this server for all the hosts in the mathematics department at Big University, for the VPN users and the local host users. Whenever an ssh or rsync client is accepted, a mail is sent to the sysadmin. The %d %c %s and %h parameters are defined in man 5 hosts_access. This rule cannot be reproduced using iptables.
3. Nothing else is to be accepted for those applications using TCP wrappers.

Figure 3. Simplified netfilter packet flow showing INPUT, OUTPUT, FORWARD, PREROUTING and POSTROUTING chains. This chart does not distinguish the work done at different network layers. The numbers in brown italics attached to the tables correspond to the steps described in Chapter 6 of the iptables Tutorial: traversing of chains and tables.

The hosts.allow.ctl script places iptables rules in filter:INPUT , filter:OUTPUT , filter:FORWARD , raw:PREROUTING and raw:OUTPUT .

3 — Firewalls based on iptables

There are many firewalls based on iptables running on GNU/Linux and the BSD's. I address here only those with which the proposed hosts.allow.ctl Bash script has been tested.

For an introduction to iptables, see the iptables Tutorial 1.2.2. The iptables packet flow is summarised in a simplified form in Figure 3, adapted from the figure by xkr47. The numbers in brown italics attached to the tables correspond to the steps described in Chapter 6 of the iptables Tutorial: traversing of chains and tables. Figure 3 does not take into account the work done at the different network layers. Here is a more complex figure by Jan Engelhardt which does.

3.1 — SUSE firewall 2

The SUSE firewall SuSEfirewall2, currently version 2, is integrated into YaST and relies on a Bash script /sbin/SUSEfirewall2 which rebuilds the iptables rules as a batch job at each system start.

The SUSE script sets up the iptables rules by defining a sequence of BASH functions which build up the set of rules. The SUSE functions populate the chains INPUT, OUTPUT, FORWARD, PREROUTING and POSTROUTING , plus additional SUSE-specific chains input_int, input_ext, input_dmz, forward_int, forward_ext, forward_dmz, reject_func and conditional chains of the form target_if_direction_zone.

The SUSE firewall offers the user three levels of configuration of the firewall:

1. The basic configuration is presented through a GUI which is part of YaST: YaST => Security and Users => Firewall.
   • Startup: Start firewall automatically at boot.
   • Interfaces: Network interfaces are assigned to External zone, Internal zone or DMZ.
   • Allowed Services: Which services are allowed on each zone.
   • Is Masquerading required?
   • Broadcast configuration.
   • Logging level for accepted and rejected packets.
   • Custom rules, for example port forwarding.
2. Non-basic configuration such as port forwarding is defined by declaring SUSE-defined variables in file /etc/sysconfig/SuSEfirewall2 .
3. There are five hook functions available for further customisation of the SUSE firewall. These functions after_chain_creation, before_port_handling, before_masq, before_denyall and after_finished normally have do-nothing definitions. A custom modification might consist in specifying additional iptables rules in one of these hook functions. To activate the custom modification, place the address of the file containing the modified hook function in variable FW_CUSTOMRULES in file /etc/sysconfig/SuSEfirewall2 and the modified function will be run as part of the batch job which sets up the SUSE firewall. Note that using these hook functions is at the responsibility of the system administrator. SUSE do not support such extensions.

TCP Wrappers support with SuSE firewall

The SUSE firewall is a Bash script which generates a stateful packet filter at each system start. The SUSE script is easily transferable to other distributions such as Debian, and provides easy access for further Bash hook scripts. On SUSE systems it comes with a GUI interface for simple setup. This GUI is a part of the YaST system administration.

The proposed Bash script hosts.allow.ctl generates IP sets representing the original client_lists, and these are connected to the firewall's iptables rules with additional iptables rules placed in the hook function after_chain_creation . The other hook functions before_port_handling , before_masq , before_denyall  and after_finished  retain their do-nothing definitions provided by SUSE.

The hosts.allow.ctl script places the hook function after_chain_creation  in file /etc/fw_custom/fw_hosts.allow. The user must activate the hook function by setting variable FW_CUSTOMRULES in file /etc/sysconfig/SuSEfirewall2 The default is to set

 FW_CUSTOMRULES="/etc/fw_custom/fw_hosts.allow"

but the directory can be changed with the scripts's -o option.

Self documentation for SUSE firewall

A lot of thought has gone into the SUSE firewall, but there are few comments in the code to help the reader. However the names of the functions in the script are helpful, and it is instructive to add these names to the rules they create. I suggest modifying the file /sbin/SuSEfirewall2 to add a helpful comment to all those iptables commmands which create firewall rules, i.e. those commmands which have -A and -I options. For example

 # Comment to be added to all -A and -I iptables commands
 CMNT='eval echo -m comment --comment \"${FUNCNAME[1]:-}[${BASH_LINENO[0]}]-${FUNCNAME[0]}[${LINENO}]\"'

 function allow_basic_established()
 {  # needed for dhcp and dns replies
    local iptables
    for iptables in "$IPTABLES" "$IP6TABLES"; do 
        $LAA $iptables -A INPUT ${LOG}"-IN-ACC-EST " -m conntrack --ctstate ESTABLISHED $( $CMNT )
        $iptables -A INPUT -j "$ACCEPT" -m conntrack --ctstate ESTABLISHED $( $CMNT )
...
    done
    # need to accept icmp RELATED packets (bnc#382004)
    $LAA $IPTABLES -A INPUT ${LOG}"-IN-ACC-REL " -p icmp -m conntrack --ctstate RELATED $( $CMNT )
    $IPTABLES -A INPUT -j "$ACCEPT" -p icmp -m conntrack --ctstate RELATED $( $CMNT )
    $LAA $IP6TABLES -A INPUT ${LOG}"-IN-ACC-REL " -p icmpv6 -m conntrack --ctstate RELATED $( $CMNT )
    $IP6TABLES -A INPUT -j "$ACCEPT" -p icmpv6 -m conntrack --ctstate RELATED $( $CMNT )
 }

Notes:

1. Be patient, there are 159 lines to update in SUSE 13.2, but a true l33t with vi and sed would have no problem. Here is an example of the commented output of command iptables -n --line-numbers -t filter -L INPUT .
2. The comment to be displayed should not include white space.
3. An iptables command may have more than one comment, but I have not seen this documented anywhere.

3.2 — Running SUSE firewall 2 on Debian

It is a simple matter to run SuSEfirewall2 on Debian, however thare is no .deb package. I installed by copying the following files from openSUSE 42.3 to Debian 9.

You have to stop and disable any other firewall, and enable SuSEfirewall2 with commands systemctl enable SuSEfirewall2 and systemctl enable SuSEfirewall2_init. The firewall will start at system boot, and can be stopped and restarted using commands systemctl stop SuSEfirewall2 and systemctl start SuSEfirewall2  .

The script hosts.allow.ctl is installed on Debian exactly as with openSUSE and used in the same way.

To configure SuSEfirewall2 edit the variables defined in file /etc/sysconfig/SuSEfirewall2 using a text editor. The variables in the file are fully commented.

Individual services may be enabled using commands such as SuSEfirewall2 open EXT TCP ssh  which sets variable FW_SERVICES_EXT_TCP="ssh" .

4 — The Bash script

4.1 — hosts.allow.ctl options

Where the options are

--dry-run or -dry or -d

This is a dry run, not a production run. Give the ipsets names with the suffix -dry , and display the hook function, but do not create it. One or the other of the options --dry-run and --production must be specified. This option can only be used by root since it requires access to ipsets. The default output directory is /etc/fw_custom . The user root should create this directory and make it readable and writable by root only.

--help or -h

Read about the options.

--input or -i

Specify the input file rather than piping it into hosts.allow.ctl.

--list

Display a listing of the iptables and ipset configuration and then exit. In the ipset listings, subnets for which no traffic is recorded are omitted.

--ipv6 or -IPv6

Create and list rules for IPv6. By default rules for IPv6 are not generated. IPv6 rule creation has not been tested.

--out-dir or -o

Use the specified output directory rather than the default /etc/fw_custom. This directory will be used to restore ipsets and for the hook function file.

--production or -prod or -p

This is a production run. Give the ipsets their correct names, and generate the file which contains the hook function. One or the other of the options --production and --dry-run must be specified. This option can only be used by root, and cannot be used while the firewall is running, since it modifies ipsets in use by the kernel. The default output directory is /etc/fw_custom . The user root should create this directoty, readable and writable by root only.

--quiet or -q

Do not display the hook function during dry runs. The default is to display the hook function during dry runs.

--list

Display a listing of the iptables, ip6tables and the ipset configuration and then exit. In the ipset listings, subnets for which no traffic is recorded are omitted. To include the display of IPv6 tables, place the option --IPv6 before the option --list,

--REJECT

Use target -j REJECT rather than -j DROP for unwanted datagrams and packets. With this option a matching rule returns reject-with-icmp-port-unreachable. The default is -j DROP.

--SUSE-REJECT

Use SUSE specific chain reject_func rather than -j DROP for unwanted datagrams and packets. The chain provides TCP, UDP and other specific return values. See iptables -t filter -n -L reject_func for details.

--UA

Specify user agent identification for wget. The default is hosts.allow.ctl v1.5 ipset builder. https://rogerprice.org/hosts.allow.

--verbose or -v

Display a copious console trace of the internals of hosts.allow.ctl. There are approximately 10 lines of trace per ipset rule. You were warned. By default there is no trace.

--version

Display script name and version, e.g. hosts.allow.ctl v1.5 .

configuration-file

The traditional TCP wrappers assume that there is only one configuration file, /etc/hosts.allow, and this assumption is carried over into the hosts.allow.ctl script. At each creation of an ipset from a client_list, the previous ipsets are destroyed. The dry run ipsets are separate from the production ipsets.

4.2 — iptables options

The iptables rules generated by hosts.allow.ctl can be modified for each rule by placing optional specifications in the TCP Wrappers shell script field. The options are:

LOG log_options

The iptables rule generated by this hosts.allow rule will be accompanied (preceeded) by a second iptables rule which logs the traffic subject to the ALLOW or DENY. Only the traffic subject to the ALLOW or DENY is logged, and only the datagrams and packets in state NEW are logged. If the traffic does not satisfy the match in an ALLOW, it is not logged. If the traffic does not satisfy the match in a DENY, it is not logged. The logging is not subject to rate limiting.

LOG-WITH-LIMIT log_options limit_option

As LOG, but in addition the logging is subject to the rate limiting used by default in SuSEfirewall2. In addition to the log_options the limit_option is available without having to specify LIMIT .

log_options

--log-prefix prefix

The logging of traffic which matches the hosts.allow rule is watermarked with a specific prefix to help identify it, and to ease the debugging of the rules. The text supplied for the log prefix shall not contain white space, the backslash \ or the double quote character ", e.g. --log-prefix "not from \"them\"" wrongly contains both spaces and ". The default prefix is hosts.allow[L.R.S] where L is the line number of the rule in the configuration file, R is the rule number, and S is the service name as shown in /etc/services . I have never had to specify this option, since the default value has always suited my needs.

--log-level level

The traffic which matches the hosts.allow rule is logged at a specific priority. The default is warning which may be specified as 4 . For the other levels, see RFC 5427 clause 3 SyslogSeverity. The levels may be referenced by number or name.

LIMIT limit option

The traffic which matches the rule is subject to rate limiting, as specified by limit option .

limit_option

--rate number/period

The traffic which matches the hosts.allow rule is rate limited at the specified rate, where number is an integer, and period is one of second  , minute or hour . The default value is 3/minute .

TRACE

Log all those firewall rules which participate in the processing of the datagrams or packets identified by the TCP Wrapper rule. This option can quickly fill up your disk. You have been warned. There is at present no rate limiting, since this option is intended for debugging only, not for regular use. This option is helpful in showing that the iptables rules generated by hosts.allow do indeed perform the required filtering. The proposed Bash script will warn you if you try to use the options TRACE and ALLOW with the same rule.

filter:INPUT

By default, the iptables rules generated by hosts.allow.ctl are placed in filter:INPUT . It is not normally necessary to specify this option.

filter:OUTPUT

The iptables rules generated by hosts.allow.ctl are placed in filter:OUTPUT . See Figure 3. In this case, the daemon_list becomes a list of services on the remote machine(s), and the client_list becomes a ipset specifying a set of remote machines. This option may for example be used to allow employees to visit only carefully selected management approved sites, or to forbid access to sites with no business interest, such as Facebook. Remember that such sites have many IP addresses.

filter:FORWARD

The iptables rules generated by hosts.allow.ctl are placed in filter:FORWARD . See Figure 3. In this case, the daemon_list becomes a list of services on the remote machine(s), and the client_list becomes a ipset specifying a set of remote machines. This option may for example be used to prevent Windows 10 boxes on a subnet reporting all their activity to Microsoft as part of Microsoft's Telemetry.

4.3 — Exit codes

4.4 — Bash programming notes

1. Little attempt is made to adopt a Bash l33t coder style such as the use of LDC=":" in SUSE's /sbin/SuSEfirewall2 . For example, all the if...then...else...fi are written out in full.
2. Parameter expansion is always simple. There is no case modification, no substring removal, no alternate values. There are a few default values.
3. Folks who think it is modern to use ++ to add 1 will be disappointed. It fails in opensuse.
4. Flag variables use integer values 0 and 1. The default Bash notions of true and false are not used.
5. The function read-clients which reads the client_list is recursive, calling itself to read sublists, including those sublists created by utility program dig. The function IP_TREE_insert used to handle IPv4 ranges, and it's subfunctions are particularly recursive and therefore slooow in Bash.
6. This script uses the Bash regular expresion operator =~ introduced with Bash version 4. See also the IT World article.
7. The script contains several cat file | while read... loops which Bash executes in subshells. To overcome the invisibility of subshell variables in the outer shell, the script uses temporary files for variables. These temporary files are cleared away by an exit trap.
8. The input file is piped in to avoid having to use yet another cat file | while read... construction which would place the main working code in a subshell whose variables do not escape.
9. The trace option --verbose which activates the function msg-dbg is helpful for debugging, but is very verbose, typically 10 lines per ipset element. You have been warned.
10. No attempt has been made to improve performance of the Bash script with parallelisation. On a Dell Precision T7500 with 6 cores and 48 Gbyte memory, the script will generate more than 30000 ipset elements per minute using one core at a time. For the efficiency of ipsets see the article by daemonkeeper. The script has been tested with network/CIDR files containing 155000 networks.
11. The proposed script has been run through ShellCheck and some obvious errors corrected, but the style suggested by that Haskell program seems unhelpful, and in some cases just wrong.

5 — Using the Bash script

The following examples use the SUSE distribution and the SUSE firewall SuSEfirewall2. The firewall has been connected to the extensions defined by the Bash script by setting variable FW_CUSTOMRULES in file /etc/sysconfig/SuSEfirewall2 to:

 FW_CUSTOMRULES="/etc/fw_custom/fw_hosts.allow"

Remember that the file fw_hosts.allow contains hook function after_chain_creation which sets the required iptables rules.

5.1 — Example: Limiting access to sshd

Overview

We wish to allow access to the ssh service to a limited number of clients. All other connections are to be refused. The original TCP Wrappers rules were:

sshd : .BigU.edu 10.218.0., localhost : shell script : ALLOW
sshd : ANY                                           : DENY

In the first rule, the very useful TCP wrapper incomplete domain specification .BigU.edu which was resolved at run time is not available with SuSEfirewall2. All we have is an approximation such as 129.63.   . The shell script cannot be used with iptables; the nearest we can get is to log the successful connections.

The effect of the second rule is obtained by closing the SUSE firewall to ssh traffic. Since the iptables translation of the first rule will be executed before the final traffic rejection, we obtain a generally closed ssh service with limited access by a few specified clients.

The ALLOW rule will be replaced by the rule:

sshd[tcp-only] : 129.63. 10.218.0., localhost : LOG-WITH-LIMIT : ALLOW

and the DENY rule will be satisfied by removing all access to the service in the YaST SuSEfirewall2 administration.

We test the setup with with a temporary TRACE rule before specifying the LOG_WITH_LIMIT option. Normally you should not use TRACE and ALLOW options in the same rule – it will flood your disk.

Details

The first version of the TCP wrapper demonstration configuration demonstration.conf uses the TRACE option. Note that it is risky to use TRACE on an ALLOW rule, sine the trace will be voluminous and will flood journald. However we do it here to show the effect. The floods of journald records have been swept away.

1    # Limited access to ssh service
2    sshd[tcp-only] : 129.63., 10.218.0., localhost : TRACE : ALLOW

First we try a dry run with command hosts.allow.ctl --dry-run < demonstration.conf and see (an edited version of) the proposed hook function containing the iptables rules:

 function fw_custom_after_chain_creation { true 
  # Restoring ipsets when firewall (re)starts
  ipset destroy; ipset restore < /etc/fw_custom/hosts.allow.ipsets
  iptables -t raw -A PREROUTING -p tcp --dport 22 -m set --match-set hosts.allow-rule-1-inet src.dst -j TRACE -m comment --comment "[2.1.ssh]"
  iptables -A INPUT -p tcp --dport 22 -m set --match-set hosts.allow-rule-1-inet src.dst -j ACCEPT -m comment --comment "[2.1.ssh]"
 }

where the first iptables rule is the translation of the TRACE option and the second represents the configuration file sshd rule, as confirmed by the comments. The client_list has been translated to a dry run ipset which we display using command ipset list hosts.allow-rule-1-inet-dry :

 Name: hosts.allow-rule-1-inet-dry
 Type: hash:net
 Revision: 6
 Header: family inet hashsize 64 maxelem 65536 comment
 Size in memory: 1688
 References: 0
 Members:
 10.218.0.0/24 comment "[2.1]"
 129.63.0.0/16 comment "[2.1]"
 127.0.0.1 comment "[2.1]"

We are now ready to install the SuSEfirewall2 modification. We stop the firewall with the command systemctl stop SuSEfirewall2.service , create the working ipsets with the command hosts.allow.ctl --production < configuration.conf , and restart the firewall with the command systemctl start SuSEfirewall2.service .

For the moment we leave the ssh service available in the SUSE firewall, and attempt an ssh connection from client kananga 10.218.0.16 to server pinta 10.218.0.14. Once the connection is established the server pinta the command hosts.allow-journal + ssh --since 13:02 reports:

         Current boot
 Dec 14 13:02:36 pinta SuSEfirewall2[20560]: Firewall custom rules loaded from /etc/fw_custom/fw_hosts.allow
 Dec 14 13:02:36 pinta hosts.allow.ctl[20581]: Hook file /etc/fw_custom/fw_hosts.allow: function fw_custom_after_chain_creation
 Dec 14 13:02:36 pinta hosts.allow.ctl[20582]: restores ipsets from /etc/fw_custom/hosts.allow.ipsets
 Dec 14 13:26:53 pinta kernel: TRACE: raw:PREROUTING:policy:3 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 Dec 14 13:26:53 pinta kernel: TRACE: filter:INPUT:rule:4 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 Dec 14 13:26:53 pinta kernel: TRACE: raw:PREROUTING:policy:3 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 Dec 14 13:26:53 pinta kernel: TRACE: filter:INPUT:rule:2 IN=wlp0s29f7u1 OUT= SRC=10.218.0.16 DST=10.218.0.14 ...
 ...

and command iptables -n --line-numbers -t filter -L INPUT reports:

 Chain INPUT (policy DROP)
 num target     prot opt source       destination         
 1   ACCEPT     all  --  0.0.0.0/0    0.0.0.0/0   /* set_basic_rules[768] */
 2   ACCEPT     all  --  0.0.0.0/0    0.0.0.0/0   ctstate ESTABLISHED /* allow_basic_established[685] */
 3   ACCEPT     icmp --  0.0.0.0/0    0.0.0.0/0   ctstate RELATED /* allow_basic_established[699] */
 4   ACCEPT     tcp  --  0.0.0.0/0    0.0.0.0/0   tcp dpt:22 match-set hosts.allow-rule-1-inet src /* [2.1.ssh] */
 5   input_ext  all  --  0.0.0.0/0    0.0.0.0/0   /* fork_to_chains[1488] */
 6   LOG        all  --  0.0.0.0/0    0.0.0.0/0   limit: avg 3/min burst 5 /* finish_chains[1507] */ LOG flags 6 level 4 prefix "SFW2-IN-ILL-TARGET "
 7   DROP       all  --  0.0.0.0/0    0.0.0.0/0   /* finish_chains[1508] */

This shows that the connection was indeed accepted when in state NEW by filter:INPUT:rule:4 which carries the comment [2.1.ssh]. Thereafter the connection had status ESTABLISHED and was accepted by the SUSE rule allow_basic_established[685] .

• Note 1: To clarify the operation of the SUSE firewall, I have added the following options onto every iptables instruction: -m comment --comment "${FUNCNAME}[${LINENO}]" . This provides the additional comments /* ... */ that you see in the iptables output.
• Note 2: The trace output is voluminous, I show only a very small part.
• Note 3: You can display further details such as the volume of the traffic subject to each rule with the iptables -v option, e.g. iptables -v -n --line-numbers -t filter -L INPUT . This option also explains the apparent anomaly of filter:INPUT:rule:1 which looks as if it accepts everything. In fact it only applies to loopback traffic, which is always accepted.
• Note 4: The command iptables -t filter -n -L INPUT will only show the details of filter:INPUT if the SUSE firewall is running.

When we are sure that iptables is doing its job, we use YaST to turn off the ssh service in the SuSEfirewall2, and replace the hosts.allow rule by

 1    # Limited access to ssh service
 2    sshd[tcp-only] : 129.63., 10.218.0., localhost : LOG_WITH_LIMIT : ALLOW

The DENY of all others is expressed by turning off the ssh service in the SuSEfirewall2.

5.2 — Example: Blocking undesirables from httpd

I run a modest http server so that I can test things before putting them on my public site. There is very little real traffic, but the file /var/log/apache2/access log fills up as secret government agencies, organised criminels, content thieves and many others try to take as much as possible. Lists are available of known undesirables, and these can be used with TCP wrappers to filter the traffic getting through to the http server.

I use the IPverse IP address block lists which provide the IPv4 and IPv6 addresses for many countries. Since these change regularly, it will be necessay to rerun the hosts.allow.ctl script regularly to update the ipsets. Just as an example, I have chosen some random undesirable countries. I live in one of them – this will help testing. Your list may well be very very different.

Some country IP lists are not available at IPverse IP address block lists, for example .uk . You can find a list for the UK at Country IP blocks. Select the format Netmask, click on Create ACL make a local copy of the resulting file. Place the address of this file in the client_list.

 1    # Keep "undesirables" away from web server
 2    httpd[tcp-only] : .be .ca .ch .fi .fr .li .lu .mc .va /etc/fw_custom/.uk\
                      : LOG_WITH_LIMIT : DENY

After restarting the firewall and waiting a few minutes, the simplest way of checking that this is working is with the command

 maria:~ # iptables --verbose -n --line-numbers -t filter -L INPUT | grep dpt:80
 8  27 1520 DROP  tcp  --  * *  0.0.0.0/0  0.0.0.0/0  tcp dpt:80 match-set hosts.allow-rule-1-inet src /* [2.1.http] */

Figure 6. The efficiency of ipsets vs iptables rules. Source: daemonkeeper.net. Please read the full article.

where

• 8 is the line number in filter:INPUT.
• 27 is the number of packets dropped.
• 1520 is the number of bytes dropped.
• DROP is the target of the rule.
• tcp is the protocol.
• 0.0.0.0/0 is the source IP address: the iptables rules do not do the filtering, this is done by the ipset.
• 0.0.0.0/0 is the destination IP address.
• hosts.allow-rule-1-inet is the client_list which contains thousands of IPv4 network addresses.
• [2.1.http] shows which of our rules is the source of the iptables rule.

It is clear from this example that placing the client_list in an IP set is a far more elegant solution than generating thousands of iptables rules. Read a very interesting article showing how much more efficient ipsets are.

5.3— Example: Blocking Windows Telemetry

Microsoft provides a Telemetry service for Windows 7, 8 and 10. This service sends a lot of user data to Microsoft so that Microsoft can improve the quality of Microsoft services. Many people understand this to mean Sell your data and take offence, considering it to be an invasion of their privacy. There are articles explaining how to turn Telemetry off.

However system administrators in businesses in which employees and guests bring their devices to work, may well find themselves thinking of blocking Microsoft Telemetry in a firewall. Others have thought of this and propose black lists of Telemetry servers. The example I use here is provided by BlockWindows, but I do not caution this list as in any way accurate.

To use the blocking list you have a choice:

1. Copy the contents of file hostslist provided by BlockWindows into a local file, say /etc/fw_custom/BlockWindows, and make those contents one single line, with spaces between the entries, not newlines. Add the local file name /etc/fw_custom/BlockWindows to the client_list. hosts.allow.ctl will place the IP addresses corresponding to the FQDN's in the hostslist into an ipset.

    1    # Block access from local machines to MSFT telemetry
    2    all : /etc/fw_custom/BlockWindows : filter:OUTPUT  LOG-WITH-LIMIT : DENY
    3    all : /etc/fw_custom/BlockWindows : filter:FORWARD LOG-WITH-LIMIT : DENY

2. Use the (highly) experimental TCP Wrappers extension MSFT-TELEMETRY to the client_list.

    1    # Block access from local machines to MSFT telemetry
    2    all : MSFT-TELEMETRY : filter:OUTPUT  LOG-WITH-LIMIT : DENY
    3    all : MSFT-TELEMETRY : filter:FORWARD LOG-WITH-LIMIT : DENY

In both cases, it would probably be better after a while to remove the LOG-WITH-LIMIT, or at least set a low rate, in order to avoid filling the logs with useless data.

For the moment I have not tested this, since I am not a Windows user.

6 — Helper script to display the journal

The helper script hosts.allow-journal offers a simple way of picking out the records of interest to the firewall in the syslog journal. The script begins by picking out only those records created since the most recent boot and then applies further filtering.

The helper script first tries to use journalctl, but if this is not available, it will try to use the file /var/log/messages. When using file /var/log/messages only the records for the current day are displayed, and the --since option is not available.

Usage hosts.allow-journal options

Where the options are

--include RE

+ RE

Display only the journal records which satisfy the regular expression RE. The option may be used repreatedly and the effect is additive, it adds further expressions to the initial default RE which is SuSEfirewall|SFW2|hosts.allow|TRACE

--exclude RE

- RE

Exclude from the display any records which match the regular expression RE . This option may be used more than once, the effect is cumulative. The default value is <nil> .

--since hh:mm

-T hh:mm

Display only those journal records dated since this time today. For example --since 18:05 . By default there is no temporal filtering.

Example  In general you will want to add at least the service name used in the rule you are testing, for example:

 hosts.allow-journal + ssh

If your output is overwhelmed by the results of successive tests, note the time of your most recent test and use the --since option:

 hosts.allow-journal + ssh --since 03:19

7 — Conclusion

After some experiments using the SUSE GNU/Linux distribution, and the proposed hosts.allow.ctl Bash script, here is my reply to the question in the title, and some of my conclusions.

1. Can iptables and ipsets replace TCP Wrappers?:  The answer is partially. It is inaccurate and unhelpful to say You don't need TCP wrappers, use the firewall instead. TCP Wrappers operate at application level whereas the SUSE firewall which is based on iptables operates mainly at network level. The best practice of defense in depth is weakened, and the full power of TCP wrappers cannot be reproduced. The required custom modifications to the SUSE firewall are not supported by SUSE. The user is on his own in a difficult area since iptables is no easier to read or hack than sendmail.
2. TCP Wrappers are an example of declarative programming in which the programmer says what is to be done, without saying how. On the contrary, iptables is a prescriptive or imperative programming language in which the programmer says how the job is to be done. Many programmers, particularly those in the functional programming and logic programming areas believe that the declarative paradigm leads to simpler, clearer programs with fewer bugs. Migrating from TCP Wrappers to iptables removes this advantage, especially the simplicity of the configuration file.
3. It is possible to recover the declarative programming paradigm, and reproduce a useful subset of the facilities offered by TCP Wrappers by using a script to convert the hosts.allow configuration file to ipsets, accompanied with iptables rules to use them, and a hook function to tie the additional rules to the SUSE firewall.
4. The existence of such a script, with extensions to the hosts.allow format opens up the possibility of using TCP Wrapper style rules to manage some useful network filtering beyond that attempted by TCP Wrappers. For example:
   1. UDP filtering and filtering of any service, especially those which never included TCP wrappers, such as openvpn and the http server Apache.
   2. Updating white lists and block lists from Internet sources without changing the iptables rules.
5. The use of IP sets is an elegant and simple way of expressing the TCP Wrapper client_list, and preferable to adding yet more iptables rules.
6. In the longer term, possible extensions include filtering of outgoing and forwarded traffic, for example to support blocking of Windows telemetry.

8 — Documentation

• TCP Wrappers:  See the man pages hosts_access and hosts_options. The Wikipedia has an article which also discusses the history of the project. If you are looking for examples, Google finds many discussions on how to use TCP wrappers.
• iptables:  The man page for iptable is not much help if you are trying to learn — See the early chapters of the Iptables Tutorial 1.2.2 by Oskar Andreasson for an introduction. Chapter 10 is out of date and unreliable. Appendix E points to further resources.
• iptables:  The figure Packet flow in Netfilter and General Networking by Jan Engelhardt shows the packet flow at each network layer. Missing *nat INPUT chain, see the netfilter user mailing list, Re: Why isn't DNAT happening for host-originated packets?, 04 Dec 2015.
• ipset:  See the man page.
• iptables with ipset:  See the netfilter site which also has a FAQ and HOWTOs.
• iptables with ipset:  See Advanced Firewall Configurations with ipset by Henry Van Styn, dated Mar 19, 2012. See also Securing Your Server using IPSet and Dynamic Blocklists by Lin Song.
• Debugging:  See iptables debugging
• Linux Advanced Routing & Traffic Control HOWTO (LARTC) by Bert Hubert and others. An advanced text described as A very hands-on approach to iproute2, traffic shaping and a bit of netfilter..
• SUSE firewall customisation:  See examples in /etc/sysconfig/scripts/SuSEfirewall2-custom
• Finding network allocations:  I use the search function of TCPIPUTILS.com which offers itself as The ultimate online investigation tool, but you get more than that if you pay.

9 — Downloads

Either click to download or use command   wget -c https://rogerprice.org/hosts.allow/file_name

10 — Legal

The web page that you are reading is published under the Creative Commons Attribution-NoDerivatives 4.0 International (CC BY-ND 4.0) license.

The man pages discussing TCP Wrappers were written by Wietse Venema.

The Bash script hosts.allow.ctl is free as in freedom software: you can copy it, use it, redistribute it, and modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

The script is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this page. If not, see https://www.gnu.org/licenses/.

Comments, bugs, whatever: SUSE-specific? I read the opensuse@opensuse.org mailing list. They have some mailing list rules and mailing list archives. An iptables problem? There is a netfilter user mailing list. Read their rules carefully. Don't send them HTML. If you do, they will ignore you.

© Copyright 2005-2022 Roger Price < webmaster at rogerprice dot org >
In order to facilitate access from all browsers, now and in the future, these pages conform to the International Standard ISO/IEC 15445 and the corresponding W3C Recommendations.

Last change: Tue 08 Nov 2022 06:58:49 PM CET https://rogerprice.org/hosts.allow