Hercules Version 4: TCP/IP networking with Hercules

This page describes how to set up TCP/IP connectivity between a Hercules machine and the outside world.

Since Hercules runs as a user process under the control of a driving system (usually Linux/x86 or Windows), it does not have direct access to the driving system's network adapter. This presents a problem in establishing connectivity between the network and the TCP/IP stack of an operating system running under Hercules.

But thanks to a technique originally demonstrated by Willem Konynenberg, it is possible to establish a virtual point-to-point link between the TCP/IP stack running under Hercules and the TCP/IP stack of the driving system. The driving system is then used as a router to pass IP frames between the Hercules TCP/IP stack and the rest of the network, as shown in the below diagram.

        |     Linux/x86 Driving System   |
        |                                |
        +-------------+                  |
        |  Hercules   |                  +--------+
        |-------------|                  |  eth0  |
        |   OS/390    |      TCP/IP ------------------> Network
        |   TCP/IP    |                  ||
        |-------------|         |        +--------+
        |    CTCA     |         |        |
        ||         |        |
        +------|------+  |
        |  /dev/tun           tun0       |
        |      |                |        |
        |      +----------------+        |
        |       Virtual CTC link         |
        |                                |

The virtual CTC link is provided by the Universal TUN/TAP driver developed by Maxim Krasnyansky. This driver creates a tunnel which appears to Hercules as a character device (/dev/tun0 or /dev/net/tun) and appears to the driving system as a virtual network interface (tun0). The Hercules 3088 driver makes the tun device appear as a CTCA (Channel to Channel Adapter) to the S/390 operating system running under Hercules. Each end of the link has its own IP address which is distinct from the IP address of the driving system's real network adapter.

Windows users   should refer to Fish's CTCI-WIN web page which provides similar "TunTap" functionality on Windows platforms.

Installing the TUN/TAP Driver (Linux 2.4)

The TUN/TAP driver is delivered as part of the Linux 2.4 kernel, and if you are using one of the popular Linux distributions you will find that the TUN/TAP driver is already installed. If not, then you must rebuild the kernel with the configuration option CONFIG_TUN=m specified.

Note that the version of TUN/TAP in Linux 2.4 differs from the earlier version in that it allows access to all TUN interfaces (tun0, tun1, etc) through a single character device /dev/net/tun, instead of defining multiple devices /dev/tun0, /dev/tun1, etc.

The procedure for completing the TUN/TAP setup for Linux 2.4 is shown below.

  1. Use these commands to create the TUN device:
    (enter the root password when prompted)
    mkdir /dev/net
    mknod /dev/net/tun c 10 200
    chgrp xxxxx /dev/net/tun
    (where xxxxx is the group under which you run Hercules)
    chmod g+rw /dev/net/tun
    chmod o-rw /dev/net/tun
  2. Edit the file /etc/modules.conf (it is called /etc/conf.modules in some distributions) and add the following line:
    alias char-major-10-200 tun
    This causes the TUN/TAP driver to be loaded automatically when the /dev/net/tun device is opened by Hercules.

Installing the TUN/TAP Driver (Linux 2.6, FreeBSD, OS X)

For distributions based on the Linux 2.6 kernel you will probably find that the TUN/TAP driver is already installed and the /dev/net/tun device is already defined. If not, then follow the procedure for Linux 2.4 as descibed above.

For FreeBSD, refer to man tun. You will likely want to enable net.link.tun.devfs_cloning to create /dev/tun, but you will be in uncharted waters.

If you are using a current distribution you will need to alter the permissions on the /dev/net/tun device to allow Hercules to open it. Issue ls -l /dev/net/tun (Linux) ls -l /dev/tun0 (OSX) to determine whether Hercules can open the device.

[root@servus ~]# ls -l /dev/net/tun
crw-rw---- 1 root staff 10, 200 Nov 24 15:01 /dev/net/tun

[enrico@enrico-mbp ref.obj]$ls -l /dev/tap0
crw-rw-rw-  1 root  wheel   21,   0 Nov 22 18:31 /dev/tap0

[enrico@enrico-mbp ref.obj]$ls -l /dev/tun0
crw-rw-rw-  1 root  wheel   19,   0 Nov 22 18:31 /dev/tun0

You have two ways to make the tun device usable. Use the second only with nonstandard and/or old kernels (that is, if the first udev rule does not work for you.)

  1. The preferred way to change properties for the Linux tun device is via a udev rule in /etc/udev/rules.d
    [root@servus rules.d]# cat 99-jph.rules
    KERNEL=="tun",                  GROUP="staff", MODE="0660"
  2. Alternatively, use the following commands to set the necessary permissions:

    su (enter the root password when prompted)
    chgrp xxxxx /dev/net/tun
    (where xxxxx is the group under which you run Hercules)
    chmod 0660 /dev/net/tun

    or make the device file writable by world; this does not consist an integrity exposure, as the world at large cannot do anything with the file descriptor it gets.

    chmod 0666 /dev/net/tun

    Additional notes from Greg Smith:

    1. I find on my Fedora Core 6 system that I have to add the above 2 commands to /etc/rc.local and update /etc/udev/rules.d/50-udev.rules replacing
      KERNEL=="tun", NAME="net/%k"
      KERNEL=="tun", NAME="net/%k", GROUP="xxxxx", MODE="0660"
    2. In the hercules log you should see /dev/net/tun0 opened. I get a couple of error messages about SIOCDIFADDR and SIOCSIFHWADDR ioctl's failing but these can be ignored.

In Linux 2.6 the file /etc/modules.conf no longer exists, instead there is a file called /etc/modprobe.conf. TUN/TAP will usually work, however, without any change to the modprobe configuration.

Configuring a TUN interface

You may be able to configure the tunnel interfaces you will need at system boot; but if you cannot or will not, Hercules supplies a utility (hercifc) to do this.

At Linux boot

openvpn (http://openvpn.net/) is a utility that can open a tunnel device permanently. In addition you are likely to need a udev rule as described above. Openvpn is usually installed as a package (yum install openvpn on Fedora); if not refer to http://openvpn.net/index.php/open-source/documentation/howto.html. Note, however, that openvpn is just one way to create a permanent tunnel device; alternative methods may be possible.

To create permanent a tunnel interface, add this to rc.local (or as appropriate for the distribution you are using):

    openvpn --mktun --dev tun0 --group hercules
    ifconfig tun0 pointopoint
    echo 1 >/proc/sys/net/ipv4/ip_forward

This example assumes that Hercules is in the group hercules; it shows the tunnel device in a separate network, which requires routing as described below. ifconfig can display the permanent interface:

[root@servus ~]# ifconfig tun0
tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:  P-t-P:  Mask:
          RX packets:71282 errors:0 dropped:0 overruns:0 frame:0
          TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:4289708 (4.0 MiB)  TX bytes:2764 (2.6 KiB)

If the interface is in the same subnet as the Hercules host, you need to enable proxy arp. One way to achieve this is to turn it on dynamically:

    echo 1 >/proc/sys/net/ipv4/conf/eth0/proxy_arp

Free BSD

On FreeBSD, use the command ifconfig tun create to create an interface.

Using hercifc

The tun0 network interface in the driving system must be configured as a point-to-point link. The original design of the TUN/TAP driver did not allow the interface to be statically configured like a regular network interface — the tun0 interface does not exist until a program opens the TUN device. For this reason, Hercules provides a special program called hercifc to configure the tun0 network interface. This program is launched automatically by Hercules 3088 CTC device initialization. (Later, the ability to make an interface permanent was added; this is why openvpn and preconfigured interfaces work.)

To allow the hercifc program to issue the necessary configuration commands, you must ensure that hercifc is installed with setuid root file permissions. When Hercules is built with the configuration option --enable-setuid-hercifc=xxxxx, make install will install hercifc in /usr/local/bin with setuid root permissions. The hercifc program will be executable only by group xxxxx. Note: Unrestricted access to the hercifc program could present a potential security exposure, so you will want to ensure that hercifc can be executed only by the group which is authorized to run Hercules.

The following commands have the same effect as the configuration option --enable-setuid-hercifc=xxxxx:

su (enter the root password when prompted)
chgrp xxxxx /usr/local/bin/hercifc
(where xxxxx is the group under which you run Hercules)
chmod 4750 /usr/local/bin/hercifc

Note: you can avoid the need to install hercifc as a setuid root program by creating a permanent tunnel interface at Linux boot time, as described above.

Enabling IP forwarding

You must ensure that your kernel is enabled for IP forwarding. Popular Linux distributions usually have a configuration option to enable IP forwarding or routing:

Defining a route to Hercules TCP/IP

Client systems that connect to TCP/IP applications running in the Hercules machine need to have a routing entry to define the driving system as the gateway into the Hercules system. An example route definition for a Unix client system is shown below:

route add gw

For a Windows client, go to Settings -> Control Panel -> Network -> Configuration -> TCP/IP -> Properties -> Gateway and add the driving system's IP address to the list of gateways. Alternatively, enter a route command such as:

route add mask metric 1

If you want to avoid having to update client systems, another way is to add an appropriate routing entry to your default gateway router.

Defining the link in Hercules

You must define a CTC device pair in the Hercules configuration file: The first device number must be even. The second device must bear the same definition as the first instance and be at device number + 1 (for CTCI, the first definition beyond the device type is ignored; you only need to specify the definition with the second device). Devices should preferably be grouped (furthermore, it makes the configuration file easier to read).

For a preconfigured tunnel:

0E20.2 CTCI tun0
0E20,0E21 CTCI tun0
0E20-0E21 CTCI tun0

Using hercifc:

0E20.2 CTCI
0E20,0E21 CTCI
0E20-0E21 CTCI

Check Device Definition Statement syntax for an explanation of device grouping.

Two IP addresses must be assigned, one for the driving system's end of the link, and one for the Hercules end of the link. For this example I have chosen for the Hercules IP address, and for the driving system's IP address. Since this is a point-to-point link, any addresses may be chosen, provided that the network part of the address (192.168.200 in this example) does not conflict with any existing network addresses used in your IP network.

Configuring the Hercules TCP/IP stack


This is an example of the configuration statements which you need to include in the IPINIT00.L member of PRD1.BASE:

SET MASK     =

The CTC devices should be defined to VSE using the following statements in the $IPLxxx.PROC procedure in IJSYSRS.SYSLIB:


TCP/IP for OS/390 or VM/ESA

This is an example of the configuration statements which you need to include in the TCPIP.PROFILE.TCPIP dataset (OS/390), or in the PROFILE TCPIP file on TCPMAINT 198 (VM):

; Network      First Hop     Link Name Size   Subnet Mask  Subnet Value  =             CTCLINK1  1500   HOST

For OS/390, the CTC devices need to be defined as device type 3088 in the IODF. Use the D U,CTC command to find out which 3088 addresses are defined in your IODF.

For VM, the CTC devices must be attached to the TCPIP virtual machine.

Because TCP/IP uses long running channel programs, the missing interrupt handler should be disabled for the CTC devices. For OS/390, add this statement in PARMLIB member IECIOS00:

MIH TIME=00:00,DEV=(E20-E21)

For VM, add this command to the PROFILE EXEC file of OPERATOR 191:


Linux for S/390

This is an example of the network definitions which you need in a Linux/390 system running under Hercules:

ifconfig ctc0 pointopoint mtu 1500
route add defaultroute gw

Linux/390 will autodetect the CTC devices E20 and E21 at startup and will assign the interface name ctc0.

What to do if TUN/TAP doesn't work

Check the following (thanks to Richard Higson for this checklist):

  1. Enter the command ls -l /dev/tun0 /dev/net/tun.
    For Linux 2.4 and later kernels, the response should be:
    ls: /dev/tun0: No such file or directory
    crw-rw---- 1 root xxxxx 10, 200 Sep 13 07:06 /dev/net/tun

    For Linux 2.2, the response should be:
    crw-rw---- 1 root xxxxx 90, 0 Feb 3 2001 /dev/tun0
    ls: /dev/net/tun: No such file or directory

    (xxxxx should be the group under which you run Hercules).
  2. ls -l /usr/local/bin/hercifc should show
    -rwsr-x--- 1 root xxxxx 17333 Dec 31 20:55 /usr/local/bin/hercifc

    (xxxxx should be the group under which you run Hercules).
  3. When hercules comes up, and before IPLing your favorite OS, verify that you have your underlying network stuff up and ready to roar:
    [root]# ifconfig
    eth0      Link encap:Ethernet  HWaddr 00:12:34:56:78:9A
              inet addr:  Bcast:  Mask:
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
    tun0      Link encap:Point-to-Point Protocol
              inet addr:  P-t-P:  Mask:
    [root]# netstat -in
    Kernel Interface table
    eth0   1500   0     201      0      0      0     196      0      0      0 BMRU
    tun0   1500   0       0      0      0      0       0      0      0      0 MOPRU
    [root]# netstat -rn
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface       U        40 0          0 eth0   U        40 0          0 tun0       U        40 0          0 lo         UG       40 0          0 eth0
  4. `cat /proc/sys/net/ipv4/ip_forward` should show "1". If it doesn't, your L386 won't forward (==route) packets at all.
  5. Is the TUN/TAP driver loaded?
    1. TUN/TAP compiled into the kernel (`make menuconfig`) look for "CONFIG_TUN=m" in /usr/src/linux
    2. `lsmod` after starting hercules should show tun 3456 2 (autoclean)
  6. Look for
    Dec 14 16:47:19 wie kernel: Universal TUN/TAP device driver 1.3 (C)1999-2000
    Maxim Krasnyansky

    in syslog after starting hercules