dhcrelay over GRE tunnels and rebuilding an Ubuntu package

I’ve got a situation such as:

(Diagram made with xfig.) “R&R” stands for “Reynolds and Reynolds,” one of the biggest (if not the biggest) maker of software for automobile dealerships. A dealership gets a R&R server, terminals, printers, that sort of thing. The system seems to handle… everything. There are modules for all manner of things, and it talks back to the OEM’s (i.e., GM, Chrysler, Honda, etc.) systems as well.

R&R have been around for a while, and every dealership my company has done work for has been using R&R for quite a while. As a result, our clients have generally had relatively old R&R equipment, including many serial terminals (or newer PCs with serial connections and emulator software) and serial printers. When R&R goes to upgrade these dealerships—newer server hardware, newer software (I think), and Ethernet connectivity back to the server—some of the older equipment (particularly printers) stick around. To support the older serial equipment when everything else moves to Ethernet and TCP/IP, they plug the serial devices into Digi EtherLite boxes which are basically terminal servers as far as I can tell.

Now, when I say “dealership,” in the case of all our clients I mean several dealerships that share a common owner. Some of the dealerships are physically located adjacent to each other, connected via 100baseFX; others are in different counties. In every case, though, the dealerships have shared a central R&R server. It seems that R&R did the initial work to network the different locations together. They seem to get point-to-point lines run between the different locations, like DDS 56k lines or T-1s. Then they stick a Cisco router on each end and somehow configure the Ciscos to essentially create a bridged network: all locations are in the same broadcast domain, no routing involved. They’ve all ended up using these point-to-point lines to share a single Internet connection between all locations. They allocate IPs for all locations out of a single block of IPs, for any equipment that needs IPs (mainly PCs, to get on the Internet, I think). Yes, this creates management hassles, particularly two locations allocating the same IP.

So we come in, and we end up giving each location its own Internet connection, putting in Linux boxes in place of the Ciscos, and using a VPN to network the sites together. We route them because, well, that just seems to be the sane way to do it. R&R comes in and upgrades everything, puts in a new server, converts almost everything over to Ethernet, and puts the remaining serial devices on one or more EtherLites. Most (all?) dumb terminals get converted to PCs and now speak TCP/IP to the server over Ethernet.

Now, though, you’ve got some EtherLites at remote offices, to talk to the old serial equipment. And, so R&R tells us, the EtherLites have to use bootp to configure themselves. The R&R server has a bootp server running on it, and I’m guessing a TFTP server as well for the EtherLite to pull down its configuration files. I’ve had at least two R&R techs come in, put an EtherLite at a remote office, and then call me saying, “I can’t get my EtherLites to boot from the server, what did you break?” (“Well, uh, they’re routed now, and you didn’t tell me anything about this requirement, so…”) Cisco routers have something like an ip helper-address command that makes them forward bootp/DHCP on to a server you specify. On Linux, you’ve got several choices; the one that I ran into first on my original quest to fix this problem was dhcrelay, part of ISC dhcpd. dhcrelay is found in (at least) RH and Ubuntu.

I’d like to say this setup isn’t difficult, but honestly it kind of is. First of all, our Linux firewalls are generally DHCP servers as well, using ISC dhcpd. You need to tell them not to serve to the EtherLite’s addresses, and make sure they’re not “authoritative,” meaning they can only respond affirmatively to a request and never negatively (since another bootp/DHCP server might respond, like the R&R server). When I did deny bootp in the dhcpd configuration it was still responding to the EtherLites. So here’s a snippet of the configuration I ended up with:

class "reyrey-devices"
{
    match pick-first-value (option dhcp-client-identifier, hardware);
}

subclass "reyrey-devices" 1:0:80:ba:12:34:56;
subclass "reyrey-devices" 1:0:80:ba:12:34:57;
subclass "reyrey-devices" 1:0:80:ba:12:34:58;
# ... and so on for each EtherLite's MAC.  That first "1" octet
# indicates it's an Ethernet MAC, I believe.

subnet 10.123.45.0 netmask 255.255.255.0
{
    pool
    {
        deny members of "reyrey-devices";
        # And all my other DHCP configuration, like range,
        # DHCP options, and fixed addresses go under here too.
    }
}

That’s huge and ugly and complex, but it does work. Perhaps using some other DHCP server would make this easier.

Before I go on to how to configure dhcrelay, which is actually rather easy compared to the time it took me to figure out the about dhcpd configuration, let me mention how the VPN works. It’s an IPsec VPN, but using GRE over IPsec for greater flexibility and security.

Configuring dhcrelay is basically done through the command line. On Ubuntu, you modify the command line options for dhcrelay in something like /etc/defaults/dhcp3-relay, and it’s something like /etc/sysconfig/dhcrelay in RH. You have to tell dhcrelay a device to listen on for DHCP requests, and you have to tell it the IP of a server to forward the request to. When a device (i.e., an EtherLite) DHCPs, that is a broadcast. When dhcrelay forwards, however, it’s a unicast to the specified DHCP server. Thus, I didn’t think it would need to be told which interfaces it should listen on for the unicast reply from the DHCP server. Wrong: if you don’t tell it both the LAN interface, and the interface it reaches the DHCP server on (the GRE tunnel interface, in my case), it doesn’t work.

Now, if that wasn’t enough, on Linux every binary dhcrelay I’ve seen has used LPF (Linux packet filter). It took me a while, but I eventually decided that LPF doesn’t work right on a GRE tunnel, and I don’t exactly know why. You even get a warning about an unknown interface type from dhcrelay, though it looks like it runs anyway; runs, but doesn’t work. The solution? Recompile dhcrelay with USE_SOCKETS.

---

Which is my cue to document, by way of example, how to rebuild a package under Ubuntu. I’m going to rebuild the dhcp3 package in Ubuntu, which provide ISC’s dhcpd, dhclient, and dhcrelay. I got the basic idea for how to do this from an article entitled “Rebuilding Debian packages.”

First, to make sure you’ve got everything you need to build packages, I suggest installing: dpkg-dev gcc libc-dev dpatch fakeroot devscripts ccache debhelper. Don’t forget to put /usr/lib/ccache in your path before /usr/bin so you get ccached versions of all the binaries.

Next, run apt-get build-dep dhcp3 which will install everything you need to rebuild the dhcp3 package. Then:

[darkness@gateway-honda dpkg-build]$ apt-get source dhcp3
Reading package lists... Done
Building dependency tree... Done
Skipping already downloaded file 'dhcp3_3.0.3-6ubuntu7.dsc'
Skipping already downloaded file 'dhcp3_3.0.3.orig.tar.gz'
Skipping already downloaded file 'dhcp3_3.0.3-6ubuntu7.diff.gz'
Need to get 0B of source archives.
dpkg-source: extracting dhcp3 in dhcp3-3.0.3
dpkg-source: unpacking dhcp3_3.0.3.orig.tar.gz
dpkg-source: applying ./dhcp3_3.0.3-6ubuntu7.diff.gz
[darkness@gateway-honda dpkg-build]$ ls -l
total 936
drwxr-x--- 15 darkness users   4096 2006-12-23 04:09 dhcp3-3.0.3
-rw-r-----  1 darkness users  66377 2006-05-05 10:08 dhcp3_3.0.3-6ubuntu7.diff.gz
-rw-r-----  1 darkness users    777 2006-05-05 10:08 dhcp3_3.0.3-6ubuntu7.dsc
-rw-r-----  1 darkness users 870240 2005-11-14 07:40 dhcp3_3.0.3.orig.tar.gz

Sorry, I did this apt-get source once before, which is why I had them downloaded already. Now we’ll make a patch against the DHCP sources that defines USE_SOCKETS. dhcp3 happens to use software called dpatch to manage applying a series of patches, so we’ll use the utilities provided by dpatch to make our new patch.

[darkness@gateway-honda dpkg-build]$ cd dhcp3-3.0.3
[darkness@gateway-honda dhcp3-3.0.3]$ cat debian/patches/00list
Makefile
common
ddns-update-style-default
dhclient-script-exit-status
dhcpd-chdir
documentation
remove-excessive-junk
site.conf
add-libdst.a-dhcp3-dev
ignore-invalid-interfaces
droppriv
deroot-client
deroot-server
revert-next-server
dhcpd.conf-subnet-examples
[darkness@gateway-honda dhcp3-3.0.3]$ dpatch-edit-patch use-bsd-sockets \
                      dhcpd.conf-subnet-examples
dpatch-edit-patch: * /home/darkness/dpkg-build/dhcp3-3.0.3/debian/patches/use-bsd-sockets.dpatch
does not exist, it will be created as a new dpatch.
#
# ... then it does a whole bunch of stuff, after which:
#
dpatch-edit-patch:

Now launching an interactive shell in your work directory. Edit your files.
When you are done, exit the shell. When you exit the shell, your patch will be
automatically updated based on the changes in your work directory.

If you wish to abort the process, exit the shell such that it returns an exit
code of "230". This is typically done by exiting the shell with the command
'exit 230'.
sh-3.1$ pwd
/tmp/dpep-work.JzHxjx/dhcp3-3.0.3

(I’ll be wrapping text where I feel appropriate.) dpatch-edit-patch has, I gather, made a copy of the dhcp3 source, applied all patches up to and including the dhcpd.conf-subnet-examples patch I specified last on the command line, started a new shell in this temporary copy of the source, and now wants me to modify this copy of the source.

sh-3.1$ ed includes/site.h
5837
/USE_SOCKETS/d
i
#define USE_SOCKETS
.
wq
5831
sh-3.1$ exit
exit
dpatch-edit-patch: * Creating new patch /home/darkness/dpkg-build/dhcp3-3.0.3/debian/patches/use-bsd-sockets.dpatch
dpatch-edit-patch: Warning: debian/patches/00template not exist, using hardcoded default.
dpatch-edit-patch: /home/darkness/dpkg-build/dhcp3-3.0.3/debian/patches/use-bsd-sockets.dpatch created.
[darkness@gateway-honda dhcp3-3.0.3]$ cat debian/patches/use-bsd-sockets.dpatch
#! /bin/sh /usr/share/dpatch/dpatch-run
## use-bsd-sockets.dpatch by  <darkness@gateway-honda>
##
## All lines beginning with `## DP:' are a description of the patch.
## DP: No description.

@DPATCH@
diff -urNad dhcp3-3.0.3~/includes/site.h dhcp3-3.0.3/includes/site.h
--- dhcp3-3.0.3~/includes/site.h        2002-03-12 13:33:39.000000000 -0500
+++ dhcp3-3.0.3/includes/site.h 2006-12-23 04:19:05.000000000 -0500
@@ -135,7 +135,7 @@
the aforementioned problems do not matter to you, or if no other
API is supported for your system, you may want to go with it. */

-/* #define USE_SOCKETS */
+#define USE_SOCKETS

 /* Define this to use the Sun Streams NIT API.

(I’m just showing off using ed.) So what I’ve done is uncommented #define USE_SOCKETS, then exited the subshell dpatch-edit-patch put me in. When I did this, dpatch did a diff between the original source tree I started in and the temporary source tree where I modified some things. It saved this diff as the patch name I specified on the dpatch-edit-patch command line.

[darkness@gateway-honda dhcp3-3.0.3]$ dpatch list-all | grep bsd
[darkness@gateway-honda dhcp3-3.0.3]$ echo 'use-bsd-sockets' >>debian/patches/00list
[darkness@gateway-honda dhcp3-3.0.3]$ dpatch list-all | grep bsd
use-bsd-sockets

dpatch list-all will show you all patches dpatch will apply when it builds this package. You’ll notice our patch wasn’t going to be applied until we appended it to debian/patches/00list.

Next we want to change the version number, so it’s clear we modified this package locally. Apparently the package version is taken from the latest (top-most) entry in debian/changelog. dch is in the devscripts package that we installed above. Run dch -i and dch will insert a new skeleton entry in changelog with an incremented version number, then pop you into $EDITOR to edit the entry. Here’s my (slightly obfuscated) entry:

(3.0.3-6ubuntu7+codefu.1) dapper; urgency=low
   * Forced use of BSD sockets, to make dhcrelay work right over a GRE
     tunnel.

 -- My Name <my@email.address>  Sat, 23 Dec 2006 04:27:54 -0500

Now, building the package is easy: debuild -e PATH -uc -us. The first option tells debuild not to “sanitize” the PATH environment variable; doing so removes /usr/lib/ccache from the PATH, and I like ccache (when you screw up on some long running package build and you need to restart the whole thing over, you’ll like ccache too). Just make sure there’s nothing funny in your PATH that might affect whatever package you’re building, lest your build not be repeatable on a different machine. The remaining options tell debuild not to try to sign anything.

When this finishes, hopefully successfully, you’ll end up with fixed dhcp3* packages in the parent directory. Install them with dpkg -i, and then configure APT not to try to upgrade your new packages. Now start up dhcrelay, telling it to listen on both the GRE tunnel as well as on the LAN, and suddenly the EtherLites can boot. Hooray.

One note: if you wish to “downgrade” to the Ubuntu packages for DHCP for some reason (maybe you’ve gotten rid of all your EtherLites and other weird equipment; good for you), you can do it with something like apt-get install dhcp3-{common,server,client,relay}/dapper. The /dapper at the end apparently tells APT what distribution to look in, and it’ll gladly downgrade you (and explicitly tell you that it is downgrading) at that point. Note that, if the package you want to downgrade has had an update released for it, you might need to specify /dapper-updates instead; or just apt-get upgrade after downgrading to the original release version, which should grab any available updates for you.