I’ve got a situation such as:
(Diagram made with xfig.) “R&R” stands for “Reynolds and
Reynolds,” one of the biggest (if not the
biggest) maker of software for automobile dealerships. A dealership
gets a R&R server, terminals, printers, that sort of thing. The
system seems to handle… everything. There are modules for all
manner of things, and it talks back to the OEM’s (i.e., GM, Chrysler,
Honda, etc.) systems as well.
R&R have been around for a while, and every dealership my company has
done work for has been using R&R for quite a while. As a result, our
clients have generally had relatively old R&R equipment, including
many serial terminals (or newer PCs with serial connections and
emulator software) and serial printers. When R&R goes to upgrade
these dealerships—newer server hardware, newer software (I think),
and Ethernet connectivity back to the server—some of the older
equipment (particularly printers) stick around. To support the older
serial equipment when everything else moves to Ethernet and TCP/IP,
they plug the serial devices into Digi EtherLite
boxes which
are basically terminal servers as far as I can tell.
Now, when I say “dealership,” in the case of all our clients I mean
several dealerships that share a common owner. Some of the
dealerships are physically located adjacent to each other, connected
via 100baseFX; others are in different counties. In every case,
though, the dealerships have shared a central R&R server. It seems
that R&R did the initial work to network the different locations
together. They seem to get point-to-point lines run between the
different locations, like DDS 56k lines or T-1s. Then they stick a
Cisco router on each end and somehow configure the Ciscos to
essentially create a bridged network: all locations are in the same
broadcast domain, no routing involved. They’ve all ended up using
these point-to-point lines to share a single Internet connection
between all locations. They allocate IPs for all locations out of a
single block of IPs, for any equipment that needs IPs (mainly PCs, to
get on the Internet, I think). Yes, this creates management hassles,
particularly two locations allocating the same IP.
So we come in, and we end up giving each location its own Internet
connection, putting in Linux boxes in place of the Ciscos, and using a
VPN to network the sites together. We route them because, well, that
just seems to be the sane way to do it. R&R comes in and upgrades
everything, puts in a new server, converts almost everything over to
Ethernet, and puts the remaining serial devices on one or more
EtherLites. Most (all?) dumb terminals get converted to PCs and now
speak TCP/IP to the server over Ethernet.
Now, though, you’ve got some EtherLites at remote offices, to talk to
the old serial equipment. And, so R&R tells us, the EtherLites have
to use bootp to configure themselves. The R&R server has a bootp
server running on it, and I’m guessing a TFTP server as well for the
EtherLite to pull down its configuration files. I’ve had at least two
R&R techs come in, put an EtherLite at a remote office, and then call
me saying, “I can’t get my EtherLites to boot from the server, what
did you break?” (“Well, uh, they’re routed now, and you didn’t tell
me anything about this requirement, so…”) Cisco routers have
something like an ip helper-address command that makes them forward
bootp/DHCP on to a server you specify. On Linux, you’ve got several
choices; the one that I ran into first on my original quest to fix
this problem was dhcrelay, part of ISC dhcpd. dhcrelay is found in
(at least) RH and Ubuntu.
I’d like to say this setup isn’t difficult, but honestly it kind of
is. First of all, our Linux firewalls are generally DHCP servers as
well, using ISC dhcpd. You need to tell them not to serve to the
EtherLite’s addresses, and make sure they’re not “authoritative,”
meaning they can only respond affirmatively to a request and never
negatively (since another bootp/DHCP server might respond, like the
R&R server). When I did deny bootp in the dhcpd configuration it
was still responding to the EtherLites. So here’s a snippet of the
configuration I ended up with:
class "reyrey-devices"
{
match pick-first-value (option dhcp-client-identifier, hardware);
}
subclass "reyrey-devices" 1:0:80:ba:12:34:56;
subclass "reyrey-devices" 1:0:80:ba:12:34:57;
subclass "reyrey-devices" 1:0:80:ba:12:34:58;
# ... and so on for each EtherLite's MAC. That first "1" octet
# indicates it's an Ethernet MAC, I believe.
subnet 10.123.45.0 netmask 255.255.255.0
{
pool
{
deny members of "reyrey-devices";
# And all my other DHCP configuration, like range,
# DHCP options, and fixed addresses go under here too.
}
}
That’s huge and ugly and complex, but it does work. Perhaps using
some other DHCP server would make this easier.
Before I go on to how to configure dhcrelay, which is actually rather
easy compared to the time it took me to figure out the about dhcpd
configuration, let me mention how the VPN works. It’s an IPsec VPN,
but using GRE over IPsec for greater flexibility and security.
Configuring dhcrelay is basically done through the command line. On
Ubuntu, you modify the command line options for dhcrelay in something
like /etc/defaults/dhcp3-relay, and it’s something like
/etc/sysconfig/dhcrelay in RH. You have to tell dhcrelay a device
to listen on for DHCP requests, and you have to tell it the IP of a
server to forward the request to. When a device (i.e., an EtherLite)
DHCPs, that is a broadcast. When dhcrelay forwards, however, it’s a
unicast to the specified DHCP server. Thus, I didn’t think it would
need to be told which interfaces it should listen on for the unicast
reply from the DHCP server. Wrong: if you don’t tell it both the
LAN interface, and the interface it reaches the DHCP server on (the
GRE tunnel interface, in my case), it doesn’t work.
Now, if that wasn’t enough, on Linux every binary dhcrelay I’ve seen
has used LPF (Linux packet filter). It took me a while, but I
eventually decided that LPF doesn’t work right on a GRE tunnel, and I
don’t exactly know why. You even get a warning about an unknown
interface type from dhcrelay, though it looks like it runs anyway;
runs, but doesn’t work. The solution? Recompile dhcrelay with
USE_SOCKETS.
Which is my cue to document, by way of example, how to rebuild a
package under Ubuntu. I’m going to rebuild the dhcp3 package in
Ubuntu, which provide ISC’s dhcpd, dhclient, and dhcrelay. I got the
basic idea for how to do this from an article entitled “Rebuilding
Debian packages.”
First, to make sure you’ve got everything you need to build packages,
I suggest installing: dpkg-dev gcc libc-dev dpatch fakeroot
devscripts ccache debhelper. Don’t forget to put /usr/lib/ccache
in your path before /usr/bin so you get ccached versions of all the
binaries.
Next, run apt-get build-dep dhcp3 which will install everything you
need to rebuild the dhcp3 package. Then:
[darkness@gateway-honda dpkg-build]$ apt-get source dhcp3
Reading package lists... Done
Building dependency tree... Done
Skipping already downloaded file 'dhcp3_3.0.3-6ubuntu7.dsc'
Skipping already downloaded file 'dhcp3_3.0.3.orig.tar.gz'
Skipping already downloaded file 'dhcp3_3.0.3-6ubuntu7.diff.gz'
Need to get 0B of source archives.
dpkg-source: extracting dhcp3 in dhcp3-3.0.3
dpkg-source: unpacking dhcp3_3.0.3.orig.tar.gz
dpkg-source: applying ./dhcp3_3.0.3-6ubuntu7.diff.gz
[darkness@gateway-honda dpkg-build]$ ls -l
total 936
drwxr-x--- 15 darkness users 4096 2006-12-23 04:09 dhcp3-3.0.3
-rw-r----- 1 darkness users 66377 2006-05-05 10:08 dhcp3_3.0.3-6ubuntu7.diff.gz
-rw-r----- 1 darkness users 777 2006-05-05 10:08 dhcp3_3.0.3-6ubuntu7.dsc
-rw-r----- 1 darkness users 870240 2005-11-14 07:40 dhcp3_3.0.3.orig.tar.gz
Sorry, I did this apt-get source once before, which is why I had
them downloaded already. Now we’ll make a patch against the DHCP
sources that defines USE_SOCKETS. dhcp3 happens to use software
called dpatch to manage applying a series of patches, so we’ll use the
utilities provided by dpatch to make our new patch.
[darkness@gateway-honda dpkg-build]$ cd dhcp3-3.0.3
[darkness@gateway-honda dhcp3-3.0.3]$ cat debian/patches/00list
Makefile
common
ddns-update-style-default
dhclient-script-exit-status
dhcpd-chdir
documentation
remove-excessive-junk
site.conf
add-libdst.a-dhcp3-dev
ignore-invalid-interfaces
droppriv
deroot-client
deroot-server
revert-next-server
dhcpd.conf-subnet-examples
[darkness@gateway-honda dhcp3-3.0.3]$ dpatch-edit-patch use-bsd-sockets
dhcpd.conf-subnet-examples
dpatch-edit-patch: * /home/darkness/dpkg-build/dhcp3-3.0.3/debian/patches/use-bsd-sockets.dpatch
does not exist, it will be created as a new dpatch.
#
# ... then it does a whole bunch of stuff, after which:
#
dpatch-edit-patch:
Now launching an interactive shell in your work directory. Edit your files.
When you are done, exit the shell. When you exit the shell, your patch will be
automatically updated based on the changes in your work directory.
If you wish to abort the process, exit the shell such that it returns an exit
code of "230". This is typically done by exiting the shell with the command
'exit 230'.
sh-3.1$ pwd
/tmp/dpep-work.JzHxjx/dhcp3-3.0.3
(I’ll be wrapping text where I feel appropriate.) dpatch-edit-patch
has, I gather, made a copy of the dhcp3 source, applied all patches up
to and including the dhcpd.conf-subnet-examples patch I specified
last on the command line, started a new shell in this temporary copy
of the source, and now wants me to modify this copy of the source.
sh-3.1$ ed includes/site.h
5837
/USE_SOCKETS/d
i
#define USE_SOCKETS
.
wq
5831
sh-3.1$ exit
exit
dpatch-edit-patch: * Creating new patch /home/darkness/dpkg-build/dhcp3-3.0.3/debian/patches/use-bsd-sockets.dpatch
dpatch-edit-patch: Warning: debian/patches/00template not exist, using hardcoded default.
dpatch-edit-patch: /home/darkness/dpkg-build/dhcp3-3.0.3/debian/patches/use-bsd-sockets.dpatch created.
[darkness@gateway-honda dhcp3-3.0.3]$ cat debian/patches/use-bsd-sockets.dpatch
#! /bin/sh /usr/share/dpatch/dpatch-run
## use-bsd-sockets.dpatch by <darkness@gateway-honda>
##
## All lines beginning with `## DP:' are a description of the patch.
## DP: No description.
@DPATCH@
diff -urNad dhcp3-3.0.3~/includes/site.h dhcp3-3.0.3/includes/site.h
--- dhcp3-3.0.3~/includes/site.h 2002-03-12 13:33:39.000000000 -0500
+++ dhcp3-3.0.3/includes/site.h 2006-12-23 04:19:05.000000000 -0500
@@ -135,7 +135,7 @@
the aforementioned problems do not matter to you, or if no other
API is supported for your system, you may want to go with it. */
-/* #define USE_SOCKETS */
+#define USE_SOCKETS
/* Define this to use the Sun Streams NIT API.
(I’m just showing off using ed.) So what I’ve done is uncommented
#define USE_SOCKETS, then exited the subshell dpatch-edit-patch
put me in. When I did this, dpatch did a diff between the original
source tree I started in and the temporary source tree where I
modified some things. It saved this diff as the patch name I
specified on the dpatch-edit-patch command line.
[darkness@gateway-honda dhcp3-3.0.3]$ dpatch list-all | grep bsd
[darkness@gateway-honda dhcp3-3.0.3]$ echo 'use-bsd-sockets' >>debian/patches/00list
[darkness@gateway-honda dhcp3-3.0.3]$ dpatch list-all | grep bsd
use-bsd-sockets
dpatch list-all will show you all patches dpatch will apply when it
builds this package. You’ll notice our patch wasn’t going to be
applied until we appended it to debian/patches/00list.
Next we want to change the version number, so it’s clear we modified
this package locally. Apparently the package version is taken from
the latest (top-most) entry in debian/changelog. dch is in the
devscripts package that we installed above. Run dch -i and dch
will insert a new skeleton entry in changelog with an incremented
version number, then pop you into $EDITOR to edit the entry. Here’s
my (slightly obfuscated) entry:
(3.0.3-6ubuntu7+codefu.1) dapper; urgency=low
* Forced use of BSD sockets, to make dhcrelay work right over a GRE
tunnel.
-- My Name <my@email.address> Sat, 23 Dec 2006 04:27:54 -0500
Now, building the package is easy: debuild -e PATH -uc -us. The
first option tells debuild not to “sanitize” the PATH environment
variable; doing so removes /usr/lib/ccache from the PATH, and I
like ccache (when you screw up on some long running package build and
you need to restart the whole thing over, you’ll like ccache too).
Just make sure there’s nothing funny in your PATH that might affect
whatever package you’re building, lest your build not be repeatable on
a different machine. The remaining options tell debuild not to try
to sign anything.
When this finishes, hopefully successfully, you’ll end up with fixed
dhcp3* packages in the parent directory. Install them with dpkg
-i, and then configure APT not to try to upgrade your new packages.
Now start up dhcrelay, telling it to listen on both the GRE tunnel
as well as on the LAN, and suddenly the EtherLites can boot. Hooray.
One note: if you wish to “downgrade” to the Ubuntu packages for DHCP
for some reason (maybe you’ve gotten rid of all your EtherLites and
other weird equipment; good for you), you can do it with something
like apt-get install dhcp3-{common,server,client,relay}/dapper. The
/dapper at the end apparently tells APT what distribution to look
in, and it’ll gladly downgrade you (and explicitly tell you that it is
downgrading) at that point. Note that, if the package you want to
downgrade has had an update released for it, you might need to specify
/dapper-updates instead; or just apt-get upgrade after downgrading
to the original release version, which should grab any available
updates for you.