Thursday, 24 November 2005
Sunday, 13 November 2005
From waproamd to NetworkManager
Last night my X server suddenly started doing this weird cycle where it would (apparently) allocate over 1GB of memory, then free it; and it repeated this cycle until I killed the X server. After I killed the X server, APM suspend/resume stopped working. Fun! So I rebooted, and into the newest Fedora kernel, kernel-2.6.14-1.1637_FC4. I also upgraded ieee80211 and ipw2100 from ATrpms (ieee80211-kmdl-2.6.14-1.1637_FC4-1.1.6-6.rhfc4.at and ipw2100-kmdl-2.6.14-1.1637_FC4-1.1.3-36.rhfc4.at, respectively).
Once I rebooted, waproamd no longer worked:
Nov 13 17:17:28 darkbook waproamd(wlan0)[4107]: Scanning...
Nov 13 17:17:28 darkbook waproamd(wlan0)[4107]: Corrupt scan result (2)
That’s all I got. waproamd’s page does note that you should stop using it in favor of wpa_supplicant. Additionally, it hasn’t been updated in quite some time. Rebuilding it didn’t rectify this problem, so alas, I had to scrap it.
I think my main problem with wpa_supplicant (other than it’s kind of
ugly configuration file with lots and lots of options, compared to
waproamd scripts that just call ifup) was that it had no way to
detect and properly configure the access points at school. The
school’s APs have “hidden” ESSIDs. Literally, iwlist scan shows
them as <hidden>. I have been told that <hidden> is something
specific to the ipw2100 driver, but I’m not so sure of that these
days. At any rate, I gather there’s some ap_mode=2 setting in
wpa_supplicant these days that will let you work with hidden access
points. I’m not sure if it uses the standard RH/FC ifup/ifdown
commands; I kind of doubt it. Further, I’m not sure it has an option
to kick off scripts for particular networks. For example, at home I
need something to start OpenVPN. (With waproamd, that just goes in
the appropriate script.)
Somehow I found
NetworkManager. It’s
in FC4 already: install NetworkManager and NetworkManager-gnome.
(Without that last one, you don’t get the applet and thus have zero
control over NetworkManager, or so it seems.) You probably want to
start the NetworkManager and dhcdbd services now (via service)
and at boot (via chkconfig). You may also want to mkdir -p
/etc/NetworkManager/dispatcher.d and enable/start the
NetworkManagerDispatcher service. When that last service is
started, scripts in /etc/NetworkManager/dispatcher.d get called with
the device name as their first argument and up or down as the
second argument. So I stuck some scripts in there to start OpenVPN,
restart ntpd, that sort of thing.
(Also, I got a lot of this information from http://www.ces.clemson.edu/linux/nm.shtml.)
One problem, though. My APM scripts rmmod ipw2100 at suspend. I’m
not sure if it’s still necessary to do this; it was at one time, and
as recently as my initial FC4 upgrade on this laptop. Generally it
seems like any time I take out some of my old APM tricks to get
suspend/resume working, I end up breaking suspend/resume. Which is to
say that the same problems I’ve always had still plague me. At any
rate, I wasn’t really interested in trying to run without that rmmod
at suspend time. The problem seems to be in
HAL: it doesn’t see the
network interface after you remove and re-insert the module. With the
version that came in FC4, I couldn’t get the device “node” (I don’t
know if that’s the right term, but I’ll roll with it for now) to
appear even after I did a service haldaemon restart. I heard that
newer HAL versions had some power management features. So, I upgraded
to hal-0.5.4.cvs20051111-1 from Fedora development (which is
Rawhide…? Or not?). I couldn’t find that it does anything
interesting with power management. Further, I dug into
D-BUS and Python a bit. I
came up with stuff like this:
import dbus
slashHal = "/org/freedesktop/Hal"
dottedHal = "org.freedesktop.Hal"
bus = dbus.Bus.get_system()
spm_obj = bus.get_object(dottedHal,
slashHal + "/Device/SystemPowerManagement")
spm = dbus.Interface(spm_obj, dottedHal + ".Device.SystemPowerManagement")
def getDev(udi):
obj = bus.get_object(dottedHal,
"%s/devices/%s" % (slashHal, udi))
return dbus.Interface(obj, dottedHal + ".Device")
card = getDev("pci_8086_1043")
wlan = getDev("net_00_0d_43_8f_68_f5")
computer = getDev("computer")
bridge = getDev("pci_8086_2448")
I tried stuff like spm.Hibernate() which bitched that the security
policy wouldn’t let me do that. (I think it wants me to be “at
console,” based on my uninformed interpretation of
/etc/dbus-1/system.d/hal.conf. I don’t know how it determines if
you’re “at console.”) card is the node for the PCI device that is
the wireless card. It’s there regardless of whether ipw2100 is loaded
or not. wlan would be the node for the actual network device, if it
existed, which it doesn’t. computer is the whole computer (I guess;
it’s the root of the tree) and bridge is the PCI bridge that both of
my network cards are on. You can call Rescan and Reprobe on any
of them with no positive effect (i.e., computer.Reprobe()).
Finally I figured out that, with the new version of HAL, service
haldaemon restart was putting the proper network device node in the
tree. So now that’s part of resume. Except when you do service
haldaemon restart, NetworkManager dies, so you have to start it back
up. Here’s my /etc/sysconfig/apm-scripts/apmcontinue as of now:
#!/bin/bash
case "$1" in
suspend|standby)
#dbus-send --system --dest=org.freedesktop.NetworkManager \
# --type=method_call /org/freedesktop/NetworkManager \
# org.freedesktop.NetworkManager.sleep
service NetworkManager stop
ifconfig wlan0 down
rmmod ipw2100
service vmware stop
;;
resume)
#dbus-send --system --dest=org.freedesktop.NetworkManager \
# --type=method_call /org/freedesktop/NetworkManager \
# org.freedesktop.NetworkManager.wake
service haldaemon restart
service NetworkManager start
chvt 1
chvt 7
;;
esac
You’ll note that I found the sleep/wake methods on NetworkManager, but if you’re going to stop and restart it, not a lot of point in bothering with them as near as I can tell.
I’ll also mention that it seems you can add a new device into HAL. Why should I have to do that, though?
Saturday, 12 November 2005
Using R
Situation: I’ve got a Cricket getting distance readings from three other Cricket beacons. The data file looks like:
01:9E:14:C4:0A:00:00:A1 219
01:7D:CA:60:0A:00:00:8A 150
01:9F:FF:C3:0A:00:00:BA 205
01:9E:14:C4:0A:00:00:A1 219
01:7D:CA:60:0A:00:00:8A 150
01:9F:FF:C3:0A:00:00:BA 206
01:9E:14:C4:0A:00:00:A1 219
01:7D:CA:60:0A:00:00:8A 150
01:9F:FF:C3:0A:00:00:BA 206
01:9E:14:C4:0A:00:00:A1 219
.
.
.
The first column is the beacon ID, the second is the computed distance to that beacon. Right now our beacons send out a signal about once a second. (In one case it took me four minutes to retrieve >200 readings for each of the three beacons.) The goals are to know when we’ve sampled enough data to make a reasonable guess at what the real value for distance is, and of course to actually make that guess.
Ideally, we’d like to have our distance accurate to 1cm, or maybe 2cm; of course, it’s not clear how useful math or statistics or anything else short of prayer will be if the Cricket sends back totally bogus results at any point. We’ve had spots where its reported distance to beacon was much too high; we attributed this to ultrasound reflection. Still, if we trust that the data coming in is “generally right,” we just need some math/statistics to “scrub” the data. We need to know which data points to keep, and which to throw out. (Alternatively, I might just be using this as a diversion to learn more about statistics and statistics software.)
I’m using R 2.2.0 from Fedora Extras, with Emacs Speaks Statistics. I’m also using my old statistics text book. I took one statistics course at college. I consider myself to have done well in that course, and (surprisingly) I feel like I grok the textbook. All that said… it was one course. The guy barely spoke English. A little bit of knowledge can be very dangerous. So if you think the stuff I’m doing here is pointless, flawed, or you simply have a better suggestion for doing this, please do let me know.
Anyway. Given the above file, I read the data in like this:
> position1 < - read.table("distances.1", header=FALSE)
> position1
V1 V2
1 01:9E:14:C4:0A:00:00:A1 219
2 01:7D:CA:60:0A:00:00:8A 150
3 01:9F:FF:C3:0A:00:00:BA 205
4 01:9E:14:C4:0A:00:00:A1 219
5 01:7D:CA:60:0A:00:00:8A 150
6 01:9F:FF:C3:0A:00:00:BA 206
7 01:9E:14:C4:0A:00:00:A1 219
8 01:7D:CA:60:0A:00:00:8A 150
9 01:9F:FF:C3:0A:00:00:BA 206
10 01:9E:14:C4:0A:00:00:A1 219
.
.
.
No headers in my input file. < - is an assignment operator; you can
reverse the arrow, so long as it’s pointing to what you’re
assigning to. You can rename V1 and V2 to something better (like
beaconid and distance) with edit(position1).
position1 is now a data.frame (try class(position1)). The
beaconid column is non-numeric, so data.table made it into a
“factor.”
I don’t know enough to explain what a factor is, but you can use
it for grouping. Check this out:
> attach(position1)
> tapply(distance, beaconid, summary)
$"01:7D:CA:60:0A:00:00:8A"
Min. 1st Qu. Median Mean 3rd Qu. Max.
149.0 150.0 150.0 150.1 150.0 151.0
$"01:9E:14:C4:0A:00:00:A1"
Min. 1st Qu. Median Mean 3rd Qu. Max.
51.0 219.0 219.0 217.6 219.0 220.0
$"01:9F:FF:C3:0A:00:00:BA"
Min. 1st Qu. Median Mean 3rd Qu. Max.
85.0 205.0 206.0 203.5 206.0 207.0
attach basically makes the columns of the given object accessible
without being qualified by the object’s name. That is, without
attach(position1) first, I’d have to use position1$beaconid
instead of just beaconid. tapply takes the vector as its first
argument (the columns of a data.frame are vectors–or, at least,
they are in this case) and divides it up based on the factor given as
its second argument, then calls the given function (summary here)
with each of the resulting vectors. I think of it as analogous to
GROUP BY in SQL. By the way, if you want to undo attach, just
detach().
It looks like the first beacon is probably 150cm away. I noticed that
the mean and the median had some disagreement for the second and third
beacons. I wasn’t crazy about it. But look at the minimums: way too
low for the beacon IDs ending in A1 and BA. So trim the top and
bottom 10% of the data set:
> tapply(distance, beaconid, mean, trim=0.1)
01:7D:CA:60:0A:00:00:8A 01:9E:14:C4:0A:00:00:A1 01:9F:FF:C3:0A:00:00:BA
150.0000 219.0000 205.7964
Much better. The trim=0.1 is an argument that’s passed to mean.
Variance and standard deviation:
> tapply(distance, beaconid, var)
01:7D:CA:60:0A:00:00:8A 01:9E:14:C4:0A:00:00:A1 01:9F:FF:C3:0A:00:00:BA
0.08623402 208.02434337 217.29764082
> tapply(distance, beaconid, sd)
01:7D:CA:60:0A:00:00:8A 01:9E:14:C4:0A:00:00:A1 01:9F:FF:C3:0A:00:00:BA
0.2936563 14.4230490 14.7410190
What would those look like if you trim the data set, as we did with
mean, above?
Diversion: subset is kind of interesting. I was interested in
pulling out just one beacon’s readings into a separate vector:
> a1 < - subset(position1, beaconid=="01:9E:14:C4:0A:00:00:A1", select=distance)
> a1
distance
1 219
4 219
7 219
10 219
13 219
16 220
19 219
21 85
24 219
27 219
.
.
.
> class(a1)
[1] “data.frame”
Not a vector, but instead a data.frame; that’s hardly a problem
right now, though. By the way, you can do things like
position1[[1]] and position[1]. They both output the beacon ID
column, but in different formats:
> class(position1[1])
[1] "data.frame"
> class(position1[[1]])
[1] "factor"
I still want to trim data. I don’t see anything in R that returns a “trimmed” vector. So I wrote my own (with help from http://www.math.mcgill.ca/~steele/math680/lecture2.680.pdf).
trim < - function(vect, percent=0.1) {
nelem <- length(vect)
howmany <- round(nelem * percent)
sort(vect)[(howmany + 1):(nelem - howmany)]
}
Note that the parenthesis around howmany + 1 and nelem - howmany
are apparently quite important. In use:
> 1:10
[1] 1 2 3 4 5 6 7 8 9 10
> trim(1:10)
[1] 2 3 4 5 6 7 8 9
> trim(1:10, percent=0.2)
[1] 3 4 5 6 7 8
Once again, Emacs makes this easier: write the function in a scratch
buffer, M-C-x to eval the function in the current R process. Note
that 1:10 makes some sort of… I don’t know what it is, but I guess
it creates something that acts like a vector, if it’s not exactly a
vector.
Now lets see what the data for beacon A1 (that is, ID ending in A1)
looks like with 10% trimming from both ends:
> sd(trim(a1[[1]]))
[1] 0
So, uh. There you go.
Before I forget, I think you can do help.start() and it’ll pop up a
browser (if it can) with local HTML docs. This is helpful to have
open. You can also see how you’ve been polluting your name space:
> objects()
[1] "a1" "last.warning" "mydata" "origdata" "position1"
[6] "trim"
> rm(origdata, mydata)
> objects()
[1] "a1" "last.warning" "position1" "trim"
This might be important because it looks like, on exit, you can choose to save your current session to disk. When you start R next time from the same directory, you’re popped right back into your session. This is why they recommend that, for each project/set of data/whatever you want to work with in R, you work in a separate directory.
Lets look at beacon BA:
> levels(beaconid)
[1] "01:7D:CA:60:0A:00:00:8A" "01:9E:14:C4:0A:00:00:A1"
[3] "01:9F:FF:C3:0A:00:00:BA"
> ba < - subset(position1, beaconid=="01:9F:FF:C3:0A:00:00:BA", select=distance)
> summary(trim(ba[[1]]))
Min. 1st Qu. Median Mean 3rd Qu. Max.
205.0 206.0 206.0 205.8 206.0 206.0
> sd(trim(ba[[1]]))
[1] 0.4012177
That’s really not so bad, either.
Of course, I just realized: it would be nice if I knew if that was anywhere near the correct distance. Will have to experiment with a measuring tape at some point.
What happens if I stopped after the first 20 readings?
> summary(trim(ba[[1]][1:20]))
Min. 1st Qu. Median Mean 3rd Qu. Max.
205.0 206.0 206.0 205.8 206.0 206.0
> sd(ba[[1]][1:20])
[1] 0.5231484
Not bad. In fact, with fewer than 20 readings I still seem to get between 205.8 and 206 for the average. I can live with 0.2cm.
Will have to work more on this later, dinner time now.
Friday, 04 November 2005
OpenSwan, Monotone
Lets say you set up a tunnel with something like:
conn foo
right=1.2.3.4
rightsubnet=192.168.0.0/24
left=5.6.7.8
leftsubnet=192.168.1.0/24
auto=start
You’re on the left, your firewall has the address 192.168.1.1 inside (eth0), 5.6.7.8 on the outside (eth1). Bring the tunnel up. Ping something on the 192.168.0.0/24 (right) network. It doesn’t work. Why? Because the tunnel is going out over eth1, so your pings are going out with a source IP of 5.6.7.8, not 192.168.1.1. Solution?
conn foo
right=1.2.3.4
rightsubnet=192.168.0.0/24
left=5.6.7.8
leftsubnet=192.168.1.0/24
leftsourceip=192.168.1.1
auto=start
That leftsourceip (rightsourceip exists also, of course) adds the
correct src option to ip route add. Now you can ping 192.168.0.*
and your packets get the correct source address.
So you want to host multiple projects using
Monotone. Trouble is, it seems they
recommend one database per project. That means one server process per
project, and one port per project as well. Messy. There exists,
however, contrib/usher.cc which claims to essentially dispatch to
other Monotone servers based on the branch (I think?) being requested.
You still have to start a bunch of servers, and they’re going to eat
up ports, but those ports don’t have to be opened on your firewall at
least, and you don’t have to remind each project what port they’re
supposed to be using. Disclaimer: I haven’t actually tried usher.
(Though I believe Monotone developers are talking about making usher
a supported part of
Monotone.
I rebuilt the Monotone SRPM from FC4 for CentOS 4. One problem: the init script doesn’t work:
Nov 4 15:19:08 zeus runuser: -bash: 3: Bad file descriptor
Nov 4 15:19:08 zeus monotone: monotone-server startup succeeded
Nov 4 15:19:08 zeus monotone: succeeded
The fix? Take the daemon function from FC4’s
/etc/init.d/functions and copy it into /etc/init.d/monotone right
below the line that reads . /etc/rc.d/init.d/functions. Now worky.