darkness

Thursday, 24 November 2005

Adaptec 2410SA in Linux

site admin @ 15:48:07
I found out that the Adaptec 2410SA SATA “HostRAID” (i.e., software RAID in our drivers) card might be supported in Linux these days. I guess it uses a Marvell chipset? I just had to tell someone. I believe the 2410SA is a 4 channel PCI-X 133MHz card, whereas the 2410 which probably has better Linux drivers is “only” PCI 64-bit/66MHz.

Sunday, 13 November 2005

From waproamd to NetworkManager

darkness @ 21:07:02

Last night my X server suddenly started doing this weird cycle where it would (apparently) allocate over 1GB of memory, then free it; and it repeated this cycle until I killed the X server. After I killed the X server, APM suspend/resume stopped working. Fun! So I rebooted, and into the newest Fedora kernel, kernel-2.6.14-1.1637_FC4. I also upgraded ieee80211 and ipw2100 from ATrpms (ieee80211-kmdl-2.6.14-1.1637_FC4-1.1.6-6.rhfc4.at and ipw2100-kmdl-2.6.14-1.1637_FC4-1.1.3-36.rhfc4.at, respectively).

Once I rebooted, waproamd no longer worked:

Nov 13 17:17:28 darkbook waproamd(wlan0)[4107]: Scanning...
Nov 13 17:17:28 darkbook waproamd(wlan0)[4107]: Corrupt scan result (2)

That’s all I got. waproamd’s page does note that you should stop using it in favor of wpa_supplicant. Additionally, it hasn’t been updated in quite some time. Rebuilding it didn’t rectify this problem, so alas, I had to scrap it.

I think my main problem with wpa_supplicant (other than it’s kind of ugly configuration file with lots and lots of options, compared to waproamd scripts that just call ifup) was that it had no way to detect and properly configure the access points at school. The school’s APs have “hidden” ESSIDs. Literally, iwlist scan shows them as <hidden>. I have been told that <hidden> is something specific to the ipw2100 driver, but I’m not so sure of that these days. At any rate, I gather there’s some ap_mode=2 setting in wpa_supplicant these days that will let you work with hidden access points. I’m not sure if it uses the standard RH/FC ifup/ifdown commands; I kind of doubt it. Further, I’m not sure it has an option to kick off scripts for particular networks. For example, at home I need something to start OpenVPN. (With waproamd, that just goes in the appropriate script.)

Somehow I found NetworkManager. It’s in FC4 already: install NetworkManager and NetworkManager-gnome. (Without that last one, you don’t get the applet and thus have zero control over NetworkManager, or so it seems.) You probably want to start the NetworkManager and dhcdbd services now (via service) and at boot (via chkconfig). You may also want to mkdir -p /etc/NetworkManager/dispatcher.d and enable/start the NetworkManagerDispatcher service. When that last service is started, scripts in /etc/NetworkManager/dispatcher.d get called with the device name as their first argument and up or down as the second argument. So I stuck some scripts in there to start OpenVPN, restart ntpd, that sort of thing.

(Also, I got a lot of this information from http://www.ces.clemson.edu/linux/nm.shtml.)

One problem, though. My APM scripts rmmod ipw2100 at suspend. I’m not sure if it’s still necessary to do this; it was at one time, and as recently as my initial FC4 upgrade on this laptop. Generally it seems like any time I take out some of my old APM tricks to get suspend/resume working, I end up breaking suspend/resume. Which is to say that the same problems I’ve always had still plague me. At any rate, I wasn’t really interested in trying to run without that rmmod at suspend time. The problem seems to be in HAL: it doesn’t see the network interface after you remove and re-insert the module. With the version that came in FC4, I couldn’t get the device “node” (I don’t know if that’s the right term, but I’ll roll with it for now) to appear even after I did a service haldaemon restart. I heard that newer HAL versions had some power management features. So, I upgraded to hal-0.5.4.cvs20051111-1 from Fedora development (which is Rawhide…? Or not?). I couldn’t find that it does anything interesting with power management. Further, I dug into D-BUS and Python a bit. I came up with stuff like this:

import dbus

slashHal = "/org/freedesktop/Hal"
dottedHal = "org.freedesktop.Hal"

bus = dbus.Bus.get_system()
spm_obj = bus.get_object(dottedHal,
             slashHal + "/Device/SystemPowerManagement")
spm = dbus.Interface(spm_obj, dottedHal + ".Device.SystemPowerManagement")

def getDev(udi):
obj = bus.get_object(dottedHal,
             "%s/devices/%s" % (slashHal, udi))
return dbus.Interface(obj, dottedHal + ".Device")

card = getDev("pci_8086_1043")
wlan = getDev("net_00_0d_43_8f_68_f5")
computer = getDev("computer")
bridge = getDev("pci_8086_2448")

I tried stuff like spm.Hibernate() which bitched that the security policy wouldn’t let me do that. (I think it wants me to be “at console,” based on my uninformed interpretation of /etc/dbus-1/system.d/hal.conf. I don’t know how it determines if you’re “at console.”) card is the node for the PCI device that is the wireless card. It’s there regardless of whether ipw2100 is loaded or not. wlan would be the node for the actual network device, if it existed, which it doesn’t. computer is the whole computer (I guess; it’s the root of the tree) and bridge is the PCI bridge that both of my network cards are on. You can call Rescan and Reprobe on any of them with no positive effect (i.e., computer.Reprobe()).

Finally I figured out that, with the new version of HAL, service haldaemon restart was putting the proper network device node in the tree. So now that’s part of resume. Except when you do service haldaemon restart, NetworkManager dies, so you have to start it back up. Here’s my /etc/sysconfig/apm-scripts/apmcontinue as of now:

#!/bin/bash

case "$1" in
    suspend|standby)
        #dbus-send --system --dest=org.freedesktop.NetworkManager \
        #          --type=method_call /org/freedesktop/NetworkManager \
        #          org.freedesktop.NetworkManager.sleep
        service NetworkManager stop
        ifconfig wlan0 down
        rmmod ipw2100
        service vmware stop
        ;;

    resume)
        #dbus-send --system --dest=org.freedesktop.NetworkManager \
        #          --type=method_call /org/freedesktop/NetworkManager \
        #          org.freedesktop.NetworkManager.wake
        service haldaemon restart
        service NetworkManager start
        chvt 1
        chvt 7
        ;;
esac

You’ll note that I found the sleep/wake methods on NetworkManager, but if you’re going to stop and restart it, not a lot of point in bothering with them as near as I can tell.

I’ll also mention that it seems you can add a new device into HAL. Why should I have to do that, though?

Saturday, 12 November 2005

Using R

darkness @ 21:52:37

Situation: I’ve got a Cricket getting distance readings from three other Cricket beacons. The data file looks like:

01:9E:14:C4:0A:00:00:A1 219
01:7D:CA:60:0A:00:00:8A 150
01:9F:FF:C3:0A:00:00:BA 205
01:9E:14:C4:0A:00:00:A1 219
01:7D:CA:60:0A:00:00:8A 150
01:9F:FF:C3:0A:00:00:BA 206
01:9E:14:C4:0A:00:00:A1 219
01:7D:CA:60:0A:00:00:8A 150
01:9F:FF:C3:0A:00:00:BA 206
01:9E:14:C4:0A:00:00:A1 219
.
.
.

The first column is the beacon ID, the second is the computed distance to that beacon. Right now our beacons send out a signal about once a second. (In one case it took me four minutes to retrieve >200 readings for each of the three beacons.) The goals are to know when we’ve sampled enough data to make a reasonable guess at what the real value for distance is, and of course to actually make that guess.

Ideally, we’d like to have our distance accurate to 1cm, or maybe 2cm; of course, it’s not clear how useful math or statistics or anything else short of prayer will be if the Cricket sends back totally bogus results at any point. We’ve had spots where its reported distance to beacon was much too high; we attributed this to ultrasound reflection. Still, if we trust that the data coming in is “generally right,” we just need some math/statistics to “scrub” the data. We need to know which data points to keep, and which to throw out. (Alternatively, I might just be using this as a diversion to learn more about statistics and statistics software.)

I’m using R 2.2.0 from Fedora Extras, with Emacs Speaks Statistics. I’m also using my old statistics text book. I took one statistics course at college. I consider myself to have done well in that course, and (surprisingly) I feel like I grok the textbook. All that said… it was one course. The guy barely spoke English. A little bit of knowledge can be very dangerous. So if you think the stuff I’m doing here is pointless, flawed, or you simply have a better suggestion for doing this, please do let me know.

Anyway. Given the above file, I read the data in like this:

> position1 < - read.table("distances.1", header=FALSE)
> position1
V1  V2
1   01:9E:14:C4:0A:00:00:A1 219
2   01:7D:CA:60:0A:00:00:8A 150
3   01:9F:FF:C3:0A:00:00:BA 205
4   01:9E:14:C4:0A:00:00:A1 219
5   01:7D:CA:60:0A:00:00:8A 150
6   01:9F:FF:C3:0A:00:00:BA 206
7   01:9E:14:C4:0A:00:00:A1 219
8   01:7D:CA:60:0A:00:00:8A 150
9   01:9F:FF:C3:0A:00:00:BA 206
10  01:9E:14:C4:0A:00:00:A1 219
.
.
.

No headers in my input file. < - is an assignment operator; you can reverse the arrow, so long as it’s pointing to what you’re assigning to. You can rename V1 and V2 to something better (like beaconid and distance) with edit(position1).

position1 is now a data.frame (try class(position1)). The beaconid column is non-numeric, so data.table made it into a “factor.” I don’t know enough to explain what a factor is, but you can use it for grouping. Check this out:

> attach(position1)
> tapply(distance, beaconid, summary)
$"01:7D:CA:60:0A:00:00:8A"
Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
149.0   150.0   150.0   150.1   150.0   151.0 
$"01:9E:14:C4:0A:00:00:A1"
Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
51.0   219.0   219.0   217.6   219.0   220.0 
$"01:9F:FF:C3:0A:00:00:BA"
Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
85.0   205.0   206.0   203.5   206.0   207.0

attach basically makes the columns of the given object accessible without being qualified by the object’s name. That is, without attach(position1) first, I’d have to use position1$beaconid instead of just beaconid. tapply takes the vector as its first argument (the columns of a data.frame are vectors–or, at least, they are in this case) and divides it up based on the factor given as its second argument, then calls the given function (summary here) with each of the resulting vectors. I think of it as analogous to GROUP BY in SQL. By the way, if you want to undo attach, just detach().

It looks like the first beacon is probably 150cm away. I noticed that the mean and the median had some disagreement for the second and third beacons. I wasn’t crazy about it. But look at the minimums: way too low for the beacon IDs ending in A1 and BA. So trim the top and bottom 10% of the data set:

> tapply(distance, beaconid, mean, trim=0.1)
01:7D:CA:60:0A:00:00:8A 01:9E:14:C4:0A:00:00:A1 01:9F:FF:C3:0A:00:00:BA 
150.0000                219.0000                205.7964

Much better. The trim=0.1 is an argument that’s passed to mean.

Variance and standard deviation:

> tapply(distance, beaconid, var)
01:7D:CA:60:0A:00:00:8A 01:9E:14:C4:0A:00:00:A1 01:9F:FF:C3:0A:00:00:BA 
0.08623402            208.02434337            217.29764082 
> tapply(distance, beaconid, sd)
01:7D:CA:60:0A:00:00:8A 01:9E:14:C4:0A:00:00:A1 01:9F:FF:C3:0A:00:00:BA 
0.2936563              14.4230490              14.7410190

What would those look like if you trim the data set, as we did with mean, above?

Diversion: subset is kind of interesting. I was interested in pulling out just one beacon’s readings into a separate vector:

> a1 < - subset(position1, beaconid=="01:9E:14:C4:0A:00:00:A1", select=distance)
> a1
distance
1        219
4        219
7        219
10       219
13       219
16       220
19       219
21        85
24       219
27       219
.
.
.
> class(a1)
[1] “data.frame”

Not a vector, but instead a data.frame; that’s hardly a problem right now, though. By the way, you can do things like position1[[1]] and position[1]. They both output the beacon ID column, but in different formats:

> class(position1[1])
[1] "data.frame"
> class(position1[[1]])
[1] "factor"

I still want to trim data. I don’t see anything in R that returns a “trimmed” vector. So I wrote my own (with help from http://www.math.mcgill.ca/~steele/math680/lecture2.680.pdf).

trim < - function(vect, percent=0.1) {
    nelem <- length(vect)
    howmany <- round(nelem * percent)
    sort(vect)[(howmany + 1):(nelem - howmany)]
}

Note that the parenthesis around howmany + 1 and nelem - howmany are apparently quite important. In use:

> 1:10
[1]  1  2  3  4  5  6  7  8  9 10
> trim(1:10)
[1] 2 3 4 5 6 7 8 9
> trim(1:10, percent=0.2)
[1] 3 4 5 6 7 8

Once again, Emacs makes this easier: write the function in a scratch buffer, M-C-x to eval the function in the current R process. Note that 1:10 makes some sort of… I don’t know what it is, but I guess it creates something that acts like a vector, if it’s not exactly a vector.

Now lets see what the data for beacon A1 (that is, ID ending in A1) looks like with 10% trimming from both ends:

> sd(trim(a1[[1]]))
[1] 0

So, uh. There you go.

Before I forget, I think you can do help.start() and it’ll pop up a browser (if it can) with local HTML docs. This is helpful to have open. You can also see how you’ve been polluting your name space:

> objects()
[1] "a1"           "last.warning" "mydata"       "origdata"     "position1"   
[6] "trim"        
> rm(origdata, mydata)
> objects()
[1] "a1"           "last.warning" "position1"    "trim"

This might be important because it looks like, on exit, you can choose to save your current session to disk. When you start R next time from the same directory, you’re popped right back into your session. This is why they recommend that, for each project/set of data/whatever you want to work with in R, you work in a separate directory.

Lets look at beacon BA:

> levels(beaconid)
[1] "01:7D:CA:60:0A:00:00:8A" "01:9E:14:C4:0A:00:00:A1"
[3] "01:9F:FF:C3:0A:00:00:BA"
> ba < - subset(position1, beaconid=="01:9F:FF:C3:0A:00:00:BA", select=distance)
> summary(trim(ba[[1]]))
Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
205.0   206.0   206.0   205.8   206.0   206.0 
> sd(trim(ba[[1]]))
[1] 0.4012177

That’s really not so bad, either.

Of course, I just realized: it would be nice if I knew if that was anywhere near the correct distance. Will have to experiment with a measuring tape at some point.

What happens if I stopped after the first 20 readings?

> summary(trim(ba[[1]][1:20]))
Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
205.0   206.0   206.0   205.8   206.0   206.0 
> sd(ba[[1]][1:20])
[1] 0.5231484

Not bad. In fact, with fewer than 20 readings I still seem to get between 205.8 and 206 for the average. I can live with 0.2cm.

Will have to work more on this later, dinner time now.

Friday, 04 November 2005

OpenSwan, Monotone

darkness @ 15:36:16

Lets say you set up a tunnel with something like:

conn foo
    right=1.2.3.4
    rightsubnet=192.168.0.0/24
    left=5.6.7.8
    leftsubnet=192.168.1.0/24
    auto=start

You’re on the left, your firewall has the address 192.168.1.1 inside (eth0), 5.6.7.8 on the outside (eth1). Bring the tunnel up. Ping something on the 192.168.0.0/24 (right) network. It doesn’t work. Why? Because the tunnel is going out over eth1, so your pings are going out with a source IP of 5.6.7.8, not 192.168.1.1. Solution?

conn foo
    right=1.2.3.4
    rightsubnet=192.168.0.0/24
    left=5.6.7.8
    leftsubnet=192.168.1.0/24
    leftsourceip=192.168.1.1
    auto=start

That leftsourceip (rightsourceip exists also, of course) adds the correct src option to ip route add. Now you can ping 192.168.0.* and your packets get the correct source address.


So you want to host multiple projects using Monotone. Trouble is, it seems they recommend one database per project. That means one server process per project, and one port per project as well. Messy. There exists, however, contrib/usher.cc which claims to essentially dispatch to other Monotone servers based on the branch (I think?) being requested. You still have to start a bunch of servers, and they’re going to eat up ports, but those ports don’t have to be opened on your firewall at least, and you don’t have to remind each project what port they’re supposed to be using. Disclaimer: I haven’t actually tried usher. (Though I believe Monotone developers are talking about making usher a supported part of Monotone.

I rebuilt the Monotone SRPM from FC4 for CentOS 4. One problem: the init script doesn’t work:

Nov  4 15:19:08 zeus runuser: -bash: 3: Bad file descriptor
Nov  4 15:19:08 zeus monotone: monotone-server startup succeeded
Nov  4 15:19:08 zeus monotone:  succeeded

The fix? Take the daemon function from FC4’s /etc/init.d/functions and copy it into /etc/init.d/monotone right below the line that reads . /etc/rc.d/init.d/functions. Now worky.

Powered by WordPress