darkness

We had a client’s hardware raid controller decide to lose its configuration, thus making their array unusable. It has been our experience with RAID controllers that you can’t say “here’s how the array is configured, don’t blow away the data on the drives please.” This particular controller was an AMI Megaraid controller, and after taking a lengthy route around it, I now believe there was at least a 50% chance that this controller would let us configure an array without initializing it. However, we didn’t want to risk that 50% chance that it would ruin our client’s data of which they had no backup. So my boss called me in to see what I could do.

I read up about partition tables, RAID 5 and how parity is calculated, NTFS, and I even found someone who had done what I was thinking: use Linux software RAID to access the array and at least get the data off. Oh, did I mention that no one had any idea how the array was configured, really? Six SCSI drives, no idea how they were allocated. Later my boss told me that, before the breakage, the controller was alerting him that it was constructing two logical drives. So I was kind of hoping it was three drives to each logical device, in RAID 5, and as it turned out later I was right.

First I had to activate the drives in the controller, as they were “ready” but not “online;” as it turns out I had to make logical disks of all the separate disks. This makes sense, of course, though I don’t think all other controllers do it this way, and I think several that I’m used to working with definitely don’t do it this way. Then we bought a couple of 120GB IDE drives. I used one to load RH9 on and the other to back up each SCSI drive to. sda got backed up to hdc1, sdb to hdc2, and so on, using dd. This took a while, naturally (no ATA/66 even).

After the backup was complete I disconnected the SCSI drives to keep myself from munging them in my sleep-deprived state. Then I built Linux Disk Editor which is OK. I used it with the switches --paranoid -t no. The first tells it to open the device in read only, the second tells it not to try and detect the filesystem type, since it doesn’t know about NTFS. It worked well as basically a hex editor. Let me take this moment to say: Linux seems to be lacking a real good hex editor that can open devices. Something like LE looks like it has a nice interface, but it won’t open devices. Oh, and the graphical ones were no help either, like KHexEdit. Of course, I just found a page with a bunch of Linux hex editors and I haven’t tried a few of them, so maybe I just didn’t look hard enough. Get with Freshmeat developers, please.

One thing that puzzled me initially was that I could see the partition table on four of the two drives, and on the combinations sda/sdc and sdd/sdf they were the same partition table. I started to think maybe they had two RAID 1 arrays with a hot spare for each, but after trying the arrays out, no luck. BTW, I “tried them out” using the Linux-NTFS Red Hat packages which work most excellently and have been kept up well for RH9. I’d use this driver later to test the drives once I figured out how to reconstruct them. Save yourself some worrying and remember to give -r to mount to make sure the mount point is read only. I believe the Linux NTFS drivers provided are read only anyway, but I’m always nervous when I dicking around with data I have no backup of.

I browsed around on the disks a bit and found that the “parity algorithm” was left symmetric. It also looked like the block size was 8K. I could usually tell this by seeking somewhere into the disk and finding some identifiable data, and watching for where it abruptly stopped on one disk and started on the other. Identifying the parity in this case was very easy as well: I had three xterms open with LDE for each of the three drives in (what I theorized was) the array, find a block of readable data, look for two readable and one that isn’t; the one that isn’t is the parity block. For a reference on how the various “parity algorithms” (as Linux calls them, at least) work, try this page on Linux RAID Information.

Next I tried a method for getting Linux software RAID to make an md device out of my former hardware RAID members as detailed in that post. Of course, once I did this I had an image of an entire disk, partition table and all, on /dev/md0. There is a patch for using partitions on a software RAID device but I didn’t have it and I didn’t feel like rebuilding the kernel. So I dd’ed just the one partition off of the software RAID device onto the 120GB boot drive which had plenty of space. Once that was done I was able to successfully mount the files on a loop device. That is to say, this whole mess of stuff worked.

I found the other array had a failed SCSI disk. Perhaps not so surprisingly after all this, using failed-disk in raidtab also worked. After I had pulled the two partitions off the second array, I fdisked the drive holding the images of all the array members to write the partition files back to it (dd). Voila, someone else could pop the drive into an NT machine and pull the data off of it. In fact I probably could have taken the old SCSI array, reconfigured it, replaced the broken drive, and dded the drive images back on to it; except I think we were still leery about erasing the original data, or what was left of it.

In other news, I’ve created two scripts that use Abcde to rip all my audio CDs into FLAC files. I think I had to patch Abcde a bit too. I’ll try and get that all posted at some point.

Tonight I opted to start ripping my CDs. I’m ripping to FLAC because I’ve now got the disk space for it, and I figure I should rip my CDs before they get stolen from my car or something. (Note: please don’t steal my CDs from my car. I KNOW WHO YOU ARE.) I was putting off ripping to FLAC and keeping albums in FLAC on a permanent basis because I was unsure that there was a standard for FLAC data a la ID3v2.4.0. It seems that, at least in FLAC 1.1.0, there is a metaflac command which can be used to add this kind of data to a track.

I went out looking for programs to use. I have written my own script to do ripping and encoding (with LAME) in a combination of bash and python. I liked that script well enough, but I figured that maybe someone has something written that I could use to greater effect, nicer interface, something like that.

As it turns out I was mostly wrong. First I looked at Emptytree Seedy. Emptytree looks cool, and might do what I want, but honestly I don’t need something quite this heavy… I think. Its interface does look nice, but honestly I’d probably prefer something with a console interface. Also I’m not entirely sure that Emptytree Seedy supports the method of operation that I want: pop a CD in the drive, CD is ripped, optionally normalized, encoded to FLAC, tagged to the best of CDDB’s ability, and then the drive is ejected. When next a CD appears in the drive the software should detect it and start ripping again. As I seem to recall, this is basically how my shell script worked.

So then I went through all four pages of CD audio ripping applications at Freshmeat. The ones I found that might be acceptable are — in no particular order – Jack, rip, YaRET, and I just found abcde in another Mozilla window and it looks acceptable.

First I tried YaRET. I got it all nice and configured, installed libcdaudio, normalize, AppConfig, Audio::CD, and libmad, and… it doesn’t seem to work. I set it to normalize the entire album (batch mode) and after ripping it just sits there. I don’t feel like debugging Perl right now, so I think we’re moving on.

Next I tried abcde. Got it set up. Think it works. Realize it doesn’t expect to keep being handed CDs. The problem here is that I could be ripping the next CD while normalizing and encoding the last. Get discouraged.

So now I’m kind of maybe writing my own, which is pretty insane I bet. I might end up using abcde in parts, really. Hey, yeah, that makes the most sense. abcde seems to have a nice interface for telling it exactly which portions of its abilities you’re wanting to use (like “rip,tag,encode” or something like that). I’ll just start two processes: one that rips, and one that normalizes/encodes/tags. Hopefully that’s easy. Much easier than writing from scratch.

While I’m thinking about it, Disc-Cover looks kind of neat. I think I also saw a reference to a CD Cover Database somewhere; might need to check in to that later.

I decided to keep my Discover Classic, despite the fact that I wanted the Discover Platinum. Maybe I can get them to quickly bump my credit limit up to the supposed $5,000 minimum needed for the Platinum. Anyway, I had the dilemma of signing my card. Some people sign, “Please see ID” or something similar. I’ve always been a card signer. Google around, though, and you find people saying that the official Visa/MasterCard policy is to refuse a card that is not signed with the person’s signature. One of the first hits I found, in fact, was a LiveJournal entry from someone who claims to have had their card rejected at the post office because it wasn’t signed. I went ahead and signed it, more because that’s what I’ve always done and… who really gets their card checked anyway? These days a lot of the people that even look at the back of the card will ask to see my ID regardless.

I don’t think I’ve mentioned this in my web log yet: “The Ur-Quan Masters,” the open source port of the 3DO version of Star Control 2, rocks. I can now play SC2 on my Linux/x86 machine, my Windows desktop, and my Linux/PPC Powerbook. Of course, my save games only work on x86, and I suspect this is because the save game files don’t pay attention to endianness. I recommend a CVS build. For Linux this should be quite easy. For Windows, try Striker’s CVS builds. Also recommended: mining map, and once you get into Quasispace, this map of Quasispace outlets will help; information about organisms on planets; and a SC2 constellation search engine. Beware the search engine, though: it may give you spoilers when you search on a constellation and it has something interesting in it (”comments” field).

Pan is cool. I was using Forte Agent to leech on my Windows box today, and for whatever reason I decided to fire up Pan and hit my local news server whilst Agent was attacking Giganews. Turns out Pan supports multiple news servers, which I don’t think Agent does. It also defaults to several parallel connections to my news server. I think its task management is better, and it has a task list that I don’t think Agent has. Plus it’s in Linux. It also joined up all the different parts to binaries, which Agent seems to require me to do manually. Overall very impressed. I think I’ll be using this to leech until I find something wrong with it, or I find something better.

I also decided to upgrade Mozilla on my Windows box today. I was going to go with Firebird, but it doesn’t have an installer; they lose. I like things to be listed in Add/Remove Programs, sorry. I did notice the list of Firebird extensions though, and then realized that many/most of them work in Mozilla as well. In particular I think the Yahoo! Companion bar for Mozilla has the potential to score points: bookmarks that are stored at Yahoo! Finally we can have synchronized bookmarks between Unix and Windows? Perhaps. Aggreg8 looks cool, but I didn’t try it. I get the feeling its developer is on temporary hiatus, and I don’t think it can beat News Is Free — other than the fact that I could add my own RSS feeds. Maybe I’ll look at it for that fact some day. I also installed Diggler and Tabbrowser Extensions. I suspect the latter has some bugs, but it might be nice to at least be able to move tabs around in order, or move them from window to window. I always want to do that.

I downloaded and installed Nero 6. I didn’t care for much other than its burning software: I don’t really trust Nero/Ahead to deal with stuff like audio mixing and video formats and such. Nero Burning ROM is… about the same. I didn’t notice any mind-blowing new features. The interface is now puzzling and kind of ugly, honestly. Writing this I’m starting to wish I hadn’t upgraded. Maybe it’ll grow on me.

I was looking for an interface to take notes on a project I’m working on. tag suggested PersonalBrain. It’s pretty cool. I wish it had an export feature (this may be in PersonalBrain 3.0, which is in beta; check their front page for download information), the ability to do numbered lists, and the ability to draw in the notes… like OneNote.

OneNote is a cute program. It has about everything I’d like it to, and the interface isn’t bad. Except it just doesn’t feel as useful without the easy interface to writing on the notes. Even though I have a crappy stylus, it’s just not enough (er, at least through VNC – I need new KVM cables). I have the feeling, though, that this application would be extra super special when used on one of these “tablet PCs” I hear about from time to time. Interestingly I’ve never heard anything like “tablet PCs are really taking off.”

In the end I think PersonalBrain’s method for associating nodes is going to win me over. I’ll just hope that I can manage to get the data out at some point if I ever want a printed representation of my notes.

I’m listening to Velocity Girl right now. I think I should start doing my take on some of the new music I’m listening to. I hesitate to call it a “review” because that sounds too definitive. Sometimes when I read a negative review of a film, movie, or book that I liked, I tend to get kind of defensive. Sometimes adding pronouns like “my” or “I” can help, like “my opinion/review” or “I think…” At least, they help in my mind.

Weirdo.

grub got screwed up on a system where every device is RAID 1, including the boot partition. I needed to reinstall it. So we try grub-install /dev/hda, right? Nope, grub says there’s “no corresponding BIOS drive for /dev/md0″ or some such. grub-install /dev/md0? Keep dreaming.

This is the solution for reinstalling grub on a software RAID 1 boot drive that I found, and tried. (Well, really Doofus tried.) It worked apparently. Amazing.

More amazing is that the Red Hat installer gets grub onto RAID 1 devices just fine. Port that into grub-install, bitches! Thank you.