darkness

Wednesday, 27 September 2006

Of SSH agents and screen

darkness @ 14:42:48

Updated 2007-05-21: I think I have some changes to the option parsing code, some indentation fixes, and a note that it actually requires gawk (not just awk as I found on default *BSD).

I run into a problem frequently:

  1. On host A (perhaps my desktop computer) I add my SSH keys into the ssh-agent instance I’m always running under.
  2. I SSH into host B. I can do ssh-agent -l on host B and see the keys that I added on host A because of agent forwarding.
  3. On host B I start a Screen session.
  4. I do some work, then detach the Screen session and log out of host B.
  5. Later, on host C (perhaps my laptop computer) I add my SSH keys again.
  6. I SSH from host C into host B.
  7. I reattach my Screen session on host B.

At this point, in the Screen session on host B, I don’t have any keys in my agent. In fact, I can’t add any: the environment variables point to the agent forwarding from my SSH connection from A to B, but that connection has been closed.

Here is my solution, all from my ~/.bashrc:

### SSH agent forwarding under a long running screen

# We need to know where our Screen FIFOs are kept so we can check for
# a duplicate session name.
[ -d ~/.screens ] || { mkdir ~/.screens; chmod 700 ~/.screens; }
SCREENDIR=~/.screens
export SCREENDIR

# Think of it as a parameterized constant.
get_screen_auth_sock() { echo ~/.ssh/agent-screen-"$1"; }

# Clean up dead sockets.
find ~/.ssh -maxdepth 1 -path "$(get_screen_auth_sock '*')" -type l \
| while read link; do
    [ -e "$link" ] || rm -f $link
done

sshscreen() {
    if [ "x$STY" != "x" ]; then
        echo "don't use sshscreen from inside screen" >&2
        return 1
    fi

    if ! type -P gawk >/dev/null; then
        echo "sshscreen requires gawk (not found)" >&2
        return 1
    fi

    local OPTIND=1 opt pattern session num_sessions sock
    local create=0 reattach=0 set_name=""
    while getopts ":r:x:d:R:D:S:" opt; do
        if [ "x${opt##[rxdRD]}" = "x" ]; then
            reattach=1
            if [ "x${OPTARG##-[a-zA-Z]}" = "x" ]; then
                OPTIND=$(($OPTIND - 1))
            else
                pattern="-S $OPTARG"
            fi
        elif [ "x$opt" = "x:" -a "x${OPTARG##[rxdRD]}" = "x" ]; then
            # Reattach option with no option argument.
            reattach=1
        elif [ "x$opt" = "xS" ]; then
            create=1
            session="$OPTARG"
        elif [ "x$opt" = "x:" -a "x$OPTARG" = "xS" ]; then
            echo "-S requires an argument" >&2
            return 1
        fi
    done

    if [ $create -eq 1 -a $reattach -eq 1 ]; then
        echo "sshscreen can't handle -S and a reattach option as well" >&2
        return 1
    elif [ $create -eq 0 -a $reattach -eq 0 ]; then
        # I assume we're creating a new session.  I attempt to mimic
        # the default Screen session name here.  I fear the
        # portability of "hostname -s".  (I mean, really, I fear the
        # portability of a whole lot of this.)
        create=1
        session="$(tty | sed 's!^/dev/!!; s/[^a-zA-Z0-9]/-/g').$(hostname -s)"
        set_name="-S $session"
    elif [ $reattach -eq 1 ]; then
        # Three argument form of match() is a GNU extension.
        session=$(screen $pattern -ls \
              | gawk '/[ \t]+[0-9]+/{match($0, /[0-9]+\.([^ \t]+)/, m);
                     print m[1]; c = c + 1} END{exit(c)}')
        num_sessions=$?
        if [ $num_sessions -le 0 ]; then
            echo "no matching sessions found" >&2
            return 1
        elif [ $num_sessions -gt 1 ]; then
            echo "more than one matching session, please be more specific" >&2
            screen $pattern -ls
            return 1
        fi
    fi

    if [ $create -eq 1 ]; then
        find $SCREENDIR -print | sed 's/^[0-9]*\.//' | fgrep -q -- "$session"
        if [ $? -eq 0 ]; then
            # We can't have a duplicate session name, because they
            # would share the same SSH agent socket.  Note that by
            # "session name" I mean the portion of the name listed by
            # "screen -ls" with the leading "<pid>." removed.
            echo "session name '$session' not unique," \
                "please specify a different one with -S" >&2
            return 1
        fi
    fi

    sock=$(get_screen_auth_sock "$session")
    if [ -e "$SSH_AUTH_SOCK" ] && [ ! -e "$sock" -o -L "$sock" ]; then
        ln -sf "$SSH_AUTH_SOCK" "$sock"
    fi

    # It isn't necessary to specify SSH_AUTH_SOCK when doing a
    # reattach, only when creating a new session.
    SSH_AUTH_SOCK=$sock screen $set_name "$@"
}

Installation: put it in your ~/.bashrc. Adjust $SCREENDIR or make ~/.screens where your screen sockets will live. screen may require certain permissions of this directory that you don’t have set; if so, it will bitch and refuse to start the next time you run it, which is fine. If you have existing screen sessions, don’t try moving the socket files. I did that and was unable to ever reattach that session; I had to kill it. Instead, maybe set SCREENDIR to wherever your system keeps screen sockets.

The quick version: instead of using screen to create new sessions or to reattach detached sessions, use sshscreen. So instead of running screen -S foo irssi to create a new session with irssi in window 0, use sshscreen -S foo irssi. (Just sshscreen works to start a new session too, of course.) Instead of running screen -x foo to reattach, use sshscreen -x foo. sshscreen might accept all the other parameters you’re used to giving to screen (but some don’t do what you expect, like sshscreen -ls won’t list anything). To be safe you might want to give the switch with the session name first. Note that sshscreen won’t have any effect on a screen session unless that session was started with sshscreen.

This all works by making SSH_AUTH_SOCK point to something like ~/.ssh/agent-screen-foo when running in the Screen session foo. ~/.ssh/agent-screen-foo is really a symbolic link to the appropriate SSH agent socket, and this link is updated to your current agent socket every time you use sshscreen to reattach.

As an example, you log in to host B via SSH and reattach using sshscreen -x foo. This causes ~/.ssh/agent-screen-foo to be symbolically linked to the value of SSH_AUTH_SOCK (the agent socket for your current SSH connection).


Some implementation notes:

Note that, on my system at least, when [ -e ... ] is used on a dangling symbolic link, it returns false.

while getopts ":r:x:d:R:D:S:" opt; do

This tries to grab every option that you might use to create or reattach a session, so that it can figure out the name of the session you’re trying to operate on. It’s not very well thought out, I can guarantee you; but so far it works for me. In particular, I think it might break if you included some options prior to the option that specifies which Screen session to operate on. (This could possibly be fixed by including more Screen options, just so getopts doesn’t think it has encountered a non-option argument.)

You don’t always have to tell Screen a session name, though. If you try to create a session without naming it, sshscreen tries to make one up for you. If you only have one session and do something simple like sshscreen -x, your lone session should get reattached correctly.

One area where screen works and sshscreen doesn’t is creating two sessions from the same host and TTY. The reason for this: the default session name is something like <pid>.<tty>.<host>. However, sshscreen doesn’t know the PID of the “window manager” Screen process, which is what <pid> refers to. So while screen can create two sessions with the same <tty>.<host> (but different <pid>s), sshscreen only knows <tty>.<host>. sshscreen will refuse to create two sessions with the same name.

(If you didn’t understand any of that, let me summarize the impact of it. Some day you might get a session name '...' not unique error; just use -S somethingrandom and you’ll be fine.)

session=$(screen $pattern -ls \
          | gawk '/[ \t]+[0-9]+/{match($0, /[0-9]+\.([^ \t]+)/, m);
                                 print m[1]; c = c + 1} END{exit(c)}')

Screen lets you specify a partial session name. I thought about emulating this behavior and searching $SCREENDIR myself, but that seemed like a lot of trouble. So instead I came up with this nasty gawk script (third argument to match() is a gawk extension) to search for the given session name. We have to get the session name, so that we know what name we need to use for the agent socket link. We have to get this name before calling Screen so that we can get the value of SSH_AUTH_SOCK.

Friday, 22 September 2006

Quickly: DRAC III/XT and CentOS 4

darkness @ 14:38:46

I administer a server at a colo. This server is a Dell server with a DRAC out-of-band management card. First, the good: the card lets you control power to the server, gives some monitoring I believe, and gives you remote access to the (non-GUI, unless you install their helper software which is basically VNC) console. Finally, it has a remote floppy feature: you can upload a floppy image to the card and it’ll boot off of it. With these sorts of features, the only reason I’d need someone at the colo to touch the box is if I have a hardware failure of some sort—and I can troubleshoot the problem that much easier with ability to see the POST, go into BIOS, and boot diagnostics floppies if I so desired. I’ve already used the remote console/floppy features to upgrade RHEL 3 to CentOS 4 and to move the software from 2 separate disks to software RAID 1.

Now, the bad: fucker is hard to get working. The colocation facility I use had some problems with it in the beginning. Specifically it seems to choose random ports to upload the floppy image with, so their firewall was blocking it. Also, assigning the user name and password to log into the DRAC web interface was apparently a little difficult for them to get right and to integrate with their all-encompassing provisioning (I’ll call it) system.

Furthermore, fucker doesn’t work with Linux. I swear it did at one point in time, but that’s not true today with Firefox 1.5 and Sun Java 1.5 on Fedora Core 5. I couldn’t get it to work in Windows with the same Firefox/Java, either. Of course, the only thing that ended up working was IE (also with Sun Java, oddly). Well, in truth, the first time I tried to load the DRAC remote console (== Java applet that looks like VNC under the hood to me) applet, my XP VM (running in VMware) did a BSOD. It worked fine after a reboot of XP.

But when I say “it doesn’t work with Linux,” it doesn’t just stop there. At this point I’m more murky about whether I’m blaming the DRAC card or Linux, though. (Maybe I should blame both.) First of all, though, let me say this: my server BIOS also has a “console redirection” feature. This is the typical feature you’re used to seeing where it redirects the console out the serial port. The DRAC has nothing to do with this feature, nothing to do with the serial port. Just turn it off (unless you really want to use the serial port; my colo box has nothing hanging off the serial port AFAIK, and certainly nothing I have access to). The DRAC intercepts the video card and keyboard, uh, “subsystems” directly. Somehow.

Aaaanyway, so I boot the system, I see everything I should in the DRAC remote console. I can enter and interact with the BIOS. I can interact with Grub. I can see the kernel booting, see all the init scripts running. But my keystrokes seem to have no effect once the kernel boots. I press enter, nothing happens on the screen. strace and other means of snooping on various ttys shows nothing coming in. This is very strange, since as I mentioned I used the DRAC in the past to good effect.

The problem? It was working in kernel 2.4 (RHEL 3). Turns out, something in kernel 2.6 breaks the DRAC. The solution is to add i8042.dumbkbd=1 to the kernel command line. i8042 is apparently the driver (and probably the chipset) that handles the PS/2 keyboard and mouse communication. I note I didn’t see anything during boot from the kernel that led me to believe it had any problem interacting with the keyboard; I even got the little serio: line saying it found the keyboard. Other people have reported serious input problems with the DRAC and this option but it seems to work great for me.

So is the DRAC a shitty keyboard emulator? Possibly. I’m still of the opinion that they should have made sure the basic peripherals like PS/2 keyboard and PS/2 mouse (see my previous rants about mouse breaking through my KVM) keep working, but (A) that’s a lot of (buggy!) hardware to test with, and (B) it’s free so submit a patch or shut up, I guess.

Final note: research leads me to believe that others have this problem, but for a different reason. I suspect the DRAC 5, and maybe DRAC 4 cards, emulate a USB keyboard. So you need to make sure the USB HID modules get loaded for your DRAC keyboard to work under those circumstances.

Wednesday, 20 September 2006

More notes on my experiences with Debian Sarge

darkness @ 21:20:36
  • Not having less after a (minimal) install was killing me.
  • It would be nice if apt-get install would keep a log of what packages I was installing, so I know which ones I’ll want for my next Debian installation (which might be… today).
  • Debian seems to generally have more granular packages than RH/FC. For example, NTP is broken down into ntp, ntp-server, ntp-simple for a configuration, ntp-doc, ntpdate, and I think a few more. I consider this granularity to be generally a good thing.
    • One exception I’ve found: OpenSSH seems to be entirely in ssh (OK, OK, maybe there’s an ssh-askpass or something too). RH/FC breaks this up into openssh-server and openssh-client.
    • In fact, I found the package name ssh a little scarier than openssh; was I installing some ancient SSH 1.x? Of course, the description line said “openssh” so I’d know it was safe. (One could now talk about how RH/FC calls Apache httpd, for example.)
  • When looking for the name for NTP (xntpd? ntpd? ntp? NTP?) I went looking for the URL to the upstream on packages.debian.org’s information about ntp. I can’t find an upstream URL anywhere. RPM spec files often have this URL in the URL: field. Where is it for debs? This is useful information.
  • I installed ntp-simple and it started ntpd. Then realized I needed ntpdate. I installed ntpdate, which killed ntpd, ran ntpdate, and then… didn’t restart ntpd. Oops?
  • Little things: apt-get install doesn’t ask for confirmation if it only needs to install what you specified on the command line (i.e., no dependencies needed). yum install always asks for confirmation (which is probably why I often use yum -y install). I didn’t realize how annoying that confirmation prompt in yum was until I stopped having it so often with APT.
  • Argh dir_index isn’t turned on for ext3 filesystems by default. Did I miss where you can enable that from the installer? dir_index makes things so much faster with ext3. Now I have to enable the option on all my filesystems, then boot in such a way that I can e2fsck -fD them. Pain in the ass.
    • Slight offset to this pain in the ass: set SULOGIN=yes in /etc/default/rcS, then enter you root password and e2fsck from there. RH/FC doesn’t have this “always run sulogin” sort of feature, AFAIK, and now I wish it did.
    • Before anyone says “Sarge is old” as a defense to why it doesn’t use directory hashing: Sarge uses 2.6.8. RHEL 4 uses 2.6.9 (well, you know, RH patched 2.6.9) and AFAIK it turns on dir_index by default. Assuming I don’t hit some catastrophic filesystem corruption issue, I don’t see why Sarge doesn’t turn it on.
  • APT pinning is still really cool. I just used it to upgrade Sarge’s kernel to the one from Sid. Surprisingly to me, the system was still bootable and everything.
    • One problem I had was that Perl started bitching about missing locales, since the locales package was upgraded along with the kernel. I have no idea if this really screwed anything up, but I ran dpkg-reconfigure locales again, checked to make sure I had en_US ISO-8859-1 and en_US.UTF-8 selected. When I ran dpkg-reconfigure by hand it seemed like there were some actions taken at the end to actually build the locales that weren’t taken at install time. Maybe it would be wise to upgrade locales by hand first (along with any dependencies; libc perhaps?), then upgrade the kernel.
    • In the process of removing the old kernel-image package it went into config-files status. I gather this means “we’ve removed all of this package except for configuration files you might want to keep around.” Fine, great. So I ran dpkg --purge kernel-image-whateverversion. Now, according to dpkg -l that kernel-image package is in the “not installed” state, with “purge” being the desired state. Poking around more I see I can spot some “unknown” desired state, “not installed” state packages with stuff like dpkg -l '*kernel*'. But it’s not showing me all avilable packages, as judged by doing apt-cache pkgnames | grep ^kernel. Maybe dpkg -l only knows about packages you’ve had installed at one time or another? I made some of these “weird” dpkg -l entries disappear with dpkg -\\-clear-avail or dpkg --forget-old-unavail. All in all, I find this confusing. I hope there’s a good reason for dpkg to know about uninstalled packages and then expose this information so easily.
  • On a side note, while looking for information about APT, I found that the “Debian Reference” has some good suggestions on running an unstable desktop on an otherwise stable machine using chroot. I find that to be a pretty cool idea.
  • I also gather that Aptitude, which I previously thought to be a GUI program only, has both a curses and a command line interface. Furthermore, if you use it to install a package, and then to remove that same package, the removal will also attempt to remove any unneeded dependencies of the target package. In other words: fewer orphaned packages hanging around. I gather there are some other tools that attempt to do stuff like this, deborphan being one. Anyway, this is a cool feature of Aptitude.
    • Er, but wait: why does it seem like Aptitude’s command line (i.e., aptitude install) doesn’t tell me about recommended and suggested packages, like apt-get install does? I guess I’ll keep using apt-get

Since APT won’t keep track of it, I will! Packages I installed:

ssh less screen ntp-simple ntpdate rsync sysv-rc-conf vim iproute lsof sudo lynx elinks cvs strace ltrace lftp nmap iptraf ltrace units binutils smbclient dnsutils tcpdump

May want to add apt-file to this list.

Tuesday, 19 September 2006

Impressions of Debian

darkness @ 12:50:23

I’m installing Debian Sarge on a fairly average x86 box: Pentium 4, Asus P5P800-VM motherboard, 1GB RAM, two SATA hard drives.

I went ahead and set myself up for a network boot/install using PXELINUX. I do Linux installs often enough that this could potentially save me some time. I took the pxelinux.cfg/default file from netboot.tar.gz and modified it for my purposes. The installer’s version of “expert mode” simply sets DEBCONF_PRIORITY to a lower value, which means you see all messages of that (lower) priority and above.

It netbooted pretty much fine. I had downloaded the netinst ISO and made it available locally via Apache, but it took me a while to figure out how to tell the installer to use my local mirror. The trick: when asked what country you’re in, in order to find a close mirror, scroll to the top and select something like “manually enter location.” I found this kind of unintuitive, but once I did find it the installer gladly used the extracted netinst files.

The initial problem I had with Sarge was that the Promise “RAID” (ha) controller in the system, a FastTrak TX2300, was apparently not supported by the installer’s older kernel. I fixed this by moving the SATA drives on to the motherboard’s controller. Then I removed the FastTrak (which I never asked to be installed in this system) and promptly throwing it into the pond behind my apartment. Promise sucks. Seriously.

The installer is text mode, mostly fast, and easy to use. It let me install onto LVM on software RAID (once I installed the necessary installer modules—and I find the concept of those little installer “plug-ins” pretty cool). One oddity: it kept telling me the partition table on one of my software RAID devices was invalid, and telling me to reboot before using it. Stupidly I did actually reboot, and still got the error. I guess I’m just supposed to ignore this because everything works now.

The install of the base system went quickly and I was then booting into Debian. One thing I noticed was a whole bunch of messages:

devfs_mk_dir: invalid argument.<4>devfs_mk_dev: could not append to parent for /disc

Some later searching finds Debian bug #320379 which says “yes, this is a problem; no, we’re not fixing it in stable.” It relates to devfs, I guess, which is now gone. I suppose they didn’t fix this in stable because it’s really only cosmetic… but I find myself kind of wishing they would have fixed it. It’s probably of some value to see some of the boot messages that precede this wash of errors.

Despite the fact that I had only the base system installed, the Debian boot process is still kind of noisy for my tastes. In Red Hat, you get single lines telling you what’s being started/stopped followed by OK or ERROR. With that it’s pretty easy to see at a glance if something didn’t work, and if you want to see what was on stdout/stderr you can find it in /var/log/messages. Debian appears to just let the programs run wild; for example, I’m treated to dhclient’s copyright/advertising message every time it gets run. I suppose all this noise is a matter of taste…

I still love what’s included in Debian’s minimal install, which is to say almost nothing. RH/FC’s minimal install can be rather large. I strongly suspect APT is still more powerful than yum, as evidenced by stuff like pinning (thanks phealy).

Comparing the installers is pretty much a wash: they ask a lot of the same stuff. I don’t care about graphical vs. text mode; I only started using graphical in RH because the text installer couldn’t do fun stuff like software RAID and LVM (at least, at the time; I don’t know what the status is today). Debian had that weird warning about no partition table on the software RAID device. Debian was nice enough to ask for parameters for kernel modules, and I don’t think RH/FC does that (though I suppose there’s almost certainly somewhere you can supply parameters). The Debian partitioning tool was nicer, letting me set things like nosuid,nodev at install time. It seemed faster and easier to navigate with the keyboard, too.

One thing that worries me slightly is an apparent lack of an easy “remote install” feature in Debian. Sometimes a server is far away from me, so I send someone else a RH or Fedora CD, have them type linux vnc at the boot: prompt, answer a few questions about language and testing the CDs, then have them read me the IP at the bottom of the screen. Then I can VNC in and complete the (graphical) installation. I didn’t see anything quite this easy from the Debian installer, but I did see something: there’s some kind of SSH server module in the installer. It probably would take a little more time to walk someone through selecting that and getting it going over the phone, but it should still be possible at least. Maybe this is easier in Etch.

When the post-boot configuration (the name of which I’ve already forgotten; was it base-config?) ran it was pretty easy to use. No option to set up an NTP server, unlike RH/FC, but that’s because there’s no NTP server installed by default. (Like I said: minimal is really minimal.) Setting up Exim was a little confusing. It gave me four or five choices, the two most promising being something like “Internet host, receive mail via SMTP” and “Local only: not on a network, no outbound.” The choices were confusing to me: I don’t want to receive mail from the Internet at large via SMTP, but I do need the box to be able to send out mail via SMTP (i.e., for the root alias). I ended up selecting the first option, “Internet host” or whatever, and happily found it defaulted to binding the SMTP server to 127.0.0.1, which was what I was looking for in the first place.

One final thing that bothered me about the installation was that there was no SSH server started by default. I said before that I liked how minimal the Debian minimal installation was. However I consider an SSH server to be a basic part of any system I install. Maybe there are lots of people out there that don’t want an SSH server on by default, I don’t know. As I mentioned before, I sometimes do remote installations, and I’d hate to have to walk someone through even more stuff after the system boots to disk just to get the ssh package installed. However, I’m betting that there’s some semi-easy way to tell the installer to include ssh in its base installation. (Of course, I’d also need to set a root password after the installation. If you have a shell, which the installer gives you, you can do this; but how easy is it? RH/FC asks you for a root password at install which I probably prefer.)

Sunday, 17 September 2006

Mixing setuptools and distutils

darkness @ 14:10:44

I’m making a Python package I’m currently calling ApyConfig, and I’m using setuptools to distribute it. My package depends on pyparsing, which is distributed using distutils. This has caused a minor problem with dependencies.

My setup.py:

import ez_setup
ez_setup.use_setuptools()

from setuptools import setup, find_packages

setup(name = "ApyConfig",
      version = "0.1",
      packages = find_packages("src"),
      package_dir = {'': 'src'},
      install_requires = ["pyparsing"],
      test_suite = "tests",
      dependency_links = ["http://sourceforge.net/project/showfiles.php"
                          "?group_id=97203",  # pyparsing
                          ]
      )

(That URL was the only one I could use to get easy_install to download pyparsing for me, right now at least. My theory is that setuptools has some built-in intelligence about SourceForge, but some changes SourceForge made broke setuptools.)

If you know me, you know I’m the kind of guy that puts everything outside of my home directory (and maybe /srv and /opt) under control of the system package manager, most commonly RPM in my case. distutils has long been able to build an RPM from a Python package. So I extract a pyparsing distribution, run python setup.py bdist_rpm, and install the resulting RPM. Then I go to run ApyConfig’s tests with python setup.py test:

[darkness@morgase ApyConfig]$ python setup.py test
running test
running egg_info
writing requirements to src/ApyConfig.egg-info/requires.txt
writing src/ApyConfig.egg-info/PKG-INFO
writing top-level names to src/ApyConfig.egg-info/top_level.txt
writing dependency_links to src/ApyConfig.egg-info/dependency_links.txt
reading manifest file 'src/ApyConfig.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/ApyConfig.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
...
  File "/usr/lib/python2.4/site-packages/pkg_resources.py", line 483, in resolve
    raise DistributionNotFound(req)  # XXX put more info here
pkg_resources.DistributionNotFound: pyparsing

“Oops.” The problem here seems to be that pyparsing is missing an .egg-info directory. A little searching around, some experimentation, and I thought the right thing to do when installing pyparsing would be more like:

python -c 'import sys; sys.argv[0] = "setup.py";
           import setuptools; exec open("setup.py")' bdist_rpm

Except, of course, this doesn’t work with bdist_rpm. It would if I were just doing an install, I think; but bdist_rpm packs the source, builds an RPM spec file, and that spec file then runs setup.py without my little setuptools patch above.

In short, the easiest thing to do in a situation like this seems to be something like:

perl -pi -e 's/distutils.core/setuptools/g' setup.py
python setup.py bdist_rpm

This works like a charm, at least for this package. The moral of the story: if you want to use the various bdist_* targets, it seems like you need to patch setup.py. If someone knows a better/proper way to get around this, please let me know.

Powered by WordPress