I. Great Albums

ALBUM OF THE YEAR: Washington Square Serenade by Steve Earle
Cut: City of Immigrants

I’m sad to see Texas-born Earle moving further afield, leaving “Guitar Town” (”This place never been my home,” he laments in the opening cut Nashville Blues) for New York City (I suspect he was chasing a skirt). But so far the effect on his music has been positive, especially on my selected track City of Immigrants. Earle tends to write the same rock and roll song over and over, but on this album he’s all over the map stylistically, with far more hits than misses.

Barenaked Ladies Are Me - Barenaked Ladies
Cut: Running out of Ink

I haven’t been crazy about the Ladies with the exception of their brilliant debut album Gordon. But they’ve shrugged off their major label and figured out how to do things their way. This album is a gem, with singer-songwriter quality writing and top-pop-act quality arranging.

Alright Still - Lily Allen
Cut: Smile

This cheeky bird from England had a huge hit with Smile across the pond, and it did OK here. Her album packages shallowness as product, with most songs dissing ex-boyfriends (”When I see you cry/It makes me smile”) or guys with the audacity to flirt with her. I only discovered her because I was showing a friend Rhapsody and he thought she was cute and clicked on her.

Five Score and Seven Years Ago - Relient K
Cut: Bite my Tongue

If you were looking for rock on the dial in 2007, your best bet was the Christian stations. This Green Day-esque quartet dropped a fun album with half religious themes, half young-and-in-love songs. My favorite was probably “Faking My Own Suicide” but I chose “Bite my Tongue” as more representative.

Here & Now - America
Cut: One Chance

Two thirds of the trio from the ’70s has been soldiering on with state fair appearances for thirty years, recording lots of albums that everyone ignored. But suddenly it all came together again for them in a great album. Sadly this album, one of my most-played in the first half of 2007, mysteriously got pulled from Rhapsody mid-year.

Lifeline - Ben Harper
Cut: Fight Outta You

I live with a young, hip, musically-inclined theology student who turned me on to Ben Harper.

Soundtrack to I’m Not There - Various Artists
Cut: Pressing On - John Doe

This album of Dylan covers could have been my album of the year but it just overwhelmed me. It goes on for about two hours and for every cut I wish I knew when it was written, what else Dylan was writing then, and what else the covering band has done. This one I know: about 1980, during Dylan’s evangelical era (Gotta Serve Somebody), recorded by one of the two principals of legendary L.A. punk band X.

II. Great Cuts from Less-Great Albums

Someday - John Mellencamp

Like “Lay My Hammer Down” from Steve Earle’s album, this is a millennial vision from a non-religious album.

The Simpsons Theme - Green Day

OK, this isn’t a great cut, it’s a proxy. My real album of the year was Green Day’s American Idiot, released in 2003 but discovered by me in 2007 and played about 50 times.

Heart of Glass - The Puppini Sisters

This British trio got a lot of play in the NPR crowd with their Andrews Sisters-style covers of 40s hits. Enjoyable enough, but they only broke new ground on their covers of slightly-more modern hits. Here, someone transcribed and harmonized a skat.

Hop a Plane - Tegan and Sara

The third album from these Canadian identical twins is full of songs I almost like.

Living in the Future - Bruce Springsteen

After a two-year detour into traditional folk (see my 2006 best-of list) the Boss returns to rock. Not any better than his old stuff, but then again not any worse.

South Texas Girl - Lyle Lovett

Probably my least-favorite LL album but still full of stuff worth paying attention to. My family had a Ford Galaxy growing up, a close relative of the Ford Fairlane in this song, so it puts a lump in my throat.

The Longer the Waiting - Josh Turner

My favorite love song of the year. The Irish sea-ditty is an underutilized genre among country artists.

Vietnam Cowboys - Ray Davies

Kinks’ songwriter Davies delivers the album I keep hoping Mark Knoppfler’s new albums will be — mild satire, with a groove.

Too Much of Everything - Tinsley Ellis

I don’t know anything about this artist, I just loved the guitar work on this cut from someone else’s 2007 best-of-the-blues list.

Candyman - Christina Aguilera

Christina 1, Britney 0 (if the metric they care about is appearances on my best-of list). Though Brit’s Piece of Me came close.

I haven’t found any kind of a ‘chsh’ facility in CygWin, so I did it by hand. I edited /cygwin/cygwin.bat, replacing the line “bash –login -i” with “tcsh -l” (((edited 20070729 to add -l))).

(((edited 20070729 None of the following is needed anymore, now that I added the -l for login to the tcsh command)))

I don’t know how bash gets its PATH; it’s not in /etc/bash.bashrc or the default ~/.bashrc. But somehow when I started CygWin with bash it had /usr/local/bin:/usr/bin:/bin before the Windows path in its $PATH. So I modified my ~/.cshrc to add those:

setenv SHELL /bin/tcsh

You can access my playlist on Rhapsody here:

Click here

You can even listen to it! When you click on a play button it will pop up the Rhapsody Player window. If you haven’t used Rhapsody before you’ll have to go through an install process, but I will vouch that the plugin is not malware, and it even works on Linux! (Anyone can listen to something like 25 songs a month for free; to hear more you have to subscribe.)

Anyway, on to my choices:

Thunder on the Mountain by Bob Dylan

From the album of the year, in my opinion. Bob is back with a collection of all original material that finds a groove early on and stays in it all record long. All killer no filler. I could have picked any of a half dozen tracks from this record; none stand out as better than the others.

Back to Earth by Willie Nelson

This maybe doesn’t deserve to be on the list on its own merits but was a sentimental pick — I saw Willie perform this at Hollywood Bowl a few weeks before it was released. And it is a fine bit of songwriting, though the performance here is uninspired.

O Mary Don’t You Weep by Bruce Springsteen

A barely-recognizable boss on a strong album of folk songs made famous by Pete Seeger.

The Lighthouse’s Tale by Nickel Creek

This is a bit of a cheat — this song has been out for a few years but did appear on a greatest hits album in 2006. But this band was new to me in 2006, a favorite of my new housemate James. Bluegrass for Generation Y.

Crazy by Gnarls Barkley

When Gnarls Barkley started getting popular I assumed that with a name like that they would be too rap for me. Then I heard the music.

The Yeah Yeah Yeah Song by Flaming Lips

I thought this went in this, the hip new bands section of my playlist. But James tells me they’re a 90s band that made a comeback. I hated this song the first time I heard it but it grew on me.

Chinese Translation by M. Ward

This is my current favorite song, combining shallow pop sensibility with a slightly-profound lyric.

Mas Que Nada by Sergio Mendes, remixed by The Black Eyed Peas

This is the coolest track from an album of hip-hop remixes of Mendes tunes — unfortunately still not as cool as this 1972 original:

click here

Sea of Love by Tom Waits

Tom waits doesn’t just cover a song, he smothers it, almost strangles it.

It’s a Heartache by Rod Steward

OK, I’m not proud of this choice. But Stewart is, AFAIK, the first aging rocker recording an album of standards to choose rock standards. And he makes them his own — when listening to his covers, I can’t remember what the original sounded like.

O Come All Ye Faithful by Twisted Sister

You might assume Twisted Sister’s Christmas album would be all tongue in cheek, and much of it is. But this, the only religious song on the album, is done straight up, and it works for me.

Like the 309 by Johnny Cash

Another sentimental pick — the last song Johnny Cash wrote. Cash had a busy 2006, releasing five albums including two live albums — pretty good for a dead guy! I don’t think it’s a great song, but does feature some nice three-part guitar work.

OK, that’s it for 2006. Stay tuned, 2007 is already shaping up to be another good year for aging boomers like me — America has a surprisingly strong album of originals out, and John Mellencamp has released one that seemed a little preachy on a first listen but is growing on me.

I read my email on an internet server that I administer; it lives in a commercial data center and is always on and always connected to the internet with a permanent IP address. It is a linux Fedora Core 5 (FC5) distribution running linux kernel 2.6.18. Email addressed to me on this machine is accepted by sendmail-8.13.8. If sendmail sees a .procmailrc file in my home directory, it will pass all my mail to the procmail program (I have procmail 3.22) instead of dropping it directly into my mail box.

When I am logged on with emacs running, I can run the RMAIL program which transfers any messages it finds in my mail box into my RMAIL file.

So my basic plan to get going with bogofilter is to create a .procmailrc file that causes procmail to run bogofilter. If bogofilter decides the message is spam, I just output it to a special mailbox that I will almost never look at. If bogofilter is not sure the message is spam (if it classifies it either as ham or as unsure) then I will have procmail drop the mail in my normal system mail box. This is going on all the time, whether I am logged in or not. When I do log in and run emacs/RMAIL, only messages that are in my normal system mail box get read. Spam just accumulates in the special mail box, and I can glance at it every day or two to see if I am losing legitimate mail to my spam box.

When you first download bogofilter, either from source or from an RPM, it has logic but it has no data. Before it can begin classifying your email as spam or ham, it needs data on your email. So I started sorting my email a week before I installed bogofilter. I’ve been getting around 400 spams a day, and about 30 emails that I want to read. So during this week, I continued to let everything get delivered into my RMAIL file. Then, I copied every single message out of my RMAIL file into one of two files — one file for spam and one file for ham. To do this, I used the Ctrl-O command in RMAIL, which copies the current message to a unix-mbox-format file. So if I was in RMAIL and I typed “Ctrl-O~/spam.mbox” then the current message would get appended to spam.mbox in my home directory. I used keyboard macros to make this easy — specifically, I defined a keyboard macro for spam, named it, and attached it to F9; and defined a different keyboard macro for ham, named it, and attached it to F5. After that, as I was reading my email each day, I pressed F9 on every message that was spam and F5 on every message that was ham. At the end of a week, I had about 2000 messages in spam.mbox and 200 messages in ham.mbox. I would use these messages to train bogofilter.

Finally I was ready to install bogofilter. Don’t be afraid to install it; it doesn’t do anything when you install it except make the program available. You have to take additional steps before it will actually start being used. I installed it from a FC5 RPM. Now the ‘bogofilter’ executable exists in /usr/bin/bogofilter. I created my personal database by passing in my already-existing spam.mbox and ham.mbox and telling bogofilter how to classify them. I ran the following logged in to my normal user account:

$ bogofilter -s -M < ~/spam.mbox
$ bogofilter -n -M < ~/ham.mbox

The -M option means I am passing in a whole mailbox not just a single message; -s tells bogofilter that all these messages should be considered spam; -n tells bogofilter that all these messages should be considered ham. At the end of this, bogofilter had created my personal word count database in ~/.bogofilter/wordlist.db. Now bogofilter knows what it needs to know to start classifying my email. However, I still haven't done anything dangerous -- though bogofilter now knows how to filter my mail, I haven't actually done anything to make it start doing so.

Now I was ready to create my .procmailrc script to tell procmail how I want to use bogofilter. First, I decided I don't like the -u option in bogofilter. When you specifiy -u, bogofilter decides if something is spam or ham, and assumes it is right so it then updates its database with all the words it found in that email. If you later decide bogofilter made a mistake you can correct the database by calling '$ bogofilter -nS' or '$bogofilter -sN' but I didn't want to mess with that. Maybe it works great in practice but it just rubs me the wrong way. My plan is to rely mostly on the training I already did, and to only use mistakes to train bogofilter in the future. If I find ham in my spam box, I will tell bogofilter it is ham and have it update its DB; likewise if I have spam in my regular mailbox I will tell bogofilter it is spam and have it update its DB. I also left alone the config file that the RPM installed in /etc/bogofilter.cf; this config file has everything commented out so bogofilter will use all defaults.

So with those considerations, I developed the following .procmailrc, based partially on the examles in the bogofilter docs.

--------------------------------------------------------------------------
# Set to yes when debugging
VERBOSE=no

# Remove ## when debugging; set to no if you want minimal logging
## LOGABSTRACT=all

# Replace $HOME/Msgs with your message directory
# rules that direct mail to a folder will put the folder in this dir
MAILDIR=$HOME/Mail # Make sure this directory exists!

# Directory for storing procmail-related files
PMDIR=$HOME/.procmail

# Put ## before LOGFILE if you want no logging (not recommended)
LOGFILE=$PMDIR/log
## INCLUDERC=$PMDIR/testing.rc
## INCLUDERC=$PMDIR/lists.rc
## INCLUDERC=$PMDIR/perl.rc

#
# ':0' is just a magic literal introducing a new rule
# lines beginning with * define a condition for selecting mail
# the last line of a rule is the action to take

# flags: f = filter, w = wait and check filter's exit code
:0fw
| bogofilter -e -p

# if bogofilter choked, try again later
# flags: e = error - only executes if previous rule returned an error
# the MTA will retry to deliver it later
# 75 is the value for EX_TEMPFAIL in /usr/include/sysexits.h
:0e
{ EXITCODE=75 HOST }

# OK, anything that gets to here has been classified, and Bogosity header added
:0
* ^X-Bogosity: Spam, tests=bogofilter
spam.mbox

# Most mail will fall through to end, and get delivered to my mail spool
-------------------------------------------------------------------
When you save the .procmailrc that is the first moment that you are actually turning on filtering and diverting putative spam away from RMAIL. I only had to wait a couple of minutes before I saw that ~/Mail/spam.mbox had been created, and when I looked in it, it contained spam! Yay!

So that's basically the setup I have been running with for about a week now.

As for the results, I am both pleasantly surprised at how well it works and bitterly disappointed that, as well as it works, it doesn't work nearly well enough. By using the default config file, bogofilter classifies everything as spam (spamicity > 0.99), ham (spamicity < 0.45 I think) or unknown. So far, about 800 messages have gone to my spam.mbox, and I believe they are all spam. I’ve received a few true personal messages and noticed that they get classified as ham with very low spamicity values. However, I’m still getting about 200 spams a day into my RMAIL. So with the default cutoffs and the little bit of training I’ve done so far, only about 1/2 my spam is being diverted. Some messages with very obvious spam words (like penis and \/iagra) failed to get classified as spam.

I’ve defined a keyboard macro that I hit every time I see spam in my RMAIL. It pipes the content of the RMAIL buffer to ‘bogofilter -s’ and then deletes the message from RMAIL. I hope with this ongoing training bogofilter will get better and better at recognizing my spam.

It turned out to be a selinux problem. Recall that FC5’s “targeted policy” for selinux is to restrict only the most dangerous processes, i.e. internet services. I guess that when you run httpd from the command line it has no special role, but when you run it from the SYSV system it has a restricted role (httpd_t). In my case I was doing something the typical setup wouldn’t do (reading in a HTTPS SSL password from a file) which selinux was crushing. The evidence for this was in the /var/log/messages file.

I went through a process of modifying the selinux policy until things worked. Sometimes giving permission for one operation would just cause it to try a new operation that was forbidden, so it took several iterations of trying it, checking /var/log/messages for audit messages, changing the policy, and trying again. Eventually I would up with the following list of audit messages that needed to be corrected:

Jul 30 11:31:33 cecosstream kernel: audit(1154284293.730:20592): avc: denied { execute } for pid=9248 comm=”httpd” name=”ssl.key.dialog” dev=hda2 ino=164115 scontext=user_u:system_r:httpd_t:s0 tcontext=user_u:object_r:httpd_config_t:s0 tclass=file
Jul 31 21:32:25 cecosstream kernel: audit(1154406745.527:20607): avc: denied { execute_no_trans } for pid=21243 comm=”httpd” name=”ssl.key.dialog” dev=hda2 ino=164115 scontext=user_u:system_r:httpd_t:s0 tcontext=user_u:object_r:httpd_config_t:s0 tclass=file
Jul 31 21:35:55 cecosstream kernel: audit(1154406955.512:20609): avc: denied { getattr } for pid=21280 comm=”ssl.key.dialog” name=”utmp” dev=hda2 ino=1111943 scontext=user_u:system_r:httpd_t:s0 tcontext=system_u:object_r:initrc_var_run_t:s0 tclass=file
Jul 31 21:37:46 cecosstream kernel: audit(1154407066.335:20613): avc: denied { read write } for pid=21324 comm=”ssl.key.dialog” name=”utmp” dev=hda2 ino=1111943 scontext=user_u:system_r:httpd_t:s0 tcontext=system_u:object_r:initrc_var_run_t:s0 tclass=file
Jul 31 21:40:17 cecosstream kernel: audit(1154407217.324:20623): avc: denied { lock } for pid=21370 comm=”ssl.key.dialog” name=”utmp” dev=hda2 ino=1111943 scontext=user_u:system_r:httpd_t:s0 tcontext=system_u:object_r:initrc_var_run_t:s0 tclass=file

Here’s the process to turn some audit messages into policy that will allow the transactions to take place:

# audit2allow -m local_20060731 -l < audit_20060731.txt > local_20060731.te
# checkmodule -M -m -o local_20060731.mod ./local_20060731.te
checkmodule: loading policy configuration from ./local_20060731.te
checkmodule: policy configuration loaded
checkmodule: writing binary representation (version 5) to local_20060731.mod
# semodule_package -o local_20060731.pp -m local_20060731.mod
# semodule -i ./local_20060731.pp

where audit_*.txt contains a list of all the avc denied messages from /var/log/messages.

I installed lyceum blogging software (it’s a multi-user variant of wordpress) on my web server several weeks ago. It uses a mysql database as the backend. It worked fine for several weeks, until my server rebooted for the first time. After that, all my lyceum web pages were broken or had no content, and the lyceum error log had tons of entries more or less like this:

=====database error=====
Error#: 1033
Error: Incorrect information in file: ‘./uv_lyceum/options.frm’
Query: SELECT option_value FROM options WHERE option_domain = ’system’ AND option_name = ‘template’ LIMIT 1
========================

It turns out what happened is this: when I installed lyceum, it created its DBs using a mysql backend called innodb. A few weeks later I installed some other package (drupal I think) that suggested putting “skip innodb” in my /etc/my.cnf file. I did so, but it didn’t take effect because I didn’t restart mysqld. When my machine rebooted, mysql started in a state unable to deal with innodb files. That’s what caused the above, relatively uninformative, error message. All I had to do to fix it was comment out the skip innodb line in my.cnf, and restart mysqld.

I’m in bad need of some bustin’ out.

Is there anybody out there?

Who will bust out with me?

I spent yesterday at a Water Recycling Workshop, held at an organic farm/think tank at Cal Poly Pomona. But the workshop was actually delivered by a Northern California outfit called Solar Living Institute which tries to help people apply green living ideas to their everyday lives. In fact, SLI is hosting a great Festival in a couple of weeks which I wish I could attend; perhaps I’ll make it up there next year. But then, it wasn’t actually an SLI person who delivered the training, but a guy from the Orange County Water District.

His name was Dr. Stephen Lyon and this man knows water. The morning was spent on stuff that wasn’t really relevant to what I want to do with water in my life, but it was fascinating — he crammed a one-semester course on water chemistry and the hydrologic cycle into an hour and a half (ask me about the different roles of nitrates and nitrites — go ahead, ask me!) and traditional large scale water treatment. In the afternoon he got to what I was interested in — greywater systems (cleaning up your bathwater enough to use for watering plants, so the city doesn’t have to pay to treat it) and rainwater harvesting.

It kind of made me wish I’d become an environmental engineer, instead of whatever it is I’ve become. I think he’s had an interesting career so far (he’s a few years older than me) and had a chance to both do some science and make a difference. And I’m sure he’s made a lot more money than me too!

I probably won’t blog here much; my main way to communicate with the wider world is through the Bobologue which doesn’t quite fit the rules of a blog. But I may throw a short news item up occasionally.