about:nothing

Here are some tools that I’ve put together that make a huge difference to a clunky rfe workflow. Bored already? Don’t worry, I’ve written the script already: tl;dr

Imagine power cycling a DICE server with redundant PSUs, using our lovely power bar control files. You don’t know where it’s installed, so you have to search for it. Read the rest of this entry »

I’ve been aware of alpine’s “display filter” feature for some time, used as it is for on-the-fly GPG interpretation amongst other things. But I’d never really examined the feature before now. The manual says:

The [filter] command is executed and the message is piped into its standard input. The standard output of the command is read back by Alpine.

This says it all: display filters turn out to be an extremely powerful generic mechanism for reformatting and enhancing text; it works particularly well when applied to machine generated messages. Maybe its power is best explained by the example which caused me to investigate in in the first place:

An example (the nagios bit):

A longstanding irritant to me has been a my difficulty in shutting nagios up. For a long time I’ve been relying on a filter to parse nagios’ incoming emails and generate a URL. The display filter closes the loop, automatically injecting that magic URL at the end of the message.

Here’s a simplified version of the filter, reminiscent of the one in the previous post:

#!/usr/bin/gawk -f
# Crude detection of problem type for acknowledgement link
# Don't forget to validate these inputs...
/Notification Type: / { TYPE=$3; }
/Service:/ { SERVICE=substr($0,length($1)+1,length($0)); }
/Host:/ { HOST=$2; }
# Important: this is a filter, so don't forget to print input lines back out!
// {print;}
# Now add the acknowledgement link below:
END {
    if (HOST && TYPE == "PROBLEM") {
        # this is the script which generates the URL.
        # ideally this should be replaced with some awk to do the same thing
        cmd="~/bin/nagack "HOST" "SERVICE
        cmd | getline url
        close(cmd)
        # now add the link to the email.
        print "[Acknowledgement link: "url" ]"
}

Now, to alpine’s Display Filters setting, add:

Display Filters    = _LEADING("***** Nagios")_ /path/to/nagios-filter-script

that’s it! My emails from nagios now look like:

***** Nagios *****
Notification Type: PROBLEM
Service: ssh
Host: myhost
Address: 192.168.12.34
State: CRITICAL
...
[Acknowledgement link: https://nagiosserver/nagios/cgi-bin/cmd.cgi?cmd_typ=3... ]

Important caveats:

• If you’re not careful, by adding these filters you will have introduced a trivial local shell injection attack to your mail client. Validate your inputs — just like I didn’t above!
• The developers have this to note about running filters on every message:

  Testing for the trigger and invoking the filter doesn’t come for free. There is overhead associated with searching for the trigger string, testing for the filter’s existence and actually piping the text through the filter. The impact can be reduced if the Trigger Modifying Tokens [...] are employed.

  I’ve certainly noticed a small (immeasurable, but noticeable) delay in opening messages with triggers. Large enough to be annoying if I’d planned to filter every message, even using a trivial bash filter which itself is quick to complete.

• One additional caveat on DICE: if your alpine session outlives your AFS credentials, and you’ve stored your display filters in your home directory, you will find that the display filters simply disappear. As good a reminder as any to renew, and thankfully a “renc” is all that’s required to restore your filters to former glory.

That’s it! Surprisingly trivial, and with a handful of these triggers, the benefits are huge. I’m using five so far, mostly generating clickable links to some of our automated systems, but I’d be pleased to hear what other people are doing with these filters.

Editing LCFG component source files using Vim is of course The Right Thing to do, but due to the way these source files are named (typically filename.ext.cin) vim doesn’t necessarily pick up on the filetype, and goodies such as syntax highlighting are lost.

This is easy to fix using vim’s ftdetect system. Some examples for simple types:

" These files are always POD in disguise
au BufRead,BufNewFile *.pod.cin : set filetype=pod
" Slightly contentious: a new filetype is needed, really, but this is a decent match.
au BufRead,BufNewFile *.def.cin : set filetype=cpp
" For other, unknown types, detect from the as-yet undefined shebang:
au BufRead,BufNewFile *.cin : if getline(1) =~ '^#!@SHELL@' | set filetype=sh | endif
au BufRead,BufNewFile *.cin : if getline(1) =~ '^#!@PERL@' | set filetype=perl | endif

(note the latter two lines are specified separately, rather than elseifed, purely for readability). It’s fairly obvious that this can be extended to any file type, and there’s also scope for adding an automatic mapping to allow all files of form file.typ.cin to be mapped automatically to their default .typ. “sub-extension” file type.

Anyway, the above has already improved my productivity no end so I’ll leave the latter exercise to the reader. Comments and contributions are welcome, as always — so long as they’re not suggestions to use Emacs(!)

This is a quickie script which streamlines my RPM building and submission to a single command. Note that this is entirely dependent on our shiny new Package Forge system, which feeds RPMs to multiple platforms for building and eventual submission into our RPM buckets.

All it does is chain up “rpmbuild -bs [spec]; pkgforge submit [srpm]” but it’s a nice timesaver nonetheless. Side-benefits include the automatic generation of a readable ID and provision of a tracking link for pkgforge so that you can anxiously refresh the page to watch the build progress (or you could just wait for the report email…).

So, here is is; my very simple and stupid RPM automation. Suggested name: ‘rpmbus’.

#!/bin/bash
if [[ -z $2 ]]; then
    echo "RPMbus: build -> submit assist"
    echo "Usage: `basename $0`   [pkgforge args]"
    exit 1
fi
bucket=$1; shift
spec=$1; shift
args=$*

output=`rpmbuild -bs ${spec} | tail -n 1`
pkg=`echo ${output} | sed -e 's_^Wrote: __'`

if [[ ! -e ${pkg} ]]; then
    echo "Package wasn't built: ${output}"
    exit 1
fi

id=`basename ${spec} | sed -e 's_\.spec__' -e 's_\.__g'`-`date +"%s"`
echo -e "Found source package:\n  ${pkg}"
echo "  Extra args: ${args:-none}"

read -p "Submit to '${bucket}'?" foo
if [[ ${foo} != 'y' ]]; then
    echo "Cancelled"
    exit 1
fi

echo "Submitting to ${bucket}..."
pkgforge submit --id ${id} -B ${bucket} ${args} ${pkg} && \
echo "  https://pkgforge.inf.ed.ac.uk/job/view?id=${id}"

Caveats: well, they’re numerous and they’re pretty apparent. But it took five minutes to write and it WFM :)

As much as MacOS’ Spotlight is an integral and indispensable part of my interaction with my laptop, a part of me still begrudges the “gratuitous” CPU and disk utilisation which is of course a necessary part of its operation.

However as a hardened Linux user unprepared to do without the luxury of the locate database, my inner resource miser was further upset on discovering that these databases were not shared, and with even more irritation enabled the periodic updatedb cron job, as was suggested by locate itself.

Whether it was SSD envy, a nagging sense of a job half-done or sheer procrastination I’m not sure, but last week I felt compelled to do away with the needless platter-spinning and found the answer far too quickly, in the form of the Spotlight shell utility mdfind.

One alias later:

alias locate="mdfind -name"

and I was able to destroy the locate database, and discontinue its indexing:

launchctl stop /System/Library/LaunchDaemons/com.apple.locate.plist
launchctl unload -w /System/Library/LaunchDaemons/com.apple.locate.plist

whew!

Some obligatory qualifications…

This applies largely to OS X 10.6, Snow Leopard. Leopard’s arrangement is slightly different, and I know nothing about earlier versions… And no, it doesn’t support all of locate’s arguments, but I rarely used them (apart from -i) anyway (and don’t have any case-sensitive, indexed filesystems on the mac right now). man mdfind provides workarounds for many of the more unusual uses of locate, and grep provides the rest…

Update: The landscape on OS X has changed since this post was written. Hugh Cole Baker provided in a comment an excellent mechanism for setting self-managed policy on OS X which beats my clunky wrapper; Lion’s Kerberos support has changed in a way which prevents SPNEGO working for our Cosign servers (though a fix at our end is planned); Chrome on Linux gained a proper managed configuration, which we use locally (I produced the lcfg-chrome component for this purpose).

I was most impressed by the efficient conclusion to the enhancement request for SPNEGO on Chrome, but having read that the request had been met, I struggled for far too long to discover how to activate it.

Irritated by Firefox 4 beta 7′s breakage of SPNEGO on the Mac*, but reluctant to revert 3.6, I felt it was time to reinvestigate the alleged Chrome support Read the rest of this entry »

Nagios is an extremely useful tool, until it isn’t.  Which is to say, it’s nothing but a hindrance to have nagios continue to bombard you with IMs and emails when you’re already working on the problem.

Surely you can just acknowledge the fault and shut it up…?

Well, sometimes, but it is hardly convenient to break out a Firefox session when you’re attached to a serial console with your lovely secure-shell-enabled phone.  And even if you are on a DICE machine it’s a bit of a pain to have to navigate the slightly clunky Nagios UI to find the host and service you wish to silence.

I started with a dumb bash script. Read the rest of this entry »