Category Archives: howto

a big hand for rfe

Here are some tools that I’ve put together that make a huge difference to a clunky rfe workflow. Bored already? Don’t worry, I’ve written the script already: tl;dr

Imagine power cycling a DICE server with redundant PSUs, using our lovely power bar control files. You don’t know where it’s installed, so you have to search for it. Continue reading

Editing component files with vim

Editing LCFG component source files using Vim is of course The Right Thing to do, but due to the way these source files are named (typically filename.ext.cin) vim doesn’t necessarily pick up on the filetype, and goodies such as syntax highlighting are lost.

This is easy to fix using vim’s ftdetect system. Some examples for simple types:

" These files are always POD in disguise
au BufRead,BufNewFile *.pod.cin : set filetype=pod
" Slightly contentious: a new filetype is needed, really, but this is a decent match.
au BufRead,BufNewFile *.def.cin : set filetype=cpp
" For other, unknown types, detect from the as-yet undefined shebang:
au BufRead,BufNewFile *.cin : if getline(1) =~ '^#!@SHELL@' | set filetype=sh | endif
au BufRead,BufNewFile *.cin : if getline(1) =~ '^#!@PERL@' | set filetype=perl | endif

(note the latter two lines are specified separately, rather than elseifed, purely for readability). It’s fairly obvious that this can be extended to any file type, and there’s also scope for adding an automatic mapping to allow all files of form file.typ.cin to be mapped automatically to their default .typ. “sub-extension” file type.

Anyway, the above has already improved my productivity no end so I’ll leave the latter exercise to the reader. Comments and contributions are welcome, as always — so long as they’re not suggestions to use Emacs(!)

Away with the PXEs

Occasionally, for the purposes of internal testing or continuity, it’s desirable to bring up a server with a duplicate MAC address. It’s a safe enough manoeuvre (so long as these machines operate on different wires) for the brief periods in which I require it but when this scenario involves the installation of a new server via our installroot PXE service, things are trickier.

Our PXE server is configured automagically by spanning map and, effectively, keyed on MAC, so it’s unlikely to present the correct configuration (reliably) when the new host differs from the old one in some way.

The workaround is to override the PXE configuration on the *existing* server (on the basis that you weren’t planning on reinstalling it, anyway, were you?):

!pxeclient.platforms mADD(new_plat_name) /* e.g. sl6_64 */

/* And, if you need to add or remove serial console support: */
!pxeclient.serial_port mSET(ttyS0) /* or () */

Post-PXE, the dhclient component is aware of subnet differences and will ensure your machine receives the correct profile for installation (though, to prevent future confusion, remove this as soon as the installer has done its work!).

losing locate

As much as MacOS’ Spotlight is an integral and indispensable part of my interaction with my laptop, a part of me still begrudges the “gratuitous” CPU and disk utilisation which is of course a necessary part of its operation.

However as a hardened Linux user unprepared to do without the luxury of the locate database, my inner resource miser was further upset on discovering that these databases were not shared, and with even more irritation enabled the periodic updatedb cron job, as was suggested by locate itself.

Whether it was SSD envy, a nagging sense of a job half-done or sheer procrastination I’m not sure, but last week I felt compelled to do away with the needless platter-spinning and found the answer far too quickly, in the form of the Spotlight shell utility mdfind.

One alias later:

alias locate="mdfind -name"

and I was able to destroy the locate database, and discontinue its indexing:

launchctl stop /System/Library/LaunchDaemons/com.apple.locate.plist
launchctl unload -w /System/Library/LaunchDaemons/com.apple.locate.plist

whew!

Some obligatory qualifications…

This applies largely to OS X 10.6, Snow Leopard. Leopard’s arrangement is slightly different, and I know nothing about earlier versions… And no, it doesn’t support all of locate’s arguments, but I rarely used them (apart from -i) anyway (and don’t have any case-sensitive, indexed filesystems on the mac right now). man mdfind provides workarounds for many of the more unusual uses of locate, and grep provides the rest…

something I didn’t know about mailcap

For a few weeks I’ve been idly wondering why I’ve been unable to get alpine to take advantage of the syntax-highlighted goodness of vim, when viewing attached patches. Having just won another small victory against my own ignorance, I thought it best to share.

Like any sensible mail client, alpine chooses viewers for attached files using lookups of the system mailcap files, /etc/mailcap and ~/.mailcap. Enabling plain-text viewing in vim should be as simple as assigning vim to the appropriate type(s) in ~/.mailcap (and, for some types, unchecking the alpine Show Plain Text Internally preference).

However, attempts to open plain-text files (in this case specifically text/x-patch) in the multi-talented vim editor failed: alpine simply returned a “finished” status, as if viewing had been successful. My suspicion was confirmed when I redirected vim’s ouptut (hidden by alpine) to a file:

Vim: Warning: Output is not to a terminal
Vim: Warning: Input is not from a terminal

The latter message was well known to me; it’s usually triggered by my forgetting to affix the “stdin hyphen” whilst piping input to vim.

The problem is that both vim and alpine require control of the terminal to function; vim does not simply return beautifully ANSI-escaped coloured text for later display. Attempts to somehow force alpine to relinquish control of the terminal, or for vim to take it, failed until I discovered the secret amongst mailcap’s flags, as described by the manual:

    copiousoutput
        This flag should be given whenever the interpreter is
        capable of producing more than a few lines of output
        on stdout, and does no interaction with the user. [...]

I’d seen this, but for some reason had always assumed ‘copiousoutput’ to be some sort of magic external pager, with no connection to the mailcap system. Reading on, the solution was clear:

    needsterminal
        If this flag is given, the named interpreter needs to
        interact with the user on a terminal. [...]

So, a few amendments to ~/.mailcap later:

  Text/X-Patch;    /usr/bin/vim -R -- '%s'; needsterminal

and alpine had gained magical powers to invoke terminal-based viewers. There’s more to this; in particular the ‘edit=‘ and ‘compose=‘ fields, not to mention print support. But that’s enough to get basic viewing in vim.

+1 for reading the manual. -1 for not reading it before embarking on terminal manipulation…

Chrome and SPNEGO

Update, Aug 2015: The landscape on OS X has changed several times since this post was written. Chrome on the Mac now fully supports the “defaults” mechanism to set policy defaults. Chrome on Linux gained a proper managed configuration, which we use locally (I produced the lcfg-chrome component for this purpose).

Quick guide to configuring SPNEGO on the Mac:


$ defaults write com.google.Chrome AuthServerWhitelist <cosign.server.tld>
$ defaults write com.google.Chrome AuthNegotiateDelegateWhitelist <cosign.server.tld>

Restart Chrome and rejoice. What follows is probably only of historical interest…


I was most impressed by the efficient conclusion to the enhancement request for SPNEGO on Chrome, but having read that the request had been met, I struggled for far too long to discover how to activate it.

Irritated by Firefox 4 beta 7’s breakage of SPNEGO on the Mac*, but reluctant to revert 3.6, I felt it was time to reinvestigate the alleged Chrome support Continue reading

homing pidgin

Just another bit of shell glue which took about twenty minutes but yielded lovely results as it occurred to me that the DICE-wide inventory tools can now locate (at least in theory) any machine.

Being able to find out what office I’m in is more useful a feature than you might think.  Primary of those is the ability to advertise my whereabouts to colleagues, for example on entering a server room, in case I can be of button-pushing assistance to others.  Whenever I move around, I make an effort to update my Jabber status to point this out.

In fact, the glue was very straightforward and I learned about a particularly useful new tool: the Python DBUS libraries.  DBUS is the message bus adopted by most modern “freedesktop”-compatible environments, and the Python library provides a quick and easy way onto the bus.

First, I hacked together a tiny script to establish where I am. Continue reading

Python Profiling

Profiling isn’t often important for my python programs, but when a performance black hole appears out of nowhere it’s very useful for narrowing down the problem, even on smaller scripts.

(Recently I had cause to profile a script manipulating large-ish (<50Mb) files, whose performance had taken a turn for the worse. I had a suspicion that poor list manipulation was to blame, but the results spoke for themselves: 99% of CPU time was spent in the function list.pop(0)! Based on this I discovered PEP-290 which describes the collections.deque structure, significantly more efficient than a basic list — the replacement function, deque.popleft(), barely registers a percentage of overall execution time(!)

I could have discovered this using basic profiling, but whilst playing with the profiler I discovered a very straightforward, easy-to-read python profile visualiser, and thought it worth sharing. It doesn’t do anything very beautiful or complicated, but it’s very quick, provides a neat “squaremap” view of your code performance, and importantly Just Works. RunSnakeRun is its name.

Continue reading