Chrome and SPNEGO

5 Replies

Update, Aug 2015: The landscape on OS X has changed several times since this post was written. Chrome on the Mac now fully supports the “defaults” mechanism to set policy defaults. Chrome on Linux gained a proper managed configuration, which we use locally (I produced the lcfg-chrome component for this purpose).

Quick guide to configuring SPNEGO on the Mac:


$ defaults write com.google.Chrome AuthServerWhitelist <cosign.server.tld>
$ defaults write com.google.Chrome AuthNegotiateDelegateWhitelist <cosign.server.tld>

Restart Chrome and rejoice. What follows is probably only of historical interest…

I was most impressed by the efficient conclusion to the enhancement request for SPNEGO on Chrome, but having read that the request had been met, I struggled for far too long to discover how to activate it.

Irritated by Firefox 4 beta 7’s breakage of SPNEGO on the Mac*, but reluctant to revert 3.6, I felt it was time to reinvestigate the alleged Chrome support Continue reading →

Nag nag nag nag nagios

2 Replies

Nagios is an extremely useful tool, until it isn’t.  Which is to say, it’s nothing but a hindrance to have nagios continue to bombard you with IMs and emails when you’re already working on the problem.

Surely you can just acknowledge the fault and shut it up…?

Well, sometimes, but it is hardly convenient to break out a Firefox session when you’re attached to a serial console with your lovely secure-shell-enabled phone.  And even if you are on a DICE machine it’s a bit of a pain to have to navigate the slightly clunky Nagios UI to find the host and service you wish to silence.

I started with a dumb bash script. Continue reading →

homing pidgin

Leave a reply

Just another bit of shell glue which took about twenty minutes but yielded lovely results as it occurred to me that the DICE-wide inventory tools can now locate (at least in theory) any machine.

Being able to find out what office I’m in is more useful a feature than you might think.  Primary of those is the ability to advertise my whereabouts to colleagues, for example on entering a server room, in case I can be of button-pushing assistance to others.  Whenever I move around, I make an effort to update my Jabber status to point this out.

In fact, the glue was very straightforward and I learned about a particularly useful new tool: the Python DBUS libraries.  DBUS is the message bus adopted by most modern “freedesktop”-compatible environments, and the Python library provides a quick and easy way onto the bus.

First, I hacked together a tiny script to establish where I am. Continue reading →

tar-a-dice?

Leave a reply

I recently encountered a .tar file which DICE tar wouldn’t extract, giving multiple warnings:

tar:  Ignoring unknown extended header keyword `SCHILY.[...]'

before bailing out due to previous “errors”. The file in question appeared to have been created on a Mac, whose archive utility adds extended metadata to its files.


This is confirmed and fixed in May 2007, but some more venerable Linux distributions continue to ship older versions.  Tar’s behaviour is a bug, since these warnings should be non-fatal.  However, the warnings can be avoided altogether by addition of an argument:

 --pax-option="delete=KEYWORD.*"

for each problematic keyword.

[Source: Tar mailing list]

Python Profiling

Leave a reply

Profiling isn’t often important for my python programs, but when a performance black hole appears out of nowhere it’s very useful for narrowing down the problem, even on smaller scripts.

(Recently I had cause to profile a script manipulating large-ish (<50Mb) files, whose performance had taken a turn for the worse. I had a suspicion that poor list manipulation was to blame, but the results spoke for themselves: 99% of CPU time was spent in the function list.pop(0)! Based on this I discovered PEP-290 which describes the collections.deque structure, significantly more efficient than a basic list — the replacement function, deque.popleft(), barely registers a percentage of overall execution time(!)

I could have discovered this using basic profiling, but whilst playing with the profiler I discovered a very straightforward, easy-to-read python profile visualiser, and thought it worth sharing. It doesn’t do anything very beautiful or complicated, but it’s very quick, provides a neat “squaremap” view of your code performance, and importantly Just Works. RunSnakeRun is its name.

Continue reading →

Updating Matlab Licences

Leave a reply

This is simpler than it used to be, but is a bit of a convoluted process, so I thought it worth detailing.

When our Matlab licences are renewed or added to by request, Informatics will receive a new licence file (apparently known as a “license file” to those living west of us). This will look something like:

# LicenseNo: 123456   HostID: 000000001122
INCREMENT MATLAB MLM 20 01-jan-[...] \
        VENDOR_STRING=[....] \
        DUP_GROUP=[...]
INCREMENT GADS_Toolbox MLM 20 01-jan- [...] \
        VENDOR_STRING=[...] \
        DUP_GROUP=[...]
[...]

Note the absence of SERVER and VENDOR lines: other FlexLM vendors provide these lines, but I’ll talk about this later. It’s worth noting that if either line is present in a Matlab licence file, it should be removed.

This file needs to be packed into our matlab-license rpm. Retrieve the latest SRPM (be careful, sometimes the dates are more instructive than the versions!) and install it to your RPM build directory. You’ll find a tarball along the lines of matlab-license.tgz — this has a directory structure resembling:

matlab-license/
matlab-license/roles/
matlab-license/roles/matlabclassroom/
matlab-license/roles/matlabclassroom/env.455.matlab
matlab-license/etc/
matlab-license/etc/teaching/
matlab-license/etc/teaching/license.dat
matlab-license/etc/research/
matlab-license/etc/research/license.dat
matlab-license/etc/research/other.dat
matlab-license/CoreFiles/
matlab-license/CoreFiles/env.450.matlab

Note that the licences are built from multiple files in each of the teaching/ and research/ directories. Let’s say we’re replacing the research licence, since this file is the more complex of the two.

$ cd /tmp/
$ tar -zxf ~/RPM/SOURCES/matlab-license.tgz
$ less matlab-license/etc/research/license.dat
# MATLAB license passcode file.
#
# This portion contains the main set of research licences
# LicenseNo: 123456   HostID: 000000001122
INCREMENT MATLAB MLM 20 01-jan-[...] \
        VENDOR_STRING=[....] \
        DUP_GROUP=[...]
INCREMENT GADS_Toolbox MLM 20 01-jan- [...] \
        VENDOR_STRING=[...] \
        DUP_GROUP=[...]
[...]
$ less matlab-license/etc/research/other.dat
#
# This portion contains Some Other licences
# LicenseNo: 123456   HostID: 000000001122
INCREMENT MATLAB MLM 21 01-jan-0000 11 3D20000[...] \
[...]

It’s important to note that these files might have come from different accounts and it is likely that we are only replacing one of the files — possibly none if this is a brand-new addition. If we’re replacing just the main Research licence, we need to leave the other files alone; if we’re adding a new licence, simply add a file to the research directory with any name you wish (so long as it ends .dat).

Paste your replacement licence INCREMENT lines into this file. Tgz the file back up and place it in SOURCES.

Once your new tarball is back in sources, you need to amend the specfile. Update any matlab versions amongst the %defines at the top of the file, and increment or reset the Release as appropriate. Build and submit as normal:

$ rpmbuild -bs SPECS/matlab-license.spec
$ pkgforge submit -B inf -p sl5,sl6 SRPMS/matlab-license-x.y-z.noarch.rpm

(Note that this will submit new matlab-license-teaching, matlab-license-research and other RPMs. If there are no major version changes and we’re simply modifying the research licence, it’s OK just to deploy the matlab-license-research RPM, but it’s good practice to submit them all, and certainly the SRPM.)

Continuing our assumption that we’ve just made a trivial change to the research licence (a new INCREMENT line for example), edit the dice/options/matlab-research-server.h and live/matlab-research-server.h headers to include the new RPM:

$ svn di core/include/dice/options/matlab-research-server.h
+!profile.packages               mEXTRA(+matlab-license-research-7.7_7.8-7.inf/noarch)

$ svn di live/include/live/matlab-research-server.h
+/* REMINDER: Remove this next Friday! */
+!profile.packages               mEXTRA(+matlab-license-research-7.7_7.8-7.inf/noarch)
[...]

Finally, if we need to lock down one of our new licences to a specific user, we can do this using the live header and the FlexLM component’s options-file support:

$svn di live/include/live/matlab-research-server.h
[...]
+/* RTxxxx: Reserve one Statistics Toolbox licence */
+!flexlm.options_research        mADD(gr1 re1)
+flexlm.option_research_gr1      USER_GROUP gr1 userone usertwo userthree
+flexlm.option_research_re1      RESERVE 1 "Statistics_Toolbox" USER_GROUP gr1

Clearly, this reserves one Statistics Toolbox licence for the given group of users.

Having made all these changes, commit to the repository. You can push the required changes to the FlexLM server without restarting: the FlexLM component will perform a reload on picking up the options-file changes, though these may not make sense until updaterpms is run so the recommended intervention is as follows:

[matlab-research]$ om updaterpms run
[INFO] updaterpms: 1 installs, 0 removals
[INFO] updaterpms: Flagging matlab-license-research-7.7_7.8-7.inf/noarch for upgrading to
[WAIT] updaterpms: Checking dependencies
[INFO] updaterpms: (1/1) upgrading to matlab-license-research-7.7_7.8-7.inf/noarch

[matlab-research]$ om flexlm configure
om flexlm configure
[INFO] flexlm: Reconfiguring /usr/lib/lcfg/conf/flexlm/research
[INFO] flexlm: Reloading flexlm configuration for research (could take a minute)...
[OK] flexlm: configure

You may check the result of your actions by examining the Flexlm web-status page, which for the Research matlab licence is http://matlab-research:1881/usage-research.html. This should detail your new licences in the upper section, and any reservations alongside the list of ‘in-use’ licences in the lower section.

Be aware: if you make a mistake with INCLUDE, EXCLUDE and RESERVE lines and have to change them, reservations can end up becoming locked. A restart of the FlexLM server will clear this but it’s worth knowing in case you’re wondering where a licence went!

For more information on configuring FlexLM, see the relevant RAT Unit documentation pages for FlexLM and Matlab, and also the lcfg-flexlm manual page (the definitive reference for configuring a lcfg-flexlm server).

UPDATED 15/03/2012 to reflect the updated RPM procedure where licences are automatically constructed from their constituent parts in etc/{research,teaching}.

Real Mac UK Keyboard Layout

11 Replies

Edit: Oct 2013

Rejoice! OS X 10.9 “Mavericks” now provides a keyboard layout called “British – PC” which faithfully maps a real UK keyboard (so far as five minutes’ examination has shown) onto the Mac (so long as caps lock is not engaged).

Original article (for OS X upto 10.8 “Mountain Lion”):

I hate the Apple UK keyboard layout. I detest it. Defensible as it may be to enhance cross-Atlantic keyboard familiarity, and rational as it might be to place double-quote above single, I still cannot stand it. If I were a ‘switcher’, and had decreed that, from last March, only Apple computers were sufficiently worthy to be graced by my fingertips, I might have come to live with it but I object to having to remap my brain-finger pathways every time I move from one platform to another (remembering which clipboard / paste buffer to use is struggle enough).

All this is an elaborate way of saying that I’ve made a set of truly UK-compatible keyboard layouts for my MacBook Pro and the standard UK USB keyboard I sometimes plug into it. These layouts work for 10.5 “Leopard” and possibly others. They can be found by in my RealUKKeyboardLayouts.zip file.

In this zip file you’ll find the two layouts and two similarly-named .icns files which allow you to identify the layouts at a glance. They’re not very pretty but they do the job.

Unzip the archive and, for each user who wants to “type proper” again, place the files in ~/Library/Keyboard Layouts/ (creating the directory if required). Log out and back in to allow Mac OS to discover the new layouts (supposedly /Library/Keyboard Layouts/ can be used for system-wide layouts, but this was not the case on my first attempt and, as I’m the only user of my laptop who cares about such things, I didn’t delve further).

Now open the “International” Preference Pane. On the “input menu” tab, check “Show input menu in menu bar”. You’ll need this because, just occasionally, Leopard will switch you back to a system ‘blessed’ layout and you’ll spend hours cursing your mistaken “@”s until you figure out what’s happened. Now scroll through the keyboard layouts and check “Real UK” and “Real UK – IBM/PC” to allow these layouts to be selected from the input menu.

Close the preference pane, select the appropriate layout from the menu bar, and that’s it.

Oh yes: in case you were wondering, these layouts were created with the eccentric but indispensible Ukelele which guides you through the entire process. This will be particularly useful if you want a layout for a keyboard other than my provided MacBook Pro or IBM/PC layouts.

monitoring postgresql

2 Replies

This should serve as yet another account of “naïve” addition of monitoring to an LCFG component.  It proved to be surprisingly straightforward: PostgreSQL on my test server is now being monitored; built-in support is now available on any host:

#include <dice/options/postgresql-monitoring.h>

though take care to follow the instructions within.

Continue reading →

“tailcfg”

2 Replies

Since I first became fed up with tunnelling through to the same machine just to tail the same LCFG server log (about the third or fourth time I had to type the tiresome sequence of commands, a good few years ago) I’ve had a small function which does it for me.  However, for some reason I’d never thought to take advantage of the fact that all the logs end up tailed through my script for my manipulation, until last week.

The result of a little ANSI manipulation is a startling improvement in readability!

$ tailcfg
04/06/09 10:23:52:    publishing deferred: foo.inf.ed.ac.uk (XML)
04/06/09 10:23:52:    publishing deferred: quux.inf.ed.ac.uk (XML)
04/06/09 10:34:13:    processing: myhost [1/1, pass 1]
04/06/09 10:34:15:                1 error(s), 0 warning(s) (XML published)
04/06/09 10:34:15:    processing: netsrva [1/8, pass 2]
04/06/09 10:34:15:                (dhcp/all.map changed by myhost)
04/06/09 10:34:23:                0 error(s), 0 warning(s) (XML deferred)
04/06/09 10:34:23:    processing: netsrvb [2/8, pass 2]
04/06/09 10:34:23:                (dhcp/all.map changed by myhost)
[...]

It’s of course straightforward to achieve this. Why it took me so long to think to do it, I’m not sure.

function tailcfg() {
    ssh lcfg1 "tail -n 100 -f /var/lcfg/log/server" | sed \
        -e "s_\([1-9][0-9]* error(s)\)_`c red`\1`c off`_" \
        -e "s_\([1-9][0-9]* warning(s)\)_`c yellow`\1`c off`_" \
        -e "s_\([^1-9]0 \(error\|warning\)(s)\)_`c green`\1`c off`_g" \
        -e "s_\(processing: \)\(\<\w\+\>\)_\1`c bd`\2`c off`_"
}

Note the use of the `c` application; mine is a small binary whose source I’ve lost, but you can just as easily recreate this with a small script (I’ve left out the unused colours to save time / space…):

function c() {
    case $1 in
        red)    code='33[0;31m' ;;
        green)  code='33[0;32m' ;;
        yellow) code='33[0;33m' ;;
        bd)     code='33[1m' ;;
        off)    code='33[00m' ;;
        *)      echo "Invalid colour / state." >&2 ;;
    esac
    [[ -z ${code} ]] && return 1
    echo -e $code >&1 # note forced output to fd1.
}

My Favourite VirtualBox Settings (as of 2.2.2)

Leave a reply

VirtualBox improves with every release, and is now seems to be at a stage where it can be more than a mere curiosity for DICE users, as proven by our LCFG release test VMs, the ongoing Desktop Virtualisation project and my use of a VM as an emergency standin for failed server hardware.

Configuring the virtual machines is a process of trial-and-error, and not all settings suit all guests.  However as of version 2.2.2, this is my annotated list of my current ‘best’ DICE-on-DICE configuration:

Basic Settings:

OS Type: Linux -> Red Hat (32/64-bit)
Base Memory: (>= 384Mb ample for low-load commodity servers, but >= 1Gb apparently required for 64-bit guests)
Video Memory: >= 8Mb (for full-screen X on a DICE Host)
Boot Order: Hard Disk (with PXE on-demand only)
ACPI: Enabled (allows safe unattended shutdown, if nothing else!)
IO APIC: Disabled (causes massive clock skew if enabled)
VT-x/AMD-V: Enabled (provides some performance improvement, if mainly to host)
Nested Paging: Disabled (not yet supported on our hardware, but no harm in enabling)
PAE/NX: Enabled (of course PAE is not applicable on 64-bit, but there’s no reason to disable it on either arch)
3D Acceleration: Disabled (but worth enabling if you are using the Guest as a desktop, with Guest Additions)

Sound:

The SoundBlaster 16 device is detected, and can be used, by a DICE guest.

Note that any sound device will cause problems if you plan on running your guest ‘headless’.

Network:

Both Intel Pro (Desktop) and PCNet-Fast III devices work, providing 1Gbit and 100MBit network connections, respectively.

I’ve noticed that on Optiplex 755 hardware (on either virtual device), for each PXE boot request made by a guest, two requests are sent by the host: the first with the hosts’s MAC, and the second with the guest’s.  Clearly not ideal but I’ve not made any more detailed investigation into this: my DICE-install PXE boots are necessarily attended, and therefore easily restarted on failure.

Details on the specifics of bridged-mode network configuration can be found on the official DICE VirtualBox wiki page.

Hard Disk:

The standard ATA controller works very well. DICE + SATA will be tested soon.

Expect me to change my mind over time, not least because I note that 2.2.4 is already available…

Wine, and restoring some meaning to EXE.

Leave a reply

As the platform becomes increasingly stable, Wine apps are beginning to look like first-class citizens of an X environment. It makes sense that they should be executed as such:
# echo ':DOSWin:M::MZ::/usr/bin/wine:' > /proc/sys/fs/binfmt_misc/register

This allows DOS and Windows executables with magic number MZ to be executed as regular binaries, by passing them to the standard wine binary. Note that this does not require the file to be named '.exe' so, as ever, take care with +x!

or slightly more permanently, in very hacky LCFG:

!cron.additions mADD(winexe)
cron.owner_winexe root
cron.add_winexe @reboot "echo ':DOSWin:M::MZ::/usr/bin/wine:' > /proc/sys/fs/binfmt_misc/register"

(In theory we should be able to user the ‘kernel’ component, but it doesn’t seem to have the ability to manipulate the ‘register’ portion of fs.binfmt_misc.)