Simon's Musings

March 27, 2008

Using packages from upstream

Filed under: Uncategorized — sxw @ 1:39 pm
Tags: , ,

One of the most time consuming things about building new software is working out whether a package has already been packaged ‘upstream’ in Fedora. Doing so is important, because if it’s available in EPEL we should be using that, and if it’s available in any version of Fedora it’s quite likely that that version can be ported to Scientific Linux with less effort than building a package from scratch. Besides, we can’t submit a package upstream if it’s already there šŸ™‚

Fedora has a couple of databases which store package information. The Fedora Package Database provides the definitive information on packages, along with which systems they have been built for. Unfortunately, the package database doesn’t currently seem to have a searchable interface, meaning that finding a package is a matter of appending likely names to the end of a URL. koji, the Fedora build farm, also provides packaging information – but only for those systems which koji builds for – it will not indicate whether packages are available within EPEL.

Fedora Package Database

To get information from this database, append the likely name of your package to For instance, if you’re interested in perl-HTML-Tree, then navigate to From this page, you can see which architectures the package should be available for. If ‘Fedora EPEL 5’ is listed, then run, don’t walk, to your nearest Extras repository.

If an EPEL package isn’t available, the clicking on the Build Status link will take you to the koji page for this package. Of which, more below.


Koji is the Fedora Project’s build system. It only currently only builds Fedora operating systems (EPEL is built using an older build system called plague). Koji provides a powerful search interface that lets you find packages, but it won’t tell you whether they exist in EPEL or not. koji can be found at

Once you’ve found the koji page for your package, clicking on a package in the ‘Builds’ section will let you download the SRPM which the build system spat out. From this SRPM, it is simply a matter of adding a ‘.1.inf’ to the Release field, and a suitable changelog comment, to produce an SL5 package suitable for our ‘world’ repository. I have a script which automates this step, and automatically builds and submits both i386 and x86_64 versions of the package.

Of course, in the long term we should be contacting the maintainers of those packages in Fedora, but not EPEL, and asking them if they’d mind us looking after an EPEL branch of their package. This would require more of us to be Fedora developers, though.

March 15, 2008

New apacheconf and monitoring thoughts

Filed under: Uncategorized — sxw @ 5:05 pm
Tags: , ,

Yesterday, I shipped a new apacheconf component, with some significant changes to its monitoring support.

Apache is a complicated beast, with many different mechanisms for configuring it. Apacheconf doesn’t necessarily handle all of these different options, and sometimes work arounds are necessary. For example, apache supports providing multiple ip:port combinations to a VirtualHost directive. Apacheconf only supports providing one. For this reason, Neil had configured a service with two VirtualHosts, both with the same server name. Unfortunately, apacheconf assumed that all of the server names would be unique on a given hosts, and so builds its Nagios service descriptions (which must be unique) based on these server names. Upshot of this is that we end up with a monitoring configuration that won’t load.

I’ve made two changes to help mitigate this. Firstly, every apacheconf virtualhost now has a
vhostnagiosmonitor directive, which can be set to false to disable monitoring for that virtual host. Secondly, the apacheconf translator now keeps a list of all of the service descriptions it has created, and adds uniquifiers to any duplicates (initially the IP address and, if that isn’t sufficient, a number).

In addition to this, a new lcfg-monitor has shipped containing a number of bug fixes.

In the long run, we need to give lcfg-monitor the ability to take a list of machines and components for which monitoring is disabled – so that, if this happens again, we don’t end up having to rush to fix broken configurations, or components, just to keep monitoring running for everyone else.

March 14, 2008


Filed under: Uncategorized — sxw @ 9:56 am
Tags: , ,

I’ve been experimenting with Mercurial, as a means of streamlining the way I work with the OpenAFSĀ  CVS repository. In particular, I’m trying to improve the management of my disconnected operation code, as well as better controlling the large number of patches I’m producing as part of the prototyping and error removal exercise.

Because I tend to flit backwards and forwards between different pieces of code, I tend to find that with CVS I have a large number of different checked out sandboxes. For big projects, such as the disconnection work, there’s no history, or ability to revert changes without taking a snapshot of the sandbox, which is both time consuming and inefficient. For smaller projects, there’s either a huge number of different sandboxes (and the related ‘where did I do X?’ problem), or lots of code ends up being intermingled within the same sandbox, and lots of things have to be unpicked before patches can be sent upstream.

These actually end up being two different problems, and it looks like there are two different mercurial workflows that are best suited to handle them. For disconnection, what I really need is a way of tracking, and managing, my code changes, and I’m using mercurial as a normal SCM to achieve this. With the prototyping changes, what I really need are patch queues – I have a large number of changes which I’m trying to arrange into manageable chunks in order to send upstream. Depending on my testing schedule, I may have a large number of patches awaiting submission. In this case, mercurial’s patch queues seem like by far and a way the best fit.

I’m intending on making my mercurial repositories for both of these tasks publicly available. For now, I can offer a mercurial import of the OpenAFS ‘upstream’ CVS at Other repositories are likely to appear there over time.

Local users may be interested to note that they can get mercurial on a DICE machine by including dice/options/mercurial.h in their profile.

Theme: Rubric.