Userfriendly it ain’t

March 17, 2009

It has long been accepted wisdom that these mouse-driven office “productivity” applications are in some way intrinsically more userfriendly than the command line or text file driven applications we are more accustomed to in Unix. Today I had the opportunity to prove to myself that this just isn’t the case.

I had a file containing two columns of data, first column is a date, the second column is a number. All I wanted to do was convert this into a nice graph with the dates on the X-axis and the numbers on the Y-axis. It seemed to me that this is exactly why spreadsheet applications like those provided in openoffice and gnumeric were designed. When it comes to office applications I am a novice but I do expect them to be usable with a bit of common sense and intuition. But, oh no, no matter what I tried I just could not get it to do what I wanted. First I had a huge battle to persuade it that the date column contained dates. Then I couldn’t get the chart style I wanted, once I decided to opt for what it thought best I then couldn’t easily label the axes, then I couldn’t easily add a title. After a long while of cursing and mouse-clicking I had a chart which more or less resembled what I wanted but could I print it?? Nope. It was spread horizontally across two pages so would not print out on one a4 sheet without a rotation but there was no obvious way to do a rotation. At this point I gave up…

A long time ago in a career far, far away I was an astronomer. You might think that most of what astronomers do involves looking through telescopes. The reality is a bit more dull, they spend their days sifting and analysing enormous amounts of data looking for the proverbial needle in a haystack. In those days the application of choice for plotting graphs was Super Mongo (or SM for short). This did a good job but it was a pig to use so I got into using a newer project named gnuplot, it was a bit limited in some places but the interface was so much nicer than SM it was generally the tool I used. Knowing what many astronomers are like they are probably still battling on with SM rather than try anything different and new but then they like using Fortran as well…

I’ve not used gnuplot in years so I had forgotten anything I ever knew about the syntax. However, having given up on the not-so-friendly spreadsheet I decided to give gnuplot a try. 5 minutes with a tutorial from IBM and the gnuplot manual and I had exactly the graph I had originally wanted with no cursing and swearing involved. Here’s the whole script:

set terminal png         # gnuplot recommends setting terminal before output
set output "report.png"  # The output filename; to be set after setting

set xlabel "Week"
set xdata time
set timefmt "%d/%m/%Y"
set format x "%d/%m/%y"
set xrange ["01/11/2007":"31/03/2009"]

set ylabel "Hours"
set yrange [0:25]

set boxwidth 3

plot "report.txt" using 1:2  with boxes lw 3 title 'Build Tools Project Effort'

It could be a lot simpler depending on the data, some of that gubbins is associated with making it parse and print dates correctly. This is stored in a file (e.g. report.gnuplot) and then passed to gnuplot on the command line:

$ gnuplot report.gnuplot

You now have a file named report.png. gnuplot also has a very nice interactive interface where you can type in your commands and try out stuff. It has also gained many features over the years and has some impressive abilities, it will also save to pretty much any file format you like which is perfect for inclusion in LaTeX docs or using in web pages. What’s not to like? It’s simple, straight forward and well documented. It’ll be a long time before I make another attempt to use a spreadsheet.


LCFG Updates

March 13, 2009

It’s been a very busy week for updates to the LCFG web services. We have finally moved to svn over webdav for our headers & package lists repository and we now have the start of a move from CVS to subversion for the source code repository. All this means that we can finally involve external contributors, we can allow them read and write access to any of our projects and we can host their projects so that everything is in one central location. Accompanying this is the opening of access to the LCFG bug tracker so that external contributors can file bugs and manage bugs on their own projects. All this is possible due to the wonders of cosign and iFriend.


Splitting lcfg-utils

March 4, 2009

Recently I have been working on converting the MPU managed LCFG components to being built via the new build tools. I’m down to the very last few now, one that I have been avoiding for ages is lcfg-utils since the current package build scripts are a bit crazy, it is all in a bit of a tangled mess. Yesterday I finally bit the bullet and started ripping it apart with the aim of separating it out into related units – one for the C code, one for the core Perl utilities and one for the Perl package utilities. Along the way I also had in mind enabling the building of a shared library for lcfgutils and a few other niceties.

I was pleased to find that the new build tools really did make the job much easier than I had expected. The two packages of Perl code, LCFG::Utils and LCFG::PkgUtils use Module::Build so could be uploaded to CPAN. The newly pared down lcfg-utils package provides the shared library and a few basic utilities uses CMake. There is also an lcfg-utils-devel package on Redhat systems which holds the header file and static library for any build processes where that is required.

I now have it all nicely organised and ready for testing. I believe it all works, it certainly appears to on my machine but it will need further testing to check that I haven’t introduced any nasty bugs. These are fairly important libraries and utilities so a certain amount of cautious checking is required. If you want to give it a go you can do so with the following lines added to an LCFG source profile:

!profile.packages       mEXTRA(+lcfg-utils-1.3.0-1\
                                                 lcfg-utils-devel-1.3.0-1\
                                                 perl-LCFG-PkgUtils-1.0.0-1\
                                                 perl-LCFG-Utils-1.0.2-1)

If you are feeling really brave you can also try out a new version of updaterpms which uses the lcfgutils shared library, you just need:

!profile.packages  mEXTRA(+updaterpms-3.1.5-1)

UKUUG Advanced Perl Workshop

March 2, 2009

Last Thursday I was in London to attend an "Advanced Perl Techniques" workshop organised by the UKUUG. The tutor was Dave Cross, who has written a couple of Perl books. He has a good style of delivery, he was generally very knowledgeable, the presentation was well structured and amazingly it all ran to time (that takes real talent). Given the title and the list of topics I had high hopes of learning some really cool new things to do with Perl. Here’s the list of subjects which were covered:

  • What’s new in Perl 5.10
  • Dates and times
  • Testing (including coverage analysis)
  • Database access with DBIx::Class
  • Profiling
  • Object oriented programming with Moose
  • Templates
  • Web application development with Catalyst

Specifically, I wanted to learn more about DBIx::Class and Catalyst and find out whether I am using Moose in the right/expected way. I guess, looking at that list again now, I should have realised that it is a lot to get through in one day and necessarily it was only going to be a shallow coverage of each topic. Other than the Catalyst stuff at the end I thought it was all pretty good (if lacking in the really deep detail I wanted) and I did get some useful bits and pieces from the day. I felt the Catalyst section was done very lazily though, it had the feeling of being added as an after-thought and I wondered if it was actually just copied from the standard Catalyst documentation.

I was interested to learn that "DateTime" is considered the "best" module to be using for all time and date manipulation. It certainly has capabilities way beyond that which I was previously aware. I also found the profiling section interesting, I will definitely be looking at "Devel::NYTProf" in more detail sometime soon. The "What’s new in Perl 5.10" section was also particularly good and has encouraged me to start looking at the new features in more detail and, at least, start using them in any scripts I write for personal use. It’s a shame we won’t see 5.10 in RHEL5 but that’s the price we pay for system stability. By the time we get RHEL6 it will at least have had any bugs found and fixed by users of Fedora, Debian and Ubuntu.

All in all, it was worth going to the workshop. At some point in the future I’d love to see a "Really Advanced Perl" workshop which really goes beyond the beginners guide for DBIx::Class, Moose and Catalyst and demonstrates some of the more complex possibilities.