Simon's Musings

March 27, 2009

AFS & Kerberos Best Practices Workshop

Filed under: Uncategorized — sxw @ 1:01 pm
Tags: , , , ,

Once again, I’m presenting at the AFS & Kerberos Best Practices Workshop. This years event is at Stanford University from June 1st-5th

I’m giving two talks, the first on prometheus, our new Identity Management System. The second is about how to contribute to OpenAFS. The abstracts are

Prometheus is an LDAP based provisioning system, which is designed to manage a wide variety of user databases, including AFS’s PTS and a Kerberos KDC. It is highly flexible in the databases it supports, and permits very fine grained delegation of control. It has a role-based access control model, and allows the creation and management of roles by any authorized user. It is instance aware, allowing users to create many instances of a primary account, request keytabs of those instances, and delegate particular permission sets to individual instances. Prometheus is designed to be a distributed as possible, permitting provisioning of system maintained by disparate groups without requiring those groups be trusted by the system itself. This talk will discuss the design goals behind Prometheus, provide an update on implementation progress, and demonstrate a running system.

and …

OpenAFS has a huge, daunting codebase, with a relatively opaque system of patch submission, review and application. It takes mountains of skill, and years of persistence to get your first patch into a state where it can be submitted, let alone accepted into the hallowed halls of the code tree…

Nonsense!

This talk will attempt to blow away some of the misconceptions with regards to contributing to OpenAFS. It will provide a first-timers view of the steps, both technical and political, to crafting a patch for submission into OpenAFS. We’ll take a whistle stop tour of the tools now involved in the process, from the code repository, to the patch review system and the bug tracker. We’ll talk about code review, bug triage and testing, with a view to inspiring participation in these areas.

Finally, we’ll talk about some low hanging fruit that anyone could get started on, and write their first bit of OpenAFS code …

In addition to keynotes from Morgan Stanley and Carnegie Mellon, the conference features a number of talks about research computing storage (including one from the nanoCmos project), and looks like it will have a great mixture of academic and commercial topics.

The hotel block (at the very reasonable Stanford Guest House) expires April 1st, with the early bird deadline being April 21st.

UKUUG Spring Conference

Filed under: Uncategorized — sxw @ 12:16 pm
Tags: , , , , ,

I’ve just returned from the spending 3 days in London at the UKUUG Spring Conference. I presented a Kerberos tutorial on the first day, and spent the following 2 as a conference delegate. The tutorial was well attended, with over 50 people there on the day, and seemed to go really well with a lot of good feedback from the attendees.

The second and third days were taken up with the conference proper. There seemed to be more delegates than in previous years, although the number of talks was smaller, with only one conference track. Whilst holding the conference in London obviously served to increase its appeal to those living locally, the venue wasn’t entirely ideal. Whilst the space for the talks was fine, there was a lack of break out and foyer space, making lunch and coffee breaks a scramble for space, and in depth conversations out of the conference hall harder.

The talks themselves covered a good mixture of topics, with security, LDAP and monitoring being particularly prevalent. The conference started with a presentation from Barry Scott of Centrify about integrating Unix boxes with Active Directory.  This gave a good overview of the situation (and said some nice things about the Kerberos tutorial), but talked more about their commercial product than what was possible with the available open source tools. From my perspective, this was a slightly missed opportunity, although the overview would have been of use to anyone contemplating that integration.

Later in the day,  Andrew Findlay gave a very strong and well presented talk on LDAP access control policies. (there is also a pdf paper) Whilst this continued the logical progression from what Andrew’s said about LDAP ACLs at previous conferences, it wrapped all of his current thinking up into a single, easily digestible block. It reconfirmed some of my design choices with prometheus, and challenged others. 

After lunch, there was a “Systems Monitoring Shootout“, comparing the features of various different systems monitoring packages. There were some really interesting ideas in here, including the use of NagiosGraph to produce rrd files which can then be used for trend and capacity analaysis. Following this, Jane Curry presented on ZenOss, a Zope based network monitoring tool. This appeared to be more network focussed than the service focus of Nagios, with lots of features like automatic device discovery and a very pretty looking interface. However, nothing that convinced me we should drop Nagios and use it instead. Finally in this session we had a very well presented skip through the … interesting … things you could do the the SCSI bus with sysfs, and the power of lvm in terms of disk management. 

In the final session of this day, Darren Moffat from Sun ran through some of the security features in Open Solaris. As well as a name check for my OpenSSH work, Darren talked about the new concept of role users, the move towards privileges in the kernel, and the additional RBAC work that’s in OpenSolaris. He also trailed the encryption features which will shortly be appearing in ZFS. All in all, a fascinating talk.

After Gavin Henry had talked about the replication strategies currently available in OpenLDAP, Howard Chu gave a great talk about its new MySQL NDB backend. Primarily developed with telco grade customers in mind, this allows you to share your database between MySQL and OpenLDAP, and take advantage of NDB’s clustering properties to linearly scale your load by simply adding more servers. The downside is that there are fixed constraints on attribute set size and tree depth. So, not a new general purpose backend, but a real insight into the large scale deployments that Symas is doing with OpenLDAP. I took the opportunity to quiz Howard about API stability for overlays – his answer unfortunately confirmed my view that the API isn’t stable enough to let us use them for prometheus.

Continuing the telco theme, Craig Gellen spoke about OpenNMS, a network management system which was designed from the ground up for large scale enterprise and telecommunications customers. Again, this system seems more network than systems monitoring focussed, and probably far too complex for our needs, but it was really interesting to see a piece of Open Source software which is specifically targeted at this market.

The final session started with a couple of virtualisation talks. Kris Buytaert talked about the current, and ever shifting, state of the Open Source virtualisation world, including a discussion of the current allegiances of the major vendors. Following this openQRM, an open source, virtual datacentre management tool, was presented. Matthias Rechenburg’s talk focussed in particular on cloud computing. OpenQRM has an automated provisioning model, where a user can use a web interface to request (and pay for!) a certain amount of time on a certain number of auto built virtual machines. The talk concluded with a demo that both worked, and held the audiences attention – no mean feat!

Alex Howells from Gradwell gave the final talk of the day – a tour of the major external security threats he’s become aware of during his time managing systems for Bytemark and Gradwell. This was a detailed look at the common security issues on today’s internet, as well as giving helpful advice on how to counter them. Whilst some things (for example using fail2ban on external facing services) would be easy to put into practice here, others (requiring code review for everything that runs on a web server) wouldn’t be appropriate to our environment. All in all though, this was a good talk, containing a lot of things to ponder, and a great way to end the conference.

Despite having a smaller set of talks than in the past, the technical content of the conference seemed stronger than it has been in the last couple of years. Having a single track did help to improve its focus, although the reduction in moving around, coupled with the lack of break out space did reduce the opportunities to interact with other delegates. The UKUUG are changing the focus of their Summer Conference (which has typically been Linux based) to encompass a very wide scope, some of which overlaps with the LISA focus of this event. I suspect its long term future remains to be seen.

All in all, though, I think the UKUUG Spring Conference is a very useful event to attend.

November 3, 2008

AFS Hackathon and Google Summer of Code

Filed under: Uncategorized — sxw @ 7:35 pm
Tags: , , , ,

I’m now back in Scotland, having spent the last week in California, courtesy of the very nice people at Google’s Open Source Program’s office and OpenAFS. As previously mentioned, I spent the summer mentoring a student (Dragos Tatulea) who was adding support for read-write disconnection to OpenAFS. The mentoring process was hugely rewarding – from a standing start Dragos learned a huge amount about a very complex codebase, and produced a workable implementation of disconnected operation which is now part of the OpenAFS tree. Whilst mentoring was both challenging and time consuming, it also encouraged me to rapidly learn about bits of the OpenAFS codebase I’d never delved into before, and lots about Linux kernel development that I’d been trying to avoid ever knowing!

So, Google invited OpenAFS to nominate people from their Summer of Code mentoring team to attend a summit at their Mountain View headquarters, and I was kindly included. Derrick, Jeff and Matt from OpenAFS also came along. My bags also eventually joined me!

The mentors summit its self was an eye-opening experience. Organised as an un-conference, where people were encouraged to arrange sessions on topics and technologies that interested them, there was a huge amount of fascinating information, and many useful relationships created and renewed. In particular, a chance demonstration at the session talking about Android introduced me to Gerrit, a web based code review tool. I firmly hope that gerrit will be part of the OpenAFS development process, just as soon as we get moved over to git.

Immediately following the Summer of Code mentor’s conference, Google hosted an AFS hackathon – a chance for a collection of OpenAFS developers to get together, discuss the current state of our world, and make targetted progress on specific items. Much of the discussion here centered upon moving forwards on a few specific areas – the move from CVS to git, the integration of rxk5 and Hartmut’s OSD work, and the ongoing work on forming an foundation, and creating a standardisation process. 

I also spent half a day looking at improving the AFS user experience on the Nokia n810. Unfortunately the Hildon file manager widget which both the n810 file browser and all native applications use has some features that make it particularly unfriendly for network file systems. Firstly, it does all of its processing in a single thread, so file system operations which block for a long time also hang the user interface of the application. Secondly, it’s not particularly aware of ‘expensive’ operations – for example, when you open a directory it will also open all of the sub directories, and work out how many files are in them by stating every file, in every sub directory. Needless to say the performance of this is very poor when the directory you are opening is /afs.

I also spent time on bringing up a test instance of gerrit, and working up some proposals of how this could be integrated into the OpenAFS patch workflow. Whilst this is still blocked on the work on the git migration (which Max and Mike made significant progress on over the 2 days), hopefully we’ll be in a position to start using it in anger soon.

Despite the best efforts of the fog at LAX, and American Airlines, I also made it back to Scotland!

April 3, 2008

UKUUG spring conference

Filed under: Uncategorized — sxw @ 11:04 am
Tags: , , , ,

I’ve just got back from the UKUUG Spring Conference, where a group of us from Informatics (myself, Stephen, Paul and Gihan from Flexiscale) were giving talks. I talked on two subjects – the LCFG based monitoring system framework I developed last year, and the new account management system I’m currently writing. Slides from both of these talks are available on the DICE publications page, which also has Stephen’s slides from his “An end to hacky scripts” talk about the LCFG system.

Despite gaining a scripting language track, and the addition of a parallel one-day PostgreSQL conference, the event seemed smaller this year, with many of the familiar faces missing. Some unfortunate scheduling meant that switching between tracks wasn’t as easy as it could have been, with 45 minute sessions in one room scheduled against 30 minute sessions in another one. However, the event was still productive, useful and stimulating, with a number of interesting talks – slides and audio from which should hopefully be up on the conference website shortly.

Some highlights were the talk from Mark Gledhill from the BBC on “Feeding the BBC Homepage“, which provided a fascinating insight into perl and Catalyst usage at a large organisation, as well as giving a useful background on their project management techniques, and test and deployment issues. Gavin Henry’s talk on OpenLDAP 2.4 provided a valuable summary of the changes in the latest version of OpenLDAP, as well as giving some examples of practical uses for these new features. Randy Appleton’s “Today’s Software … Is It Really Bloated?” talk took a very humorous tour through a number of code size and performance statistics he and his students have been collecting over the years – a perfect start to the day after the conference dinner!

The Transitive (which I ended up seeing because it was swapped with the talk I wanted to hear – one peril of last minute schedule changes!), and ZFS talks pretty much repeated material I’d heard at other conferences, but the ZFS one, in particular, was a helpful reminder of a system I’d really like to have time to look at in more detail. Whilst I wasn’t specifically interested in the scripting language talks, I did manage to catch “USENET Gems” which provided details of a number of interesting perl quirks, which are now firmly filed as things to watch out for.

Paul and Stephen arranged a well attended LCFG BOF on the Tuesday afternoon, and Paul, Stephen and I took some time to chat on Wednesday about possible designs for the new LCFG compiler. As with all UKUUG conferences, it tends to be these unscheduled events, and impromptu corridor conversations where the real value lies. There was a large amount of interest in prometheus, both from people in the commercial sector who have deployed similar systems, and had insights to share, and those who are interested in similar systems for their own sites. Hopefully we’ll be able to build some kind of a community around this technology.

There was continued interest in OpenAFS and Kerberos, with a number of people asking questions both about the technology, and our deployment experiences. Access to the source code for the monitoring system was also in demand – I really should arrange to publish this somewhere less adhoc.

February 26, 2008

FOSDEM 2008

Filed under: Uncategorized — sxw @ 2:45 pm
Tags: , , ,

Over the last weekend, I attended FOSDEM, an absolutely mind blowing conference bringing together Free and Open Source developers from all over Europe. The scale of the conference, attracting as it does thousands of developers, and accommodating hundreds of different talks over 2 manic days, really can’t be described. You have to be there to experience it.

I made the journey to Brussels by train, a most civilised way to travel – especially given that Eurostar are quite happy to replace lost return tickets for a small fee! The weekend started with the infamous beer event on the Friday night (hence the lost ticket), before getting down to business on the Saturday. It’s hard to pick particular highlights from such a packed program, but the perl6 talk managed to be both fascinating and scary at the same time and the cmake talk was very useful given the way Stephen is going with build tools. In the dev rooms, Dan Mosedale unfortunately didn’t make it for the Thunderbird talk, but an productive discussion was had none-the-less, and Jens Kuehnel’s introduction to SELinux in the Fedora devroom helped overcome a lot of my fears (and, in fact, has succeeded in its goal, as I no longer just switch it off). The sight of 100+ folk all participating in a PGP keysigning had to be seen to be believed (eventually, we just had to go outside, as the lecture theatre just wasn’t big enough)

I signed up a few months ago to present a Lightning Talk on OpenAFS, in an attempt to grow awareness, and attract new developers. That talk certainly helped me with talking to other people at the conference, as well as being pretty well received. Both slides, and video, of the talk are available from the FOSDEM site.

February 15, 2008

Talking

Filed under: Uncategorized — sxw @ 6:14 pm
Tags: , , , , , ,

I’m giving a few talks over the next couple of months

  • UKUUG Files and Backup Seminar I’m giving a general overview of AFS from a users and administrators perspective, particularly focusing on features that will be of interest to new deployments
  • FOSDEM I’m giving a developers overview of OpenAFS as a lightning talk
  • UKUUG Spring Conference I’m currently scheduled to give two talks. The first is an overview of our monitoring system, talking in particular about the benefits (and challenges) of integrating it with LCFG. The second is about our in-development account management system, prometheus, and some of its unique features.
  • AFS & Kerberos Best Practices Workshop

I’m going to FOSDEM, the Free and Open Source Software Developers’ European Meeting

Theme: Rubric.