This article describes Prometheus, the account management system developed within Informatics (note that the choice of name comes from this Prometheus, and predates the recent film). More accurately, Prometheus could be described as an entity management system, as it manages machine details (and other things), as well as people. This article will only discuss the latter.
Prometheus was written to solve a few shortcomings with our existing systems, the major ones being:
- The trade-off between centrally managed account databases and decentralised per-service database – how we effectively manaage these.
- A varied collection of automated scripts, all written in different ways, in different places.
- The need for sysadmins to do things manually (sysadmins like efficiency and error-reduction and hence don’t like doing things manually if they should be automated).
- Being unable to effectively manage the entire account lifecycle – deletion/archival as well as creation/modification.
- Various management issues – spotting/correcting inconsistencies/anomalies.
- The need for better reporting.
A typical DICE user’s account, rather than being a single item, actually consists of four separate parts, all of which are required to make the account work…
- Authentication in Informatics is done by Kerberos, so any user needs to have an entry in the Kerberos database, the KDC.
- When a user logs into a DICE machine, the system needs to know who the user is, what groups they are in and other authorisation information. This comes from LDAP.
In Informatics, users have AFS home directories, so this requires…
- An entry in the AFS user database.
- An entry in the AFS volume database, specifying which machine hosts the users volume, where the backup is and so on.
These four separate (but integrated) parts provide a DICE user with a full account, but where does the information required to manage all of this come from? The answer is several different places – information comes from the Informatics school database, from central University feeds and from other local and central sources. The job of Prometheus is to take data from these places, act on it and populate our local databases accordingly, as illustrated in this (greatly simplified) diagram:
So far we’ve discussed only managing the databases that are needed for a full DICE account – Prometheus can also manage anything else which requires user data. For example, services which maintain their own user databases could have those databases partially or wholly managed by Prometheus.
Prometheus makes heavy use of the roles and capabilities system used within DICE. Simply put, a capability is something that a user (or other entity) can do and a role is a collection of these capabilities (and other roles). A role should describe functions that a user performs or a position they hold (e.g. sysadmin, 1st year undergrad, visitor). An example better illustrates how roles and capabilities can be used in reality…
All current members of staff possess the ‘staff’ role – they get this automatically through the school database (and they have it automatically removed when they leave). One of the capabilities this currently provides is ‘login/staffssh/remote’. Prometheus populates an LDAP netgroup of this name with all users who possess this capability. The server staff.ssh.inf.ed.ac.uk has its access controls set (via LCFG) to grant access to all users within this netgroup.
This system is very powerful and allows us to automate many things. One example of this is account creation – once someone becomes a current member of staff, a student or a visitor, the school database allocates them the appropriate role. This in turn gives them the various capabilities required for a full DICE account. Prometheus acts on these capabilities and does all the work necessary to create a full account, without any intervention. Each year’s new student intake requires that we create hundreds of new accounts. This now all happens automatically.
Prometheus is written in Perl using Moose (an object-oriented framework for Perl). It was developed using our git/gerrit not-a-service. Prometheus’s own data is stored in an LDAP directory. Two key concepts are “datastores” and “conduits”. A datastore is a representation of something which Prometheus needs to talk to (some examples of this are: Prometheus’s own entity store, a KDC, IS’s uid server, the school database). A conduit effectively joins datastores together, so, for example, there is a conduit which queries the IS UID server (using its datastore) and puts the uids into the entity store. To populate another service with user data, as mentioned previously, it would need to firstly have a datastore written to add, modify and delete entries and then a conduit written to perform these operations based on data held within Prometheus. Conduits and datastores help to ensure that data flows through our systems automatically and into the correct places.
Some of the future developments we have planned for Prometheus are:
- Full account lifecycle management – this is in the pipeline and will provide us with a means of implementing grace periods and staged expiry/archival/deletion.
- Better support for lightweight accounts – not everyone needs a full DICE account, the system is flexible and fine grained enough to manage this.
- Users should be able to manage their own identities – all DICE users have a user@INF.ED.AC.UK kerberos principal, some have additional principals for, e.g. managing long running jobs. It would be very nice if a user could manage these themselves.
- Extend and enhance – sysadmins/users could write conduits and datastores for their own services.
Much more information on Prometheus can be found here.
