Serving AFS space using Apache and mod_waklog

As part of the Informatics move to using AFS, Roger and I have been
investigating how to serve AFS files using Apache.

The primary technical consideration is that apache needs to be able to authenticate against our KDC, needs to be able to obtain AFS tokens for the principal it’s authenticated as and AFS acls need to be configured accordingly to provide access to the underlying files.

Firstly we created a Kerberos service principal of the form
afsweb/melody.inf.ed.ac.uk@INF.ED.AC.UK (melody.inf.ed.ac.uk is the host on which we’re doing our initial testing). This principal maps automatically onto the AFS id afsweb.melody.inf.ed.ac.uk (once it has been created in the pts database). The intention is that each machine which requires web access to AFS space should have one of these service principals. They can then easily be combined into AFS system group or groups (e.g. system:afswebservers).

Our initial testing focused on using k5start with pagsh when starting apache itself. This approach meant that k5start was entirely responsible for obtaining the kerberos tickets and afs tokens and keeping them up to date. Apache, being launched from the same pagsh as k5start, then had access to the tickets/tokens and could read appropriately configured areas of AFS file space.

The ACL configuration necessary was to add rl permissions for the afsweb.melody.inf.ed.ac.uk AFS id on the directory containing the files to be served and l on all its parents (see below for further
discussion on this in the context of user-authenticated access).

This approach worked well, but had some obvious drawbacks:

  • Starting k5start and apache from within a pagsh requires hacky changes to the apache and/or apacheconf LCFG components. The idea of developing a separate LCFG component purely to manage tokens using k5start was mooted (and may still happen) but the issue of PAGs complicates this – i.e. apache and k5start have to be started from within the same pagsh or httpd has no access to the tokens obtained.
  • We have no granularity – this approach allows us to use one AFS id for the entire website, but nothing finer.

So, we moved onto looking at mod_waklog to see if it offered a better solution…. it certainly seems to. We were very quickly able to
provide a simple solution accessing files as the afsweb id using only
apache directives. Having installed an RPM of mod_waklog and adding the appropriate LoadModule directive, this is what we added to the httpd.conf

WaklogAfsCell   inf.ed.ac.uk

<VirtualHost melody.inf.ed.ac.uk:443>
...
WaklogEnabled           On
WaklogDefaultPrincipal afsweb/melody.inf.ed.ac.uk /etc/afsweb.keytab
...
</VirtualHost>

This all worked perfectly, with apache able to serve files where ACLs
for afsweb.melody.inf.ed.ac.uk permitted.

One thing worth noting here is that it’s necessary for the keytab to
be readable by the user that apache runs as. We initially had the
keytab permissions/ownership “600 root root” – this worked when apache starts but when it needs to eventually renew tickets, it has long since dropped root privileges so can’t read the keytab. We changed to “600 apache apache” – it would be advisable to check the implications of this with respect to, e.g. user scripts.

Another thing that mod_waklog gives us that might be really useful is the ability to use different credentials for different locations using
the WaklogLocationPrincipal directive. We need to investigate this
further.

We can also use mod_waklog to authenticate on a user basis, by tying it in with cosign. Here’s an example segment of httpd.conf:

<Location /userweb>
    CosignProtected On
    AuthType Cosign
    Require valid-user

    CosignGetKerberosTickets On
    CosignKerberosSetupGss On
    WaklogUseUserTokens On
</Location>

This also seems to work smoothly – visitors to the /userweb location will be authenticated via cosign and corresponding AFS tokens obtained on their behalf. They will then be subjected to AFS ACL constraints as the user they are authenticated as – that user must have permission to access the file(s) being served, i.e. must have permission all the way from the root of the filesystem. For apache to obtain the user’s kerberos ticket from cosign, the cosign servers must be configured to allow the web server to request kerberos tickets, e.g. something like the following line in /etc/cosign.conf:

service melody.inf.ed.ac.uk T

It is important to note here that, due to the way in which apache
works, the principal specified as WaklogDefaultPrincipal will also
need rl ACLs on any directory from which files will be served and l ACLs on all parent directories. Other ACLs can be used however – what’s important is that the identity that apache is running as (in our example, afsweb/melody.inf.ed.ac.uk) can read the files being served. So, ACLs for system:authuser, for example, could also be used.

This entry was posted in AFS, Apache, Cosign. Bookmark the permalink.