lcfg-x509 and letsencrypt

Automated acquisition of X509 certificates is an important part of our infrastructure. We obtain certificates of two different types, both provided through services offered by the University:

  1. Certificates provided via JISC, which are signed by a CA trust chain whose root certificate is installed in most OSes and browsers by default. These are obtained manually, by submitting a CSR through a web form.
  2. Certificates signed by a trust chain whose root is the University’s CA certificate. This is an automated process, using the lcfg-x509 component and our locally written sixkts software. We can automate the signing process as we have an intermediate CA certificate, signed by the University’s root.

It is becoming increasingly impractical and unrealistic to expect users to install the University’s root CA in their devices, so most of our public facing web sites are now using certificates of type 1. above. We continue to use certificates of type 2. for other purposes, where we can manage the trust chain, e.g. for backend communications for Cosign and for TLS LDAP.

The big problem that this gives us is automation, or rather the lack of it. Pasting CSRs into web forms, waiting for an email, extracting certificates from that email, copying them to the appropriate machine, changing ownership and permissions, restarting appropriate services… it’s tedious and error-prone.

All of this brings us to Let’s Encrypt. It’s in public beta at the moment, but offers us an automated way of obtaining trusted certificates.

We already have an LCFG component – lcfg-x509 – which obtains certificates (using our sixkts technology) and then does all the administrative work necessary to put them into service. As a proof of concept I added support for letsencrypt to a development version of this component…

I decided to use the client – the official client seemed a little complex and heavyweight to me, with a hefty array of python dependencies. seemed more lightweight and better suited to our uses.

The development version of lcfg-x509 supporting letsencrypt and its corresponding schema can be found here:

An RPM for can be found here:

(SL7 versions of all the above are also available, but I’ve only tested with SL6)

Here’s some example LCFG which modifies an existing lcfg-x509 certificate, for the machine “oakwood”, to use letsencrypt:

!profile.packages mEXTRA(+letsencrypt-sh-0.0.3-1.el6/noarch)
!profile.packages mEXTRA(+lcfg-x509-0.0.31_dev-15/noarch)

!profile.version_x509  mSET(4)

!x509.le_client_path mSET(/usr/sbin/
!x509.le_wellknown mSET(/var/www/html/.well-known/acme-challenge/)
!x509.catype_oakwood mSET(letsencrypt)

This requires the following chunk of httpd config in order to respond to the http-01 challenge:

<IfModule mod_headers.c>
  <LocationMatch "/.well-known/acme-challenge/*">
    Header set Content-Type "text/plain"

Some caveats:

  • This is an early developmental release, just to see if it works.
  • It lacks useful documentation. The template for’s config file – /usr/lib/lcfg/conf/x509/ is informative, as is the github page for
  • The lcfg-x509 component is quite old and is definitely showing its age. It should probably be rewritten.
  • … We may decide to decouple letsencrypt from lcfg-x509 – supports “hooks” which can be used to provide your own code to be called at various stages of the process (specifically “clean_challenge”, “deploy_challenge”, or “deploy_cert”). This is potentially very powerful in automating challenge responses.
  • lcfg-x509 currently makes no efforts to tidy up old certificates obtained via letsencrypt
  • … or anything else it uses for letsencrypt
  • Responding to the ACME challenges could be the trickiest part in all of this – the above config relies on you having an apache server already listening on port 80. The dns-01 challenge looks promising, but requires an API to add and remove DNS records.
  • The rate-limiting for letsencrypt is quite strict – currently limited to 5 per domain per week. This doesn’t sound too bad until you realise that domain == public suffix + domain, which means everything under “” counts as the same domain. It’s probably best to use the staging URL for testing.
This entry was posted in Uncategorized. Bookmark the permalink.