The Forum server rooms are covered by a pair of Uninterruptible Power Supplies (UPSes), one of which has developed a fault in its interface to the outside world. As a result we can neither query its status, nor expect it to signal machines to shut down when the power goes off.
UPS Repair Man came on schedule on Saturday 3rd. He turned the faulty unit off, took it apart, and put some new cards in it. Unfortunately he then struggled for the rest of the day trying to get one of those new cards to recognise the specific model of UPS we have. As a result, he couldn’t start the unit up with those new cards, so he eventually stripped them all back out again, replaced the old ones, and brought the faulty unit back up in the same (faulty) state as before. He’s off to consult his base now.
On the minus side, we’ll have to go through some of this again once they get an idea of what to do next. On the plus side, turning the faulty unit off didn’t appear to cause any problem for the working one (other than to have it flash a fault light, which is apparently a firmware bug!), so we may be able to do the process on a normal day next time assuming we can find enough load to shed.