I think I’m done. There is now a final report https://wiki.inf.ed.ac.uk/DICE/FinalProjectReport364SchoolEdWeb and a catalogue entry https://wiki.inf.ed.ac.uk/DICE/ServiceCatalogueEntryInfWeb. I need to archive these blog pages and submit it for official sign off. It’s only been 4 years in the making!
As usual the last 10% of the project is taking 90% (of the real time) to complete. This post is a kick up my own backside to get this project finished.
So, even though the things to do are recorded on https://computing.projects.inf.ed.ac.uk/#364, as a note to myself, the things I need to do are:
- Proper disaster recover instructions. Currently web.inf is being mirrored to KB, but needs to be done better and documented. I think I’ll create a new VM at KB, rather than piggyback on the physical DR for www.inf.ed.ac.uk – Now done https://wiki.inf.ed.ac.uk/DICE/WebInfEdAcUKDisasterRecovery
- Document the routine tasks. Which basically means what to do when updates come out, as we no longer need to worry about our local patches, as they’ve been incorporated into the official distribution, or equivalent functionality has been provided.
- I do need to provide some info on our Feature module, and what to check of our local config after upgrades
- Lastly, write a final report.
Just testing links to videos, or embedded videos.
Link to media hopper
Embed media hopper
Link to YouTube
Embed – uses iframe
Apparently the mathjax CDN is closing at the end of April.
Our blog has a MathJax plugin that uses this CDN, so we will be affected (if anyone uses it).
This is a test to see if I can use it!
Simple shortcode version \(E=mc^2\)
Native format $$E=mc^2$$
Not getting far \(x^2\).
Ah ha, need to select not default from the plugin configuration.
f(x)=x^2 = 4
Update: Well it’s May now, and the above sort of still works. Not sure the last multiline thing ever did.
Just some notes on what worked for me to update an HP G2 Elite ME firmware, when it isn’t running Windows.
There’s a bug in HP ME firmware that can brick a machine – http://h20565.www2.hp.com/hpsc/doc/public/display?sp4ts.oid=7815289&docLocale=en_US&docId=emr_na-c05306753
They’ve released an update that contains a Windows installer and an EFI installer.
To use the EFI update I needed:
- FAT32 formatted USB drive.
- The update software and data from the HP sp78318.exe – Local-EFI/ directory and the 110181002.bin file. The directory and .bin file were copied to the root of the USB drive.
- A EFI Shell, which I Googled and found at https://www.msi.com/files/pdf/win8_UEFI_BIOS_Update_auto_en.pdf which links to the actual files http://download.msi.com/nb_drivers/ap/efi_auto.zip
- Extract the contents of that zip to USB, so you should end up with a USB:EFI/BOOT/Bootx64.efi file.
Now go into the HP G2 BIOS and enable UEFI USB booting (you don’t need to disable legacy booting).
Insert the USB into the G2.
Reset the G2, and press F9 or Escape to get to the option to boot from USB.
Select “File” and browse to the EFI/Boot dir and select Bootx64.efi (if it didn’t just automatically run itself).
You’ll probably get a warning about choosing to abort running startup.nsh, that’s an MSI file that is benign, and you could have not bothered copying it.
You can let it run, or skip it, eventually you’ll end up with at the EFI Shell prompt.
and the updater should run (without further prompting).
Remember to undo the USB booting in the BIOS (if that was your previous setting)
You could do some tidying and reorganising, eg that zip contains more than you need, but it shows that it can be done.
We are hoping to PXE boot into the EFI Shell, so hopefully removing the need to mess with BIOS settings. After some experimenting and reading, this looks like it is only possible by enabling EFI mode in the BIOS anyway, so doesn’t save you from having to fiddle with the BIOS.
Most of this is from memory, so some details may need checked.
I’ve added fail2ban to the SL7 version of our auth smtp service. None of the sendmail filters that come with the fail2ban RPM seemed like they’d do the trick for us, so I’ve just overridden the supplied
filter.d/sendmail-auth.conf with a
sendmail-auth.local containing just:
[Definition] failregex = ^%(__prefix_line)s.*AUTH failure.*\[\]( \(may be forged\))?$
Though that isn’t enough to get it to match, as the default log level (9) for sendmail doesn’t log auth failures. So we also have to run at log level 10.
Currently I’m using the local
lcfg-hostsdeny and tcpwrappers template like sshd does, but we should probably look at using iptables instead.
In the few days its been running, 21 IP addresses have been banned.
We’ve been holding off using Cosign/EASE on our EdWeb distro site, until we had a clear solution to the issue of how to become the Drupal admin user (user=1). As soon as we turn on Cosign authentication, then we’ll only ever be able to be user UUN. Even if we created functional accounts (to use IS parlance), then due to our automatic authentication via browsers on DICE, it isn’t very convenient to become anyone other than our UUN. Also, on principle, we don’t have accounts in our authentication service that aren’t associated with an actual individual.
Asking around it sounded like there were two options people used:
- Just don’t sweat it, and use drush from the command line to do all your admin type duties.
- Either give yourself (or a functional account) all the available drupal permissions, so you can do everything.
Not being that fluent with drush, and that our web editors wouldn’t have necessary command line access to the server, 1. didn’t seem the best solution. Options 2 has problems, as the EdWeb developers are a bit wary of this and are not making any guarantees (in fact more likely the opposite) that we wouldn’t be storing up problems if user regularly published with all permissions granted.
So what we’ve decided to do is something a bit like a blend of the two options above.
Ceate a new “admin user” role:
drush role-create 'admin user'
Give the existing EdWeb role “system administrator” a couple of extra
drush role-add-perm "system administrator" "administer permissions" drush role-add-perm "system administrator" "administer users"
The “system administrator” role is already one that should only be given to a few select people who know what they are doing. Generally people will not have this role.
Then as one of those users who is a “system administrator”, via the web GUI, give the new “admin user” role all the other permissions with a couple of mouse clicks, excluding the “bypass …” permissions.
The above steps should only need to be done once to get things set up.
Now if any of the existing “system administrators” need to do something as the admin (user=1) person, then they can temporarily give themselves the “admin user” role, do what we need to do, and then remove the role from themselves once they are done. They should know not to create or modify content with the “admin user” role, but if they did, then hopefully by not having the “bypass …” permissions, things would be OK, but we shouldn’t rely on that.
As I write this, I might look at adding a block that only shows up when you have the “admin user” role, to remind you that you are (nearly) all powerful.
It’s a bit of a faff to have to add and then remove the “admin user” role, but luckily the times you need to be user=1 is fairly rare. And probably just as much as a faff if we were to do thing via functional accounts.
The suggestion to not grant the “bypass …” permissions came from Mairi. In an email/slack post she said:
On the subject of the ‘bypass content access control’ permission, the problem is that because permissions are being bypassed, you wouldn’t necessarily know which hooks are firing & which aren’t. Bypassing permissions will just invisibly allow the user to do anything, with no indication of whether data integrity is OK until something goes wrong. For example, we believe it’s probably OK to publish as user 1, provided that user is configured with relevant group memberships; however, we advise against it because nothing will tell you, when logged in as user 1, that the group hooks are actually firing. Which leads to unforeseen consequences when content is inadvertently created/published without the correct permissions being in place. If you give another user those ‘bypass’ permissions, the same will apply – i.e. that user could be publishing content without the correct hooks firing.
She also pointed me at this article from Stanford https://drupaltraining.stanford.edu/node/13.
Back in July I attended IS’ first EdWeb code sprint. IS are trying out the idea to encourage more collaboration, and find an alternative source of resource to actually get EdWeb code development done.
I found it very useful, though for this first one, everyone was learning, and though I did get my submission back to their git accepted, it was a rather minor change, but real outstanding work that needed done at some point.
There’s a new updated, 1.12, which I’m about to try. It will be interesting to see if my code changes are in there.
Just a brief update on AFS for SL7.
This week all my changes have made it to stable, and there are now 1.2.x versions of the lcfg-openafs (server only) and lcfg-openafs-client components for SL7.
I spent a bit of time teasing the two components apart, so either can be installed in isolation (or together) on a machine. In hindsight it would have been cleaner to leave behind the “openafs” component in the SL6 world, and created a new openafs-server component like Stephen did for the openafs-client component. It would have made the various header files and schema files a cleaner split, but it’s done now.
Craig, I and the Unit have their AFS volumes on a gresley, a new SL7 AFS server. We need to move some more guinea-pigs, but it all seems fine.
Having a look at apacheconf-waklog.h on DICE SL7. This is actually the first SL7 web server stuff I’ve looked at. So first of all I thought I should try getting a minimum SL7 apacheconf.h web server going.
I commandeered circlevm9, a vanilla SL7 server.h VM. And added
After the profile pushed, and I ran updaterpms.
om apacheconf start didn’t “just work”.
22/07/16 12:17:35: apache configuration has been modified
22/07/16 12:17:35: Syntax OK
22/07/16 12:17:35: Failed to reload httpd.service: Unit httpd.service is mas\
22/07/16 12:17:35: ** reload httpd: Fail
systemctl gave me a suggestion:
[circlevm9]root: systemctl status httpd
Loaded: masked (/etc/systemd/system/multi-user.target.wants httpd.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Warning: httpd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
So I tried that:
[circlevm9]root: systemctl daemon-reload
[circlevm9]root: systemctl status httpd
httpd.service – The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Now, after doing an ‘om apacheconf stop’, ‘om apacheconf start’ worked and left httpd process running with /var/www/html/ as the docroot, but with all access denied. I’m presuming a reboot would have had a similar affect.
I then added a simple vhost to open up access to /var/www/html/ so that I could dump stuff in their and convince myself the basics worked.
!apacheconf.vhosts mADD(default) apacheconf.vhostname_default _default_ apacheconf.vhostdocroot_default /var/www/html apacheconf.vhostaccesslog_default /var/lcfg/log/apacheconf.access apacheconf.vhosterrorlog_default /var/lcfg/log/apacheconf.error !apacheconf.vhostverbatim_default mADD(stuff) apacheconf.vhostline_default_stuff <Directory "<%apacheconf.vhostdocroot_default%>">¶\ Options Indexes FollowSymLinks¶\ Require all granted¶\ </Directory>
With that done, I was able to drop files into /var/www/html/ and they would be served. Equally I added some symlinks to other bits of the file system, and they were followed unless file permissions said otherwise. So a symlink to /afs/inf.ed.ac.uk/ showed the contents of publicly accessible stuff, but all other access was denied by ACLs.
So now I know if I add apacheconf-waklog.h and get it working, if they symlinks to AFS show more content, then httpd will have obtained the necessary AFS PTS tokens.