I think I’m done

October 2, 2018

I think I’m done. There is now a final report https://wiki.inf.ed.ac.uk/DICE/FinalProjectReport364SchoolEdWeb and a catalogue entry https://wiki.inf.ed.ac.uk/DICE/ServiceCatalogueEntryInfWeb. I need to archive these blog pages and submit it for official sign off. It’s only been 4 years in the making!


Get it Finished

April 25, 2018

As usual the last 10% of the project is taking 90% (of the real time) to complete. This post is a kick up my own backside to get this project finished.

So, even though the things to do are recorded on https://computing.projects.inf.ed.ac.uk/#364, as a note to myself, the things I need to do are:

  • Proper disaster recover instructions. Currently web.inf is being mirrored to KB, but needs to be done better and documented. I think I’ll create a new VM at KB, rather than piggyback on the physical DR for www.inf.ed.ac.uk – Now done https://wiki.inf.ed.ac.uk/DICE/WebInfEdAcUKDisasterRecovery
  • Document the routine tasks. Which basically means what to do when updates come out, as we no longer need to worry about our local patches, as they’ve been incorporated into the official distribution, or equivalent functionality has been provided.
    • I do need to provide some info on our Feature module, and what to check of our local config after upgrades

    Done?! https://wiki.inf.ed.ac.uk/DICE/SchoolEdWeb

  • Lastly, write a final report.

Cosign/EASE drupal admin problem

October 28, 2016

We’ve been holding off using Cosign/EASE on our EdWeb distro site, until we had a clear solution to the issue of how to become the Drupal admin user (user=1). As soon as we turn on Cosign authentication, then we’ll only ever be able to be user UUN. Even if we created functional accounts (to use IS parlance), then due to our automatic authentication via browsers on DICE, it isn’t very convenient to become anyone other than our UUN. Also, on principle, we don’t have accounts in our authentication service that aren’t associated with an actual individual.

Asking around it sounded like there were two options people used:

  1. Just don’t sweat it, and use drush from the command line to do all your admin type duties.
  2. Either give yourself (or a functional account) all the available drupal permissions, so you can do everything.

Not being that fluent with drush, and that our web editors wouldn’t have necessary command line access to the server, 1. didn’t seem the best solution. Options 2 has problems, as the EdWeb developers are a bit wary of this and are not making any guarantees (in fact more likely the opposite) that we wouldn’t be storing up problems if user regularly published with all permissions granted.

So what we’ve decided to do is something a bit like a blend of the two options above.

Our solution

Ceate a new “admin user” role:

drush role-create 'admin user'

Give the existing EdWeb role “system administrator” a couple of extra
permissions:

drush role-add-perm "system administrator" "administer permissions"
drush role-add-perm "system administrator" "administer users"

The “system administrator” role is already one that should only be given to a few select people who know what they are doing. Generally people will not have this role.

Then as one of those users who is a “system administrator”, via the web GUI,  give the new “admin user” role all the other permissions with a couple of mouse clicks,  excluding the “bypass …” permissions.

The above steps should only need to be done once to get things set up.

Now if any of the existing “system administrators” need to do something as the admin (user=1) person, then they can temporarily give themselves the “admin user” role, do what we need to do, and then remove the role from themselves once they are done. They should know not to create or modify content with the “admin user” role, but if they did, then hopefully by not having the “bypass …” permissions, things would be OK, but we shouldn’t rely on that.

As I write this, I might look at adding a block that only shows up when you have the “admin user” role, to remind you that you are (nearly) all powerful.

It’s a bit of a faff to have to add and then remove the “admin user” role, but luckily the times you need to be user=1 is fairly rare. And probably just as much as a faff if we were to do thing via functional accounts.

Notes

The suggestion to not grant the “bypass …” permissions came from Mairi. In an email/slack post she said:

On the subject of the ‘bypass content access control’ permission, the problem is that because permissions are being bypassed, you wouldn’t necessarily know which hooks are firing & which aren’t. Bypassing permissions will just invisibly allow the user to do anything, with no indication of whether data integrity is OK until something goes wrong. For example, we believe it’s probably OK to publish as user 1, provided that user is configured with relevant group memberships; however, we advise against it because nothing will tell you, when logged in as user 1, that the group hooks are actually firing. Which leads to unforeseen consequences when content is inadvertently created/published without the correct permissions being in place. If you give another user those ‘bypass’ permissions, the same will apply – i.e. that user could be publishing content without the correct hooks firing.

She also pointed me at this article from Stanford https://drupaltraining.stanford.edu/node/13.

Neil


Code sprint

August 26, 2016

Back in July I attended IS’ first EdWeb code sprint. IS are trying out the idea to encourage more collaboration, and find an alternative source of resource to actually get EdWeb code development done.

I found it very useful, though for this first one, everyone was learning, and though I did get my submission back to their git accepted, it was a rather minor change, but real outstanding work that needed done at some point.

There’s a new updated, 1.12, which I’m about to try. It will be interesting to see if my code changes are in there.


Cosign and cron

May 6, 2016

Having got EdWeb working with EASE, I went back and tried it against our weblogin CoSign service, and that too works. My initial problems were probably the openldap issue that EASE had.

I’ve still not got a clear way to access a site as “admin” once EASE/Cosign is enabled, other than temporarily disabling the UoE LDAP and EASE modules, and reverting to the old way.

I’m looking at an “admin” site that uses basic auth, so I can sign in as “admin”, and though that has a slight success, it then fails with what looks like a failure to find “admin” in the LDAP and then extract an email address, and role information for him. I’m not sure if that should be considered a bug or not.

Also had a look at cron, has Kenny was having problems with the new scheduling on our test 1.10. A mail to UWS Tech suggests that using the wget and cron key is the way to do it, rather than using drush (which I’d been experimenting with).

 


EdWeb 1.10 and EASE

April 29, 2016

EdWeb 1.10

There have been a couple of updates since we last updated web.inf to 1.7, but each one has had issues which stopped us (and Maths) from upgrading.

http://dist.drupal.is.ed.ac.uk/project/uoe_distribution

But Mairi and co. have been very helpful in trying to solve the problems, and it now looks like we’re just about there, with the future 1.10 release addressing the problems, and including a Drupal security update.

The 1.10 will mean that our only local patch will be for the local search. I’ve created a new version of the patch, as the page template has changed, and though it does work, it’s introduced some extra white space I’m not happy about. However, Kenny’s happy to just apply that as it is for now, rather than delay 1.10 (when it appears).

EASE

I’ve also been looking at this again, and have made some progress. webtest.inf is now EASE authenticated, and after some problems with ldap binding to the IS server, it now seems to work for users who already have a local EdWeb account on the site (ie me).

https://www.wiki.ed.ac.uk/display/CSEDR/EdWeb+installation+hints

However it isn’t working for people who don’t have a local account, it is supposed to be created on the fly, but I’m getting error messages. Looking at those will be my next action. David McKain has given me some hopefully useful debugging options for LDAP in case LDAP is the issue:

(1) drush vset -y ldap_help_watchdog_detail 1

should turn logging on

(2) drush vdel -y ldap_help_watchdog_detail

should turn it off again

The log messages go into the Drupal watchdog table, aka the 'recent log messages'
report. You can view these either within the GUI, or using drush commands. Do
'drush help | grep watchdog' to see the drush commands.

EdWeb 1.7

January 29, 2016

Last week EdWeb distro 1.7 was released. This week I tried installing it on webtest.inf, first as a fresh install (which gave and error) and secondly updating a clone of web.inf, which worked fine.

The failure, when doing a fresh install, was the same problem Craig had last week, this was my chance to see it for myself. The problem turned out to be a new test to make sure that the PHP variable max_exectution_time was suitably large, but using drush from the command line to install, then that var is set to “0” indicating unlimited.

I submitted the issue and patch via http://dist.drupal.is.ed.ac.uk/node/446 which will be incorporated into the next release. I tested the fix on their updated dev version.

The plan is to update web.inf.ed.ac.uk on Tuesday morning.

Kenny’s been asking again about EASE and restricted pages. I must have a look at that soon.


EdWeb 1.6 update and banner

December 2, 2015

The first attempt to update a fresh copy of web.inf running on webtest.inf, didn’t work. And I got a strange PHP error. It looked like some issue with our content in the DB. A call was logged, but in the mean time they suggested it was our local theme. I doubted it, but I did try an update of webtest.inf with the web.inf data, but without our theme installed and one of the default edweb themes. Much to my chagrin, it did update. For completeness I took another fresh copy of web.inf, this time with our local theme still in use, and that too worked!

Since then I’ve tried several more times to get the update from 1.5 to 1.6 of the distro to fail, but it has not. I’ve let the web support people know via the unidesk ticket.

Hide Banner RFC

Stratos informed me that the next release of EdWeb will contain the ability to hide the banner image on certain pages. It will be configurable to hide on single pages, or complete (sub-)sites. And will be available to people with the CMS Operative role. This sounds ideal, and an improvement over my simple solution. This should mean our RFC for this can be shelved. Hopefully this change will be on central by the end of the year, and in the distro early next year.

 Change the way we apply our local theme

While testing the upgrade from 1.5 to 1.6, I tripped over the problem that you can’t edit and save the homepage content type, without picking an explicit colour theme. Unfortunately this then overrides our site wide local search and hidden banner theme. So the banner and Ed.ac.uk wide search re-appears. I’ve not found a way for our theme to trump the homepage selected theme, so I’ll probably have to look at not using a theme to override things, but just patch the EdWeb distro directly. A simple hack of replacing the shipped page.tpl.php, does indeed replace the the template, but things like:

 if( !theme_get_setting('uoe_localsearch_hidebanner')) {

Don’t read the local theme setting, so presumably they’ll all have to be patched into the shipped theme. More experimenting is required.

Neil


EdWeb update to 1.6

November 20, 2015

On Tuesday afternoon, and update to the EdWeb distro was released – 1.6. http://dist.drupal.is.ed.ac.uk/project/uoe_distribution, and the following day the site started warning us that there was an update available. So far, so good.

The following day I updated webtest.inf with a fresh copy of the live web.inf site, and did a diff of the code. It seemed mostly the latest updates to drupal core that we were missing, plus some other EdWeb changes, but it didn’t look like new functionality, and the release notes didn’t mention anything.

When I tried the recommended upgrade procedure using drush it seemed to go smoothly, more smoothly that previous updates, but the site is now broken, and all pages reporting the error:

Fatal error: Class name must be a valid object or a string in /disk/data/edweb/includes/common.inc on line 7998

More worrying, it also reports the same when doing drush commands, even drush status. I don’t really know what to do, so I’ve reported the problem to website.support@ed (UniDeskI151120-0697)  and given then copies of the site and DB before and after the update.

I suspect it is a problem with some content we have, rather than our local theme, as happened once before in the early days. Some simple debugging shows

$type_info = entity_get_info($entity_type);

Is returning an undefined value for something called “rules_config”, which seems to be a module used in the distro. My hope is that someone who knows will spot a problem with our DB, and give us some SQL to fix it.

In the meantime I’m going to revert webtest.inf back to the live web.inf, remove our local theme, and try the update again to see if that makes any difference.

Neil


web.inf mirror

November 6, 2015

While looking at the various drupal versions we have, I spotted that we had an old version on canny.inf (our www.inf DR mirror machine). A while a go I’d started setting it up to add a mirror of web.inf (drip), but obviously hadn’t quite finished it off.

So I’ve now got something going, that every four hours (when the normal www.inf DR copies are taken) it also rsync’s the data from web.inf and restores web.inf on canny.

At the moment, it was simpler to have the web mirror running on a non-standard port (I can’t remember why at the moment) so the DR copy of web.inf is actually http://dripped.inf.ed.ac.uk:8088/.

While doing this, I hit some problems with stopping and restarting MySQL, it was taking longer to start up/shutdown than expected and then it was restarted. Some things complained about mysql not being available. Co-incidentally Duncan from physics hit a similar problem, so some work may need to be done to make stopping and starting more robust. I’ll probably add a “dont do backup at stop” option too.