Coursera AI Planning MOOC Interview

[Transcript and commentary of interview by Siân Bayne and Jen Ross, School of Education, University of Edinburgh with Austin Tate on 24th September 2013 about his AI Planning MOOC]

• Artificial Intelligence Planning
• Instructors: Dr. Gerhard Wickler and Prof. Austin Tate
• Start date: 28 January 2013. Length: 5 weeks (ran also in 2014 and 2015)
• Course Web site: http://www.aiai.ed.ac.uk/project/plan/ooc/ (was also https://www.coursera.org/course/aiplan)
• AI Planning MOOC Interview – Motivation – 15-Jun-2016
• Publicly accessible statistics about each of the three Coursera sessions are in other blog posts as follows: 2013, 2014, 2015.

Participant location on the AI Planning MOOC,
11 February 2013 based on a sample of 3,335

Part of the motivation for developing this MOOC was a desire on the part of the course team to make a body of rare materials readily available online as part of a legacy approach to a niche collection of teaching material. Professor Austin Tate explained that:

We have a lot of [teaching] materials for occasional use. We were already thinking about how to package that better, particularly as I come to retirement age. We want the materials to be available as a basis for future PhD and MSc student projects. And some material we’ve got that we didn’t want to lose, in particular we’ve got materials that even some of the originators haven’t got anymore and we wanted to try and make sure some of that was brought together. So that’s the motivation – packaging it well for others to make use of, and the broader we can disseminate that the better.

This teaching material is technical content that has been held by the School of Informatics at the University of Edinburgh for some time, collected over a long period and used occasionally in the teaching of conventional classes, but not until now made widely available online. These materials and their proper care were core to the design ethos of the MOOC.

However, while one driver for the MOOC design was a desire to make available these materials as legacy, an equally strong one was the desire to build and extend a community, as Professor Tate explained:

the whole framework of it was definitely conceived as, and run as, a community of people interested in a common topic, and working together and exploring that space together. I was trying to reach different communities, and not just those interested in programming.

In part, this was approached by a course design that made success on the course achievable by non-programmers. Participants were able to gain ‘Statements of Accomplishment’ for achieving one of three ‘levels’ of successful participation:

• awareness level (aimed at those who were approaching the course as a ‘taster’ or a very broad introduction to the subject matter – 352 people passed at this level)
• foundation level (for those who had fully grasped the core course content – 148 passed at this level)
• performance level (for those who had taken their understanding of course content to a more advanced level by completing programming assignments or the creation of a digital artifact – 152 passed at this level) (Tate 2013)

This MOOC did important work therefore in opening up alternative ways of thinking about MOOC ‘completion’, and in doing so worked to broaden its own community of learners. For example, two science fiction authors took the course at awareness level as a means of achieving a richer subject-knowledge in their literary area.

This opening of community was also approached by bringing in guest ‘feature’ lectures from eminent international figures in artificial intelligence research, and then inviting these guests to interact with students within the MOOC discussion space. The MOOC space in this sense became a richly curated expression of the development team’s research networks, rare materials collections and a teaching ethos described by Professor Tate as inherently collaborative:

I like projects, I like things where you pull a lot of things together, I like multi-person things…. The bits I was doing on this course… the way we pulled people in, I think that reflects my interest in collaborative, joint things – doing things together. … I’m not so keen on people going away and doing exercises, beating their head against the screen all on their own.

In one noteworthy interaction in response to an enquiry about the use of AI planning algorithms with navigation of the NASA Curiosity rover on Mars, a guest lecturer asked a colleague at NASA who was driving the rover for the definitive answer.

The AI Planning MOOC also included use of a virtual world platform, Second Life™, for meetings between instructors, community teaching assistants, feature lecturers and students. Weekly events were run chaired by a student within the community, and set to different time zones to encourage participation.

Another issue worthy of discussion in relation to the AI Planning MOOC is the teacher time commitment that was given not just to its development, but also – and contrary to some early expectations of MOOC performance – to its delivery. The University of Edinburgh report on their first-wave MOOCs estimates that ‘around 30 days of academic (faculty) time is required for a 5-6 week MOOC, plus support and coordination time and direct costs (mainly video production and copyright clearance)’. Professor Tate noted that the AI Planning MOOC took more time than this average and commented on this time commitment in the following terms:

It was a bit too intense what we did, too heavy a workload as well. We really were very actively there… [it was] challenging in terms of time management…I’m still getting complaints about the 6 months we didn’t get on holiday!
…
You’ve got to appreciate how long this stuff takes… Be prepared to continue to engage and be part of the community while you’re doing it.

This significant time commitment is not viewed negatively by the academic staff delivering these early MOOCs – in fact, the teachers we spoke to tended to describe the time-intensive interactions with their courses as energising and addictive. For Professor Tate ‘it was a stimulating time, stimulating rather than stressful’. However, as MOOCs become more mainstream components of academic activity institutions will need to find ways of accounting for the workload they imply, particularly given that MOOC teachers in general appear to want to be highly visible and active in their courses, rather than merely video presences:

It’s not about “the material is there and you guys need to just get on with it”, when I’ve heard those comments about how we can reduce the cost of doing this, I just don’t see this. I think we’ve got to be actively involved and be seen to be actively involved as the teachers on the course.

---

Word cloud generated from interview transcript: The social aspects feature as strongly as the subject material in discussion with Professor Tate on what drove the AI Planning MOOC design.

---

Transcript of Interview of Austin Tate with Siân Bayne and Jen Ross on 24th September 2013

I’ve always considered myself as a researcher who teaches, rather than a teacher who happens to do research, so our MOOC is a very specialised module. It’s about AI planning, not the wider topic of AI itself. So it was meant to be for people who were going to do research or development in specialised areas of AI. Edinburgh’s quite well known as being at the top of research in AI planning but for a long time we didn’t do any teaching in it. But recently my colleague Gerhard Wickler has been giving a course on AI planning with some inputs from myself on AI planning applications and the context in which planning is put to use. We decided to work together to create this MOOC when the opportunity arose in mid 2012. It was also a way to get to know more about distance education and online course delivery, something I am helping colleagues with in the School of Informatics.

Content

We have a lot of materials for online, occasional use. We were already thinking about how to package that better, particularly as I come to retirement age. And some material we’ve got that we didn’t want to lose, in particular we’ve got materials that even some of the originators haven’t got any more – people at NASA and so on – and we wanted to try and make sure some of that was brought together. So that’s the motivation – packaging it well for others to make use of, and the broader we can disseminate that the better.

We wanted to be involved in the MOOC experiment, rather than just watching it. And also, the background of having the material.

Levels of Participation – Communities

I was very keen that we didn’t just have a technical, programming oriented typical computing course. I wanted people to find out how you could, and why you might apply AI planning methods. We wanted people to understand the application, even if they didn’t actually do the technical work. So we set out to design the course so it had these multiple levels, we called them ‘levels of participation’. We had three levels.

I was only able to do it because Gerhard was keen to do it as well; it would’ve been too much work to put all that together on my own, and it wouldn’t have been as much fun to be honest.

There were three levels of participation. Awareness level, and that’s the interesting one to me as it’s quite different, and then there’s the foundation level, so think of that as the core course, so you’ve got to understand the algorithms and at the very least be able to hand simulate how they would run and be able to really understand the programming level. Then there was another one called Performance Level, which was primarily for people who had access online to programming environments, so it wouldn’t suit everyone. We added something into that Performance level to try to get people into being able to score a Performance level award without programming – we called that the creative challenge. It was the creation of a digital artifact but we deliberately left it open ended, like a video, or a prezi, or a study portfolio… to get just into the scoring category for Performance award. You could actually pass without doing any programming at all. And that was a deliberate decision.

Awareness level interested me because it meant you could get to people who were never going to be programmers, they weren’t interested in programming. But they were interested in what AI Planning might do. A nice example is there were actually two authors on the course. One thing I found really nice is that we’d got people who were definitely not programmers, they were interested in science, but wanted to know what an AI entity that planned automatically might look like. And there were two science fiction authors that wrote short stories that tried to inform their writing better that took that course.

You could get awareness level by scoring on a questionnaire style, multiple choice…if you understood the lectures, all the answers were there. So anybody who took care… could get the awareness level of the course.

The core of the course was called Foundation level. You needed a mid-score, you had to really understand the algorithms, answer questions that were technically demanding, that we’d expect of Edinburgh MSc students.

Assessments – Platform

For the Creative Challenge, we had two students who just submitted a pointer to a Spielberg’s AI film clip on YouTube. I did discuss this with them directly, as it clearly didn’t meet the requirements, one said “I’ll try again” but neither of them bothered to do so. I did speak to Coursera about whether I could do anything with the platform [removing the 10% score] but the platform just wouldn’t let me. Unless it’s peer reviewed, or peer assessed, or you’ve done it with one of the automatically graded assignments…you can’t change it. The only way I could take off the 10% [for the creative challenge] was to exclude them from the course. So I did end up excluding them both just to show we were serious about it. That was the kind of an indication that we won’t just let it go and I think that’s important for the future, we don’t want to be seen as just not bothering when we’re doing the assessment.

Content

We have technical content that is quite rare and collected over a long period. I’ve collected material for instance on Deep Space 1, which was an autonomous spacecraft controlled with an AI planner, and it was the first ever spacecraft to be controlled with an onboard system (earlier we had created plans for a flying spacecraft, but they were generated by a ground based AI planner). But even JPL didn’t keep a lot of that material… Even this morning I’ve been importing a 3D model of Deep Space 1 to put into OpenSim and take some photographs of it so I can improve the quality of the images for the next run of the MOOC. That just shows what kind of care with materials and details which I consider important.

3D model of NASA’s Deep Space 1:
the first spacecraft to be controlled by an onboard AI planner

Global Community Building

I think it’s worth looking at what we were trying to do, with the community we were trying to create…. I wanted it to be as if you were coming to Edinburgh and studying AI planning at Edinburgh, so when you go away you understand the Edinburgh planning work within a global context.

We deliberately wanted to make sure people understood different approaches…we wanted those people who have differing approaches to come in and teach with us on the MOOC. And we did that through feature videos – we did 6 – and that allowed us to bring in people who held an alternative view.

Each week we had a feature video which people could see, and then interact with the person involved by having them on [the MOOC] as a community TA. I got each of them to introduced themselves…and they were typically active for about 10 days, 2 weeks. The good thing was the quality of the answers – sort of direct chat…. They were asking about something to do with robot control, and this guy said ‘I’ve just spoken to the guy who was driving Curiosity now, and he tells me they are using a variant of this algorithm, and it isn’t the one I thought’. That was the kind of thing, he’d contact the folks at JPL and talked to the guy who was driving Curiosity on Mars….it’s that kind of feeling that you’re part of a community that I want to get, that you could ask that kind of question.

A guest lecture from Nils Nilsson:
founding figure in AI research and Stanford professor

The AI Planning MOOC had 6 guest lecturers for each week and for additional study opportunities:

Guest Lecturers:
Prof. Nils Nilson (Stanford University, USA),
Dr. David Wilkins (SRI International, USA),
Prof. Jörg Hoffmann (University of Saarland, Germany),
Dr. Brad Clement (NASA JPL, USA),
Dr. Alex Champandard (AiGameDev, Austria),
Dr. Arturo Gonzalez-Ferrer (University of Granada, Spain and University of Haifa, Israel)

Teacher Belief – Course Design

[on teaching philosophy] I like projects, I like things where you put a lot of things together, I like multi-person things…. The bits I was doing on this course….the way we pulled people in, I think that reflects my interest in collaborative, joint things – doing things together. That’s why I like the creative challenges, I like people producing videos and then people commenting on them, feeding back and then building on something other people have said. I’m not so keen on people going way and doing exercises, beating their head against the screen because it’s hard, for six weeks, and then not getting the last two points. I’m not so interested in that. I think if you can get people talking, get people creating their own materials and then sharing those materials with others that’s good. So a lot of the ideas we put in were to encourage that kind of collaboration.

cMOOC/xMOOC

[on the content and pacing] There is a very definite xMOOC element of this but then the whole framework of it was definitely conceived as, and run as, a community of people interested in a common topic, and working together and exploring that space together, talking together about it and using what they were learning as a way of getting into it a little bit better.

[Some people were asking for earlier availability of and more programming exercises]

Time Commitment – Teacher Visibility

We were quite hands-on, we’re going to experiment in session 2 with a little bit less direct involvement. I’m going to experiment with that for my own involvement next time. It was a bit too intense what we did, too heavy a workload as well. We really were very actively there.

We did have 2 paid TAS. Myself and Gerhard were on every day, weekends included. And if queries were not being answered for a while I’d bring them to the attention of others. We did make sure that every issue that was raised did have an answer eventually. Several times we deliberately asked the TAs to take things on so that we didn’t seem so immediately responsive. I think there was a bit of overkill… we need to back off a little. It’s not about ‘the material’s there and you guys need to just get on with it’, when I’ve heard those comments about how we can reduce the cost of doing this…I just don’t see this. I think we’ve got to be actively involved and be seen to be actively involved as the teachers on the course. So we will be doing that, but I don’t want to be as active on the discussion forums, I want that to be something the TAs do. We’ll come in to address issue that can’t be immediately resolved by the community itself. Next time I’ll back off the discussion forum and try to get more going on in the Virtual World discussion area.

Virtual World Meeting Space

AI Planning MOOC Class Meeting in the Virtual World Second Life, with a student discussing their “Creative Challenge” digital artifact on AI Planning in healthcare.

The AI Planning MOOC also included use of a virtual world platform, Second Life™, for meetings between instructors, community teaching assistants, feature lecturers and students. Weekly events were run chaired by a student within the community, and set to different time zones to encourage participation.

Time Commitment

I think it was a stimulating time… stimulating rather than stressful. But challenging in terms of time management…I’m still getting complaints from my family about the 6 months we didn’t get on holiday – we didn’t get away last summer! So there’s time pressures, and stress in some sense. But stimulating would be the word I’d use…. Doing things in different ways with interesting people, meeting interesting people on the MOOC…suits my kind of style of working anyway.

Concluding

I would caution other MOOC teachers not to underestimate the amount of time that needs to be spent on giving the course and preparation. Too many people are saying we don’t need to be as involved in the giving of it as the preparation. I don’t see that. I would like to think our MOOCs could be seen as ways to interact with academics at Edinburgh University, on interesting topics, while you’re learning. And not just ‘Oh they’ve stuck some stuff online, and we never saw the teacher’.

You’ve got to appreciate how long this stuff takes… Be prepared to continue to engage and be part of the community while you’re doing it.