Incentives Can Be Poisonous, So How Do We Fix Them?

In “Incentives Poison Extreme Programming Values”, Craig Innes explored the dangerous effect external incentives can have on the decision to use an Agile methodology, using the third year SDP project as an example of such a scenario. In this response, I intend to expand on his article by explaining why these incentives existed and how their goals can be met in a less damaging way. I will then generalize principles from this example that can apply to a variety of situations (work environment, assignment structure) in which authorities need to monitor or incentivize work, yet do not want to disrupt the Agile process.

Milestones, friendlies, and stack ranking: why were they used?

In order to understand how to improve SDP, we have to understand why the course implemented milestones, competitive friendlies, and zero-sum marking. Only then can we redesign the course to achieve the same ends without interfering with the Agile process.

The milestones in SDP primarily served as a way to measure the progress of student teams, but also as explicit short-term goals to aid team planning. Measuring the progress of teams not only allowed for intervention if progress was slow, but also allowed for a relatively standardized set of grading against an objective measure of success. The fixed goals also allowed teams of students who were new to major project planning to have some idea of what capabilities they should have at certain dates.

The competitive nature of friendlies was another incentive to ensure that teams took the opportunity seriously. If doing poorly meant a worse seeding in the final tournament, teams would be incentived to work at performing well at these events.

Finally, the competitive marking at milestones ensured that individuals did not freeride off of the group’s efforts, while at the same time it prevented the group from conspiring to give each other high marks. The limited number of high marks ensured that the group could not grant everyone an undeserved high score, and the resulting competition incentived denying freeriders a high mark.

In summary:

  • Milestones and competitive friendlies were meant to ensure that teams took their responsibilities seriously, and provide fixed, short-term goals for members to target.
  • Competitive marking was meant to both ensure individual effort, and defend against widespread mark-fixing within groups.

How do we fix it?

Milestones are a serious problem, as they are critical to measuring and maintaining the team’s progress. Yet this can be achieved, as Craig himself suggested in his reply to a comment, with group’s setting their own milestones. Each group has a mentor, a student who has previously taken SDP and who signs off on major decisions (such as allocation of marks), who could help the members of a group decide on a reasonable milestone to target next. Teams could then adjust their milestones to measure progress which is relevant to them – some teams may want the traditional feature-by-feature approach, while others might adopt a more Agile strategy of frequent, iterative releases. Either way, this approach would grant much more flexibility to teams, while retaining a grounded approach thanks to the mentor’s input.

The solution for competitive friendlies seems obvious enough – make them noncompetitive, or even end them all together! Tournament seeding can be random, and teams will have an existing incentive to take any opportunity to simulate the conditions of the final tournament seriously. Friendlies can either be organized by the course organizers as mandatory or encouraged matches with no stakes, or it can be left up the teams to organize their own friendlies.

Finally, zero-sum marking is both understandable (the organizers want to avoid collusion and freeriding) and deeply perverse (similar to Microsoft’s “Stack Ranking”, creates an environment of mistrust and hostility, incentives hoarding code and “low hanging fruit”). This problem is much harder to patch than the previous ones, as it is a symptom of a larger issue with a dissonance between the intent and practice of SDP. The easiest patch is to simply remove the limitation on high marks, and give more authority to mentors and the course organizers to review and modify the marks given. This does generate more work for those in charge of the course, but allows teams to focus directly on rewarding members as best makes sense to them.

Conclusion

While SDP’s milestones, competitive friendlies, and stack ranking seriously interfered with Agile approaches, they can be revised or eliminated in a way that allows for a variety of team methodologies. Even in an environment like SDP, which seeks to assign each student a grade to determine their grasp of the material, structures and incentives which undermine an Agile approach can be dealt with if the course organizers are willing to meet students halfway.