Incentives Poison Extreme Programming Values

Agile methods such as extreme programming outline a set of principles and practices for developing software projects. They are ideal for small projects with volatile requirements as they favour flexibility and working software over rigorous planning and extensive documentation.

Reading over the list of rules for Extreme Programming, I couldn’t help but agree that many of the rules seemed like brilliant ways to ensure the quality and success of a project:

  • Having developers take collective ownership of the codebase
  • Integrating code with the main project build often
  • Writing unit tests before writing the code
  • Making frequent, small releases
  • Refactoring code whenever possible
  • …and many more.

What surprised me however was that on both the Extreme Programming site and the Agile Manifesto page, little is said about how to effectively get people to follow these principles, or systems which might come in conflict with these principles. It is as if the authors believe that if they just pick the right set of principles, then implementing them in any context will be trivial.donkeycarrotcliff

My argument is that even if you intend to adhere to extreme programming’s principles, incentive structures put in place by the organization in which you work can have a strong impact on whether you follow them or not. By relating my personal experience of such external incentives, I will hopefully convince you of just how powerful an influence they can have, and to be more aware of how incentives may be motivating your behaviour in the future.

As part of the software development course at the University of Edinburgh, I was placed in a team of 12 other students with the task of creating an autonomous, football-playing robot in a single semester. The robot would use images of the pitch received via bluetooth from a web camera in order to make tactical gameplay decisions. This was a project with vague requirements which would require lots of rapid prototyping to be successful, so it seemed like a perfect fit for the extreme programming methodology. We agreed to commit fully – Pair programming, Trello cards for user stories, stand up meetings, merciless re-factoring, shared code ownership. Everything

We soon found however that the assessment criteria would pressure us into abandoning many of these principles.

Incompatible, Mandatory Milestones

To ensure that teams were making tangible progress on their project as the semester progressed, the course organizers set fortnightly milestones on which a percentage of the teams total mark for the course was assessed. This seems like a good incentive to encourage students to work on the project, but two key features of the system led to negative behaviour:

  •  The milestones were set by the course organisers at the beginning of the semester  and didn’t change as the semester progressed.
  • Regardless of other functionality your team’s robot had, if it couldn’t pass the milestone, then the entire team got zero marks.

This created a conflict between following our Extreme Programming principles and achieving the highest course mark:

In principle we would like to write unit tests for all our code, refactor old modules and ensure overall system quality by following a coding standard.

In reality, if we can’t hack together some code to make our robot intercept an oncoming ball by Friday, we all lose 5% of our mark.

When pushed for time, we would always choose to hack together a façade of functionality to pass the current week’s milestone rather than follow solid development principles. We were rewarded for this behaviour with a higher course mark, but the code used for the milestone was often of such poor quality that it was of no use in the main project’s goal of building a quality football playing robot.

The conclusion we reached from this experience was that milestones are only useful if they correlate with the end goal of producing a quality final product. Being forced to meet arbitrary milestones often means developers have to choose between producing principled, quality code and meeting an arbitrary deadline.

Adding a competition element to frequent releases

Rather than fixed milestones, Extreme Programming advocates small, frequent releases which gradually display new functionality. Our robot football course actually had such a system in the form of fortnightly “friendly matches”.

Every two weeks, every team on the course would take part in a friendly tournament where 2 opposing teams would use the current version of their robot to play against each other in a game of one on one football. This ticks a lot of boxes in the extreme programming framework: It allows teams to demonstrate visible progress to their clients (the course markers) and gives teams a chance to get feedback on their systems by exposing its current design flaws.

Again however, an incentive was added which corrupted things a little. Instead of being consequence free matches, the position in which you finished in the previous fortnight’s tournament “seeded” you for the next tournament. In other words, if your robot was in the top 3, you would be allowed to “skip” the first round in the next tournament. Additionally, your robot’s position in the final tournament contributed a significant portion to your course mark, so being able to skip a round in this tournament was a massive reward.

To understand why this incentive is so insidious, it is important to recognise that the value of showcasing frequent iterations under the extreme programming methodology is to expose flaws in your system and get the necessary feedback to improve them. This incentive warps this goal so that the value chiefly lies in beating as many teams as possible. Teams would often accomplish this by explicitly coding gameplay strategies which they knew the other teams couldn’t handle at their current stage of development instead of working on the problems in their final overall system. In this environment, it is optimal to hide your system’s flaws rather than expose them.

Ranking Group Participation

Even without external influence, enforcing extreme programming principles such as pair programming and shared code ownership requires willpower. In a disciplined, trusting environment, it can certainly be done, but in an environment which forces you to consider your participation in the project as a competition against others in your team, it is nearly impossible.

In order to encourage participation from every member of a team , each team was subject to a weekly performance review, where students were given a score to grade their level of contribution and made up a small percentage of their final course mark . A score of 5 meant you had contributed exceptionally, while a 1 meant your contribution was minimal.  The key detail of this incentive system was that not everyone in the team was allowed to receive a high mark . Students were ranked from those who contributed most to those who contributed least and assigned a score respectively.

stackRankingThis was perhaps the most poisonous incentive of all, as it encouraged us to think of our teammates not as collaborators, but as competitors. As a result, it made numerous extreme programming principles difficult to uphold:

  • Collective ownership of code – “Sharing code” now means others may be able to take credit for part of your contribution . The optimal strategy to get a high rank is to take sole ownership of a module in order to say in the meeting that you were “entirely responsible for progress on the planning system this week”.
  • Code the unit tests first / Refactor whenever possible – Again, you are forced to consider what sounds better in a meeting:  “I added functionality so that the robot can now kick while moving” or “I refactored some code which was already working so that it has a clearer structure”.
  • Integrate Often – It now becomes advantageous to sometimes not integrate your work with the current build. This way, you can create the impression that you have made a large amount of progress by explaining all the new code you have just “not yet pushed to the main build”. If you were to continually integrate your work, there would be clear evidence of what you had and had not done.

Communication and trust are vital to upholding many agile practices, so incentives such as this which create an environment of distrust will suffocate such practices.

This system of “Stack Ranking” developers has become notorious in industry because of its use by many high profile companies. Many reports claim that such a system was responsible for the majority of problems at Microsoft over the last decade.

Conclusion

While agreeing to commit to an agile process such as Extreme Programming (or indeed any process) is a positive step towards a successful project,  I have shown through relaying my own experiences that the mere choice of principle is not the only factor at play. Teams must be disciplined in adhering to their chosen principles and diligent in identifying structures which pressure them to do otherwise.

I hope I have emphasised just how powerful and poisonous such influences can be, and have given enough examples to encourage you to look out for similar systems which may influence you in the future.

5 thoughts on “Incentives Poison Extreme Programming Values”

  1. Agree absolutely, and I think this is a great article. I’m curious – do you have any thoughts on how they should have run SDP to avoid this problem? Particularly with the milestones, as I think they’d be the hardest element to replace while still ensuring that teams are on-track.

    1. Hi, thanks for the response.

      I Agree, it is very easy to criticize what is wrong with these systems, but much harder to suggest stronger alternatives. This is especially true in a university environment where if a piece of work isn’t assessed, it is very likely that a lot of students just won’t do it.

      One thing I think might be worth trying is letting teams set their own milestones. This could work by each team having fortnightly meetings with their mentors/assessors in which they discuss a reasonable set of targets to hit for the next two weeks in advance. This has a number of advantages over the current model:

      – Teams have an active say in their own milestones.
      – The criteria for meeting these milestones is much less ambigious, as teams can have a discussion about what exactly their system needs to do in order to successfully achieve the current milestone
      – Targets can be *updated* based on the performance of the team. If last fortnight the milestones were far too ambitious and difficult, they can cut the scope of the milestone in the next fortnight.

  2. I believe that your experience, regardless of your final marks, is invaluable because it is indicative of the situation that exists in industry. Today it was just a course mark, tomorrow will be a promotion.

    Although I agree with almost everything you mention I am a bit skeptical towards your attitude about the environment (milestones, competition, participation ranking) you participated in. In a real scenario, the customer or your manager may have more “crazy” or unfair demands from you. If there is one lesson that you have to take out of this project is that you should not be dogmatic when following a methodology and that you should always try to adjust it in order to best fit your needs.

    Panos

    1. Hi Panos,

      Thanks for the reply. I am glad you got something out of reading my article but am a bit concerned by the final conclusion you have drawn from it. In the final line of your comment you state that the lesson to learn is that “You should not be dogmatic when following a methodology and should always try to adjust it in order to best fit your needs”. Are you suggesting that the best strategy for teams in the article would be to alter their behaviour to be more in line with the incentives provided?

      I was trying to argue the opposite: That when you are faced with a lot of incentives which may not actually line up with producing quality software, you need to be disciplined enough to know which ones to ignore. I would be interested if you could elaborate on this. Maybe it might make for an interesting response article if not just a comment?

      1. Hi, thanks for your reply.

        What I was trying to say is that following a methodology is not a goal by itself. The methodology is the tool for achieving the goals that you set.

        First of all you should clearly define your goals. In your project the goal was software quality, your final mark or both ? This process involves identifying contradictions among the goals and eliminate them (i.e. if it is infeasible taking excellent mark and writing the best software what do you do?).

        After reaching to a viable set of goals you choose what methodology you should use and adjust it to best serve all of your goals.

        Basically that is what I like about agile methods. You can view them more as a way of thinking/acting and not as a look-up table (as opposed to their heavyweight counterparts). It is very easy (and basically it is the most common scenario) to follow only a subset of the agile principles because it best serves you.

        Panos

Comments are closed.