Engineering large scale software successfully is an extremely difficult task, as such the majority of projects result in failures . Recent times have seen the desire to offset this obvious imbalance through the emergence of Agile software development. The popularity of Agile design methodologies is ever increasing with more and more companies developing software using agile approaches such as scrum and crystal every day . Despite this the success rate of large scale software engineering remains relatively low and big project failures still crop in as news stories every so often . One of the most notorious examples of a very large scale agile development project failure has been the UK governments Universal Credit .
Universal Credit, that sounds boring!!
It is, Universal credit is the UK governments attempt to simplify the welfare system in some way by replacing six of the key mains tested benefits (Job Seekers Allowance, Child Tax Credit etc) with one payment . The reform will require the support of a large scale IT system. It is hard to tell exactly what will be required from this system, ironically some areas of gov.uk were not accessible when I wrote this. However, I believe it will include some kind of web support for calculating eligibility and payment amounts. In any case the exact requirements of the software are not important for this article which is concerned with the development process used to build it.
Why single out Universal Credit?
There have been other high profile failures of large scale government engineering projects so why focus on Universal Credit. The theme of failure in Government IT projects is all too familiar, they have a history of running overtime and over budget. However, the government envisioned a different fate for Universal Credit through the use of agile development . Unfortunately though, the development of Universal Credit has induced feelings of déjà vu by sharing two key characteristics with some of its predecessors: huge cost and late deployment .
Is Agile to blame?
The failure of Universal Credit has been very public and the reasons for it have been the subject of widespread debate. Agile development has been at the root of much of this debate in fact some people have claimed that Agile is at the root of the failure and should never have been used for this project . While others suggest that Agile is not to blame as it was the victim of misuse and misunderstanding by the people involved with Universal credit .
Both of these viewpoints certainly have merit however I feel as though the first is really just a manifestation of the second. It is clear from reading numerous articles related to the agile development and the failure of Universal Credit that from the start there was confusion about the concept of agile development. For example, despite the intention to use Agile development the project was still constrained by elements of big design up front. This major contradiction to the principles of Agile development demonstrates an alarming lack of understanding. It appears as though in the context of this project the word agile has been thrown around for the sole purpose of encouraging vendors to deliver the system as swiftly as possible. In other words Agile should not be blamed for the failure of the project as it was never Agile in the first place. The reality is that the government’s attempts to use Agile development resulted in a kind of mutated waterfall model with accelerated implementation and testing phases.
Don’t trust the feds!!!
It could be said that no appropriate development methodology exists for these large scale government projects. Certainly the continued failure of projects such as Universal Credit and the even more notorious NPFIT could lead one to this conclusion. The levels of bureaucracy and public scrutiny involved in the development of such systems separates them from large scale private sector projects. Perhaps the politics involved impacts the development process to such an extent that we cannot apply the usual mainstream software engineering practices to these systems. If this is the case then there may be a need for someone to develop some guidelines or methodology for successfully delivering large scale public sector IT projects. However I feel that the development of such guidelines is unlikely as it would involve further governmental input and their track record is not encouraging.
It is also possible that the failure of large scale government IT programs is caused by a simple case of over ambition. Both Universal Credit and NPFIT attempted to provide systems which required the software to unify different approaches to problems in an enormous scale. In the case of Universal Credit there was an attempt to replace one on one sessions people had with councilors to evaluate their welfare needs with a software system. The problem with this kind of computing is that it tries to provide too much structure to something that is inherently chaotic. In some cases people interacting with people is still very important and in these situations we cannot expect software to successfully replace human interaction.
Political agendas could be at the route of the ambitious nature of government IT projects. Popularity among the general public is paramount to any government regardless of the particular party. One way in which governments can become and remain popular is if they are seen to be making significant and positives changes. This could lead to spawning of over ambitious large IT projects where perhaps a collection of smaller more subtle changes would be more beneficial. In any case it is clear the current practices process for developing government IT projects need to be assessed if disasters such as universal credit are to be avoided in the future.