Agile Methodologies in Large-Scale Projects: A Recipe for Disaster

The four “Agile Values” written within the Agile Manifesto are as follows:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The values above are to be interpreted in such a way as to value the notions on the left-hand side of the “over” more than those on the right hand side. I propose, however, that in large-scale software development, this is infeasible, and that the left-hand side of each of these values (which are more akin to traditional development models) are far more appropriate in such projects.

Working Software over Comprehensive Documentation

Lack of formal documentation or planning could appear unprofessional to clients, who may deem the project to be too risky. In large-scale projects where clients – who may even be government bodies – are likely to be spending upwards of millions of pound. Therefore, there needs to be a formal hierarchy of responsibility in the event that the project fails.

In a large development team over the duration of the project, informal means of documentation can be swallowed-up and forgotten about. It can also make the future maintenance of the project far more difficult – especially if new developers have joined your team since the project’s creation.

If, for example, a multi-billion pound software project for use in schools signed-off by the government were to catastrophically fail and be considered a write-off , members of the public and political backbenchers would conduct a scathing witch-hunt into why their hard-earned tax money had been squandered. Either way, the government is going to
be subject to ridicule. But the damage can be limited, however, if formal documentation of detailed plans can be produced to show that deep investigation into the project’s scope and requirements had been extensively carried out. If however, the government were to metaphorically shrug its shoulders and say, “We don’t have any proper formal documents to show you: however, we did meet up with a few of the developers for a brain-storming session using a whiteboard and napkins,” heads would certainly roll.

Of course, I am slightly exaggerating the realities here, but I feel that the outcome would almost certainly be the same.

Individuals and Interactions over Processes and Tools

There is a reason that set processes and tools are used in large-scale software development: they work. They provide an established set of rules and standards for developers to abide by, ensuring that the software development process is as efficient and consistent as possible. In allowing the individual personalities of developers to encroach upon the project, standards can begin to slip. Sometimes, there is no need for innovation or creativity in solving a particular problem. Take the notion of code reuse as an example: it can sometimes be easier and far more time-efficient to simply re-use an existing block of code from a repository, where crafting a new solution – which may indeed be more elegant
or objectively better than the current solution – may only provide marginal bene fits to the project in the long run.

At such a large scale, I feel that innovation can only lead to chaos and serve to hamper the project – if everyone was to decide to innovate and deviate from the standards set out by management, any sense of consistency within the project would surely be lost. It may sound like I am advocating the stifling of originality and personality within large-scale projects: to some extent, I am, but when the stakes are so high (in terms of the money involved in large-scale projects), it seems more responsible to use tried-and-tested methods of development as opposed to new and original practices and ideas that may contain hidden flaws.

Customer Collaboration over Contract Negotiation

The use of an iron-clad contract in a large-scale software development project is crucial. Don’t get me wrong: I can of course see the bene fits of customer interaction and ensuring that you are delivering a product that is exactly what the client is looking for: however, the more you collaborate with a customer during the development process, the greater the opportunity you leave for the customer to take advantage of you. The more you allow the client to alter the original plans, the longer the project will take and costs will rise. Of course, it would be the customer who would need to meet these costs, not the development company. I feel, however, that with a detailed plan from the beginning that
clearly defi nes the scope of the project, the requirements it must capture and other details such as the deadline, that the project will be far more likely to be completed on time and within budget.

Understand that I’m not saying that no interaction between the client and developers should take place: I am merely saying that the majority of this communication should take place before the implementation begins. In a large-scale project where the goal is constantly changing and looks far from the initial concept envisaged, there comes a point when a developer needs to take a stand and say, “You are asking too much of us – you asked us to do *this*, and now you want us to do *that* – the complete opposite?”

In order to alleviate this situation, more time should be dedicated initially to planning each intricate detail of the project, before the actual development work begins. Take as much time as necessary, ensuring both you and the client are happy and in agreement. A formal contract can then be compiled.

The point that I am trying to get across here is that all too often the developers of large-scale projects are all too ready to bend over backwards to meet the desires of clients whenever a new idea enters their minds. The compilation of a formal contract would protect the developers. It may be acceptable – and indeed, manageable – in a smaller project to allow for changes to be requested throughout the development cycle. In a large-scale project, however, even a small number of changes could create a huge amount of extra work – creating a “ripple e ffect” impacting upon various di fferent areas of the
project.

Responding to Change over Following a Plan

In smaller-scale projects, it may be feasible to react to changes at the request of the client (for example, a change in requirements). With quick and efficient communication, changes can propagate quickly. Within a large-scale software development project, however, with many di fferent levels for information to pass through, communication can become bogged-down, leading to delays in development. In following a stead-fast plan, communication channels and practices throughout the development team can be established early on. This helps to ensure a consistent flow of information around the  team: with no dramatic changes requested, no speci fic area of the development team will come under intense pressure, and no “ripple eff ect” takes place. Following a plan also  helps to keep the project on track in terms of time and budget: making changes is expensive.

Having a plan to stick to allows the project’s progress to be measured accurately – each stage in the development cycle can almost be “ticked-off ” after completion. When change occurs, it can sometimes be hard to calculate if the project is any further forward (or indeed behind) in terms of its development. Some changes can also spawn further changes, and create even more work not envisaged when the team initially decided that changes were indeed necessary.

There are some changes that cannot be helped, however. Changes to industry standards or APIs used in development may mean that fundamental changes may be necessary. In times like these, there is no option but to make alterations to the initial plan. Of course, I completely agree and fi nd this acceptable. However, changes to a requirements specification or a request to modify the behaviour of a range of already-implemented features mid-way through a project’s development I find to be less acceptable.
Large-scale software development projects are a serious undertaking – requiring huge investments of time and money – and should be treated as such by clients.

Concluding Remarks

In large-scale software development projects where clients have invested large sums of money and taken huge risks, I feel that it is only responsible to consider a more traditional approach to development. Trying to cut corners in choosing not to write formal documentation could be disastrous for the future of the project, and allowing clients to get too involved in the development  could allow the project to descend into chaos. A tried-and-tested traditional methodology, making use of set processes, tools, formal documentation and binding contracts help to ensure the success of the project – both in terms of deadlines and expenditure.

Large-scale development projects are serious business: agile development has no place here.

References

http://agilemanifesto.org/