Response to “Agile Methodologies in Large-Scale Projects: A Recipe for Disaster”

Introduction

This is a response to the article “Agile Methodologies in Large-Scale Projects: A Recipe for Disaster”. In this article the author uses analysis of the agile manifesto to provide supporting arguments for the claim that the use of agile development techniques in large-scale software development is “infeasible”. The article concludes that agile development techniques have no place in the development of large software. I aim to provide some counters to the arguments made against agile development and convince you all that it should always at least be considered.

Agile Development lacks professionalism……

The author suggests that a lack of documentation can be detrimental to the level of professionalism a project may appear to have. They call for the use of documentation as a means of providing an explanation for why a particular project may have failed. This point is validated using the failure of large scale government projects as an example.
Firstly, I would like to suggest that the use a government project as an example may be a little inappropriate. I feel that the levels of bureaucracy and public scrutiny involved in such projects makes them considerably different from private sector projects and as such they should be approached in a different manner (please see my previous article for more detail).

Furthermore, I would put to the author that a project which fails (due to running over budget or over time) but has extensive documentation could also appear unprofessional to clients. The failure of such a project raises the question “why spend so much time and money on extensive documentation for a system which doesn’t even work?” Therefore surely the practice of delivering working code over small iterations is more satisfying for everyone?

Don’t worry there will be documents

If agile development is used correctly then there should be excellent communication between everyone involved in the project, especially the development team and the client. So even in the event of a failure there will be a “hierarchy of responsibility” as all the people involved should know what is going on. Furthermore the deliverables that are produced by the short, fast iterations that agile caters for should be sufficient if any investigation into the failure of the project is required.

Unlike the author seems to suggest, agile development does not abandon all formal documentation. In fact documentation is still encouraged with agile methodologies it is just that development of documents is more likely to occur at the end of the development cycle in order to avoid wasting time and money creating documents which will no doubt need to be altered as the project goes forward[1]. Any documentation produced will be concise and simple yet detailed enough to include the vital information.

Flexibility is key

While I agree with the notion that for very large scale projects it is important to have a clear idea of the initial scope of the system. I reject the idea that this should come in the form of a static requirements document. In fact I would say that for a very large project it is vital that system specifications should be changeable. This is because it is simply not realistic to expect the customer to be aware of everything they require from a system at the very beginning of the development cycle. In addition to this large scale systems which are the subject of the author’s argument often take a number of years to develop and as such it is highly likely that the needs of the client will alter during the development process. It would be irresponsible for a development team to deliver a product which they know does not meet the needs of the client simply because these needs were not obvious from the beginning. I would say that such projects should be considered failures under the justification that they are not “fit for purpose”.

Additionally the author presents the idea that “each intricate detail” of a project should be carefully thought out before any actual implementation is done. I feel that this is highly unrealistic and would lead to a nightmarish development process consisting of never ending requirements capture and design phases. It is virtually impossible to create a perfect design of all aspects of the system, no matter what happens there will always be some parts of any initial design that require alteration (or even a complete re-think). This is especially true for large scale projects which could be made up of numerous components. What this means is that investing large amounts of time and money into meticulous planning before any implementation is an extremely inefficient practise as it is almost certain that such plans will change. In contrast agile development has design and implementation working side by side resulting in less time wasted on attempting to design the whole system at once.

I agree with the author’s observation that making changes in large scale projects is not a simple task and that any changes made will create a number of subsequent changes. However I do not feel that these added complications are reason enough to avoid making the changes. Such complications can be avoided if everyone involved in the project communicates with each other appropriately (one the key the concepts of agile)[2].

Similarly it is suggested that developing software using this static approach protects developers from being overworked by demanding clients who continuously request new features. This may be true in the sense that at the end of a project it will be easy to evaluate success or failure based on the original requirements. However I think it is important to remember that the software developers are providing a service which clients are paying for, therefore if a client wants a new feature or a change they should have the option. If you wish to make changes to an order you placed in a restaurant the waiter will not deny you as long as you are willing to cover any extra costs. Similarly, the responsibility for any substantial budget or timescale penalties due to changes falls directly on the client.

Conclusion

In conclusion, developing large scale software projects is an extremely difficult process. Such projects have a history of failure[3]. However this poor track record is certainly not a reason to abandon the use of agile development techniques. If anything it is actually a reason to avoid traditional techniques as much as possible as they have proven to be problematic. The criticisms of agile development made by the author of “Agile Methodologies in Large-Scale Projects: A Recipe for Disaster” are understandable however after some thought these concerns can be disregarded. Unlike agile development itself which may not be perfect but at least it makes a conscious effort to make a step towards a world in which project failure rates are significantly lower. For this reason alone it should at least be considered as a methodology for any software project large or small.

References

1. http://www.agilemodeling.com/essays/agileDocumentationBestPractices.htm
2. http://agilemanifesto.org/
3. http://en.wikipedia.org/wiki/List_of_failed_and_overbudget_custom_software_projects