Agile Methodologies in Large Scale Projects: Not a Recipe For Disaster

This article is a response to “Agile Methodologies in Large Scale Projects: A Recipe for Disaster”. The author’s claim that “agile development has no place” in real-world software development is solely based on the values outlined in the Agile manifesto. Values are concepts that drive choices. Principles define the manner in which these values are carried out. By only considering the values, ignoring the principles and providing no empirical evidence the author fails to convince me that agile development cannot be successfully applied in real-world software development. In this blog post I will counter some of the main points made by the author to prove that agile development does have a place in real-world software development.

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

The values outlined in the Agile manifesto are very broad guidelines about what gets priority in the development of software. The manifesto clearly states that the notions on the left-hand side are given more priority than those on the right-hand side. This is not ‘infeasible’ in large scale software development as suggested by the author. While there are challenges in incorporating Agile practices into large teams, empirical evidence and anecdotal evidence from professional developers shows that not only is it feasible but it produces better results than traditional approaches.

A study conducted showed that one-third of all development is run with Agile methods and two-thirds apply a Waterfall model. While the use of Agile methods is still a minority compared to the use of the Waterfall model,  23% of these Agile projects involve teams with more than 200 people. The success rates presented in the following points prove that is not ‘infeasible’ to use Agile methods in large scale software development.

The author claims that:

“Lack of formal documentation or planning could appear unprofessional to clients”.

Following Agile development does not mean that the development process lacks formal documentation or planning. The development model just gives working software more priority than producing comprehensive documentation. Agile development aims to minimise documentation and aims to ensure that correct use of the model will produce a working software for the client. In his article, “Essential, Valuable, Timely Documentation”, Ashish Sharma states that “Documentation is an important part of every system, … , but comprehensive documentation as such does not ensure project success. In fact, it increases your chance of failure”.  I would agree that this evaluation is entirely correct. Documentation is crucial for both clients and developers to aid in understanding the software and for future maintenance. However it is certainly not more important than producing a working piece of software.

Furthermore, the author’s use of a failed government project as an example where comprehensive documentation will help them with accountability is highly exaggerated (as agreed by the author). The author concludes by saying that even if the scenario is exaggerated the outcome “would almost certainly be the same” in any other case. That is that the Agile model does not produce enough documentation that could be used to justify why a project failed. I think that this is incorrect. As pointed out in the above paragraph, the use of the Agile model does not sacrifice documentation. It simply aims to keep it at a minimum. Hence the author’s claim that a client would not have any proper documentation other than those on “napkins” is unfounded and would not result in the same outcome as having no documentation at all.

Another claim made by the author is that:

“with a detailed plan from the beginning that clearly defines the scope of the project, the requirements it must capture and other details … the project will be far more likely to be completed on time and within budget”.

Empirical evidence proves that using the Agile development model can lead to a 37% reduction in time-to-market with a 26% median cost reduction. This is proof that a more traditional model need not be followed in order to deliver a working piece of software on time or within budget. In addition to this, the study cites the Waterfall model as one of the reasons why between 50 and 70 percent of software project fails. I agree with this as the Waterfall model fails to give developers a good idea of the requirements at the start. This leads to developers working on ideas rather than facts and the scope of what needs to be completed is not realised till the end of the development cycle.

This moves onto another point made by the author that:

“the majority of this communication [between clients and developers]  should take place before the implementation begins”

I do not believe that this should be the case at all. When the Agile model is used development is an an incremental and iterative process. It allows the developers to acknowledge and fully understand the evolving needs of the clients rather than setting the ‘perceived’ requirements in stone at the start of the process as is the case with the Waterfall model. Acknowledging that a customers requirements will evolve and actively inviting them to participate in the development process will allow developers to verify the solution at an early stage and fix mistakes early on.

The Agile model has a itself to produce greater customer satisfaction. 78% of those who adopt it report greater stakeholder satisfaction and 66% report better alignment between business and IT. This proves that it giving priority to customer collaboration and responding to change over contract negotiation and following a plan. So I find the author’s claim that clients requesting changes to the product that they are paying for mid-way through the project means that they are not treating it seriously rather ridiculous. Using an iterative development process, such as Agile, will allow for any changes to be easily incorporated into the software  process without massive ‘ripple effects’ as claimed by the ‘author’.

In conclusion, the increase in customer satisfaction through lower costs, improved quality of software and shorter time-to-market proves that the Agile model does indeed have a place in real-world software development. In addition to this by not fully exploring the applications of the Agile values the author misses out on the advantages of the Agile model and deems it ‘infeasible’ in large scale software development for all the wrong reasons.