Accepting Agile: Too Many Variables, No Single Formula Exists

When taught, we are given the impression that agile and plan-driven software development methodologies contradict each other completely. However, to my mind, this is not true and the solution is to find a healthy balance of the two. This idea is conveyed in the article “Management Challenges to Implementing Agile Processes in Traditional Development Organizations”[1] by B.Boehm & R. Turner. It argues that it is difficult to integrate the agile methodologies within companies with large legacy systems. Also, the authors present many reasons and suggest possible solutions.  While agreeing with the article’s claim, I was surprised, by the wide range of challenges that need to be overcome, as you are forced to look at the big picture of the whole company. I would like to add my own observations from the industry and highlight some of the reasons why I think there is no one clear approach in integrating agile methodologies within traditional companies, as there too many different possibilities.

Restructuring: Starting from Scratch Not an Option

Agile software development principles are rather recent and innovative when being compared to other methodologies such as the text-book example Waterfall and others. Hence, many significant software projects of successful firms have foundations that are directly incompatible with agile software development, since they are difficult to refactor and unfriendly to the short development cycle of an iteration. However, this is an obvious incompatibility that needs to be handled, the two software development models are different.

A less expected factor that influences the use of agile is the size of the team and the scale of the project. Agile is known to work on small projects, with a team of a few people working to solve problems, where as the legacy systems tend to be large with a larger set of people working on it with different responsibilities. Within agile, close collaboration is needed within team members, hence the tradition of scrum meetings. This may be difficult with large teams and may take more time than expected. Also, it may cause various logistical issues within a corporation if the teams are split across countries[2]. Hence, some sort of re-organization is needed that should be tailored to the company’s needs.

As I worked in a team of a larger company that is trying to take up agile with a spin off project of slightly smaller size, the size of the team was still quite large. The team had representatives in three different time zones, however after making working hour adjustments, scrum meetings could be organized daily for the whole team to attend. Having said this, not every company can afford international calls or accustomed working hours, which once again leads to no clear one solution.

Planning: Always Expect The Unexpected

Plan-driven development is based on the idea, that the original estimations and requirements will not undergo great changes. Which is used in larger companies, as in this way it is easier to plan the business model. Unfortunately, change is often unavoidable. The agile principles try to accept these changes by developing in short iterations, without any large future vision in order to adapt to short notice changes. This is a very controversial topic, as agile principles may lead to the success of the project due to catering to the latest requirements. However, to do so in larger firms a plan has to be made in order to estimate scope, cost and length[3]. Also, multiple teams work in related products, and constant change would slow down the development cycle. A possible approach would be to plan ahead and make estimations, set a goal, and then organize sprint planning as well as development iterations in order to make the original aim come true with adjustments need to be made.

Referring back to my experience in a larger firm, one simply cannot anticipate everything from the beginning. For instance as, I was working on my project for which I had set myself clear goals on a strict timeline, I found out that there was a security issue if I took the original approach. This happened because the systems are large and different parts have different specifics. Obviously such an issue had to be addressed straight away and supplementary modules needed to be built, which took time and changed the timeline.

Unfortunately, facilitating the acceptance of constant change is difficult because it is difficult to tell the difference between a reasonable and timely change and the time and cost needed to implement the alternative solution. Hence, the call has to be made on project or even change basis.

Developers: Old Habits Die Hard

An astonishing discovery was the difficulty of making sure that software engineers understand and are willing to undergo changes. Traditionalist developers favour a more plan oriented approach due to the belief that it is more predictable and safer, while the new younger developers are willing to avoid the “crushing weight of rushing bureaucracy”[3] and accept the fast pace change in technology. It is evident that there exists a difference in generations of developers. This is because software is being built by programmers and their valuable experience with other methodologies cannot be ignored. People may find it difficult to adapt or may be stubborn and not want to accept the changes.

Agile requires a change in team dynamics and management. As in agile teams engineers tend to be “multi-taskers” rather than have a certain role, which is what many are accustomed to. In this case  merging the two ways of thinking may be difficult. The project manager has to be willing to accept that within a more agile team developers are more flexible and share tasks with respect to urgency and availability. It is their responsibility to face the Human Resources Management when it comes to assigning “roles”[1].

In the team I worked with everyone was willing to pursue the changes, but the concept of “roles” was still evident. To minimize that regular deep dive sessions are organized, where employees share their knowledge. It could, be seen that every step was made to accommodate agile within the traditional methods. However, this is only a solution, as there are too many variables to guarantee success of such an approach indifferent environments.

Stakeholders: Raising the bar

The relationship with stake-holders also needs to be taken into account, which may seem of less significance in the first place. In larger firms, that are used to plan-driven development, the requirements are agreed with stakeholders at the beginning. However, agile raises the bar: stakeholders have to be much more involved and willing to collaborate. This is challenging for both as stakeholders have to become available, even if they have other priorities, while developers have to be ready to receive feedback and may need to implement short notice changes[4].

This is handled differently by corporations, as the stakeholders differ. I was lucky to work on an internal product, for which in order to get feedback from a stake holder I could simply pick up the phone and make a call. However, once again this may not always be possible and other arrangements have to be made.

To sum up…

At university we are taught about applying method A and applying method B on separate occasions. The realisation that I have made is that the key to success is to learn to apply the best from (A+B) with respect to the given situation. However, this is not easy as there is a shockingly large amount of things to consider at an attempt to obtain the best out of two contradicting solutions. In this particular case one has to take into account the already existing software systems of the company, developers, stakeholders, the business model, team size, and available budget. All the variables have to be weighed with risk and the best approach should be reached for each situation.

[1] Management Challenges to Implementing Agile Processes in Traditional Development Organizations. B. Boehm, R. Turner. IEEE Software September/October 2005.

[2] Disadvantages of agile Development. Kelly Waters, 2007.

[3] Get ready for agile methods, with care. B. Boehm. IEEE Computer 35(1):64-69, 2002.

[4]Agile Modelling: Overcoming Requirments Modelling Challenges.