Author: s1027688

Open source software is a hot topic at the moment. More and more businesses and people are choosing to use open source products over more traditional proprietary ones. The advantages and disadvantages of using open source have been well discussed and documented. This article will analyse some of the most frequently discussed reasons for avoiding the use of open source products from three different points of view; the general public, businesses and the experts (aka computer programmers). Please understand though that I am not claiming that I myself am an expert, nor indeed am I stating that all programmers should be considered experts. I am simply stating that the specific technical knowledge and overall computing skills acquired from working on software development projects give developers a unique standing in the debate.

First, some clarifications

Before discussing further I would like to clarify exactly what is meant by the phrases; proprietary software and open source software. Proprietary software is typically distributed to users for a fee under some licensing agreement which gives them the right to use the software if and only if they uphold a set of rather restrictive conditions. These conditions are there to prevent the user from committing heinous crimes such as modifying, sharing, studying, redistributing or reverse engineering. In addition to this the source code is not available. Basically the people who distribute proprietary software want their product to be used exactly as it was intended, or else! [1].

In contrast, the source code for open source software is always made available. The software and its source code are provided to the user with a license agreement which gives them the right to modify, distribute and use the software for any reason they want. It should be noted that open-source software and free software are not exactly the same thing, however for the purpose of this article we will assume that open-source means free [2].

Open source can be difficult to use

One argument which is often brought up in the open source debate is that open source software can often be difficult work with due to the fact that it potentially requires a certain amount of technical expertise to operate [3].

First off, I would like to suggest that this argument is flawed. The technical knowledge required to work with a piece of software is influenced more by the nature of that particular software than whether or not it is open source. In other words it really depends on who the intended end user is. For example word processors are a peice of software that practically all computer users require; everybody will have used one at some point. As such, installing and operating a word processor requires minimal technical skills even if you choose an open source one like OpenOffice Writer. In fact from my own personal experience open source word processors have been far less frustrating to work with than our old friend Microsoft Word. Moreover software aimed at very niche market, such as command line tools for data analysis will often require good technical understanding to use due to the technical nature of the service they provide.

I do accept that there will be some cases in which proprietary software will be more user friendly than its open source alternatives.

I need support!!

Another point which is commonly raised as an argument against the use of Open Source is that open source tools often lack a proper support network for their users [3]. This is true, however just because there isn’t a traditional infrastructure for providing support does not mean that users are abandoned to simply figure out on their own.
On the end of a quick Google search there is a whole community of users and developers available to help with your queries. Often this support comes in the form of a forum. The advantage of this over a more traditional helpline or user manual is that it provides a channel for you to ask the people who actually wrote the software. Furthermore, these people will often be quick to respond as open source developers typically have feelings of pride towards the software they produce. Perhaps this pride stems from the fact that open source products are not typically developed for financial gain but simply just to provide a solution to a problem. Surely this approach to support is preferable than waiting on hold for 15 minutes to speak to someone who most probably had nothing to do with the development of the software.

Well which should I choose then?

It depends on who you are and what you need.

If the end user in question is a computer literate member of the public then the answer is no! As long the open source option in question is not totally impossible to work with then it will not take long for one person to get used to using it. Additionally the open source option will give them free access to a piece of software which is continually being improved by members of the community. Rather than paying for software which only be maintained until such times as the new version is released then if you wish to see continued improvements you must pay yet more money for the new license.

Alternatively, if our end user is a business then the decision becomes more complicated. Rolling out a piece of software which is potentially frustrating to use could have negative effects on productivity. The management must be sure that the savings achieved from choosing the open source version outweigh any potential losses due to a drop in productivity. However since the open source version is free it would be easy to have a trial run with and see how well people adjust to it. If there is a general feeling that the open source version is good enough then go with it.

What about these aforementioned ‘experts’? In my humble opinion software developers have no excuse for neglecting to use open source software. Especially if the reason for this is that the software is not very user friendly. All software developers will have at some point been through the excruciating task of trying to fix some that is not working for mysterious incomprehensible reasons. My point is that overcoming the challenge of actually writing a piece of software is far more difficult than spending a short period of time learning how to use one. Furthermore the beauty of choosing open source as a software developer is that if you can take full advantage of that clause in the license agreement which gives you the right to modify your copy of the program. You can actually own software which does exactly what you need it to do. Surely this is better than the alternative, paying for software which does what someone else thinks you need it to do.

Conclusion

In conclusion the decision to use open source software over proprietary software should not be made based on some bias towards a particular approach. Instead it should be an informed decision made on a case by case basis. Open source is a great idea, it saves the user money and gives them greater flexibility bit this not reason enough to disregard proprietary software every time there is an open source alternative. However it is reason enough to make sure you always at least consider open source as an option.

References

1. http://en.wikipedia.org/wiki/Proprietary_software
2. http://en.wikipedia.org/wiki/Open-source_software
3. http://www.entrepreneurhandbook.co.uk/open-source-software/

Introduction

This is a response to the article “Agile Methodologies in Large-Scale Projects: A Recipe for Disaster”. In this article the author uses analysis of the agile manifesto to provide supporting arguments for the claim that the use of agile development techniques in large-scale software development is “infeasible”. The article concludes that agile development techniques have no place in the development of large software. I aim to provide some counters to the arguments made against agile development and convince you all that it should always at least be considered.

Agile Development lacks professionalism……

The author suggests that a lack of documentation can be detrimental to the level of professionalism a project may appear to have. They call for the use of documentation as a means of providing an explanation for why a particular project may have failed. This point is validated using the failure of large scale government projects as an example.
Firstly, I would like to suggest that the use a government project as an example may be a little inappropriate. I feel that the levels of bureaucracy and public scrutiny involved in such projects makes them considerably different from private sector projects and as such they should be approached in a different manner (please see my previous article for more detail).

Furthermore, I would put to the author that a project which fails (due to running over budget or over time) but has extensive documentation could also appear unprofessional to clients. The failure of such a project raises the question “why spend so much time and money on extensive documentation for a system which doesn’t even work?” Therefore surely the practice of delivering working code over small iterations is more satisfying for everyone?

Don’t worry there will be documents

If agile development is used correctly then there should be excellent communication between everyone involved in the project, especially the development team and the client. So even in the event of a failure there will be a “hierarchy of responsibility” as all the people involved should know what is going on. Furthermore the deliverables that are produced by the short, fast iterations that agile caters for should be sufficient if any investigation into the failure of the project is required.

Unlike the author seems to suggest, agile development does not abandon all formal documentation. In fact documentation is still encouraged with agile methodologies it is just that development of documents is more likely to occur at the end of the development cycle in order to avoid wasting time and money creating documents which will no doubt need to be altered as the project goes forward[1]. Any documentation produced will be concise and simple yet detailed enough to include the vital information.

Flexibility is key

While I agree with the notion that for very large scale projects it is important to have a clear idea of the initial scope of the system. I reject the idea that this should come in the form of a static requirements document. In fact I would say that for a very large project it is vital that system specifications should be changeable. This is because it is simply not realistic to expect the customer to be aware of everything they require from a system at the very beginning of the development cycle. In addition to this large scale systems which are the subject of the author’s argument often take a number of years to develop and as such it is highly likely that the needs of the client will alter during the development process. It would be irresponsible for a development team to deliver a product which they know does not meet the needs of the client simply because these needs were not obvious from the beginning. I would say that such projects should be considered failures under the justification that they are not “fit for purpose”.

Additionally the author presents the idea that “each intricate detail” of a project should be carefully thought out before any actual implementation is done. I feel that this is highly unrealistic and would lead to a nightmarish development process consisting of never ending requirements capture and design phases. It is virtually impossible to create a perfect design of all aspects of the system, no matter what happens there will always be some parts of any initial design that require alteration (or even a complete re-think). This is especially true for large scale projects which could be made up of numerous components. What this means is that investing large amounts of time and money into meticulous planning before any implementation is an extremely inefficient practise as it is almost certain that such plans will change. In contrast agile development has design and implementation working side by side resulting in less time wasted on attempting to design the whole system at once.

I agree with the author’s observation that making changes in large scale projects is not a simple task and that any changes made will create a number of subsequent changes. However I do not feel that these added complications are reason enough to avoid making the changes. Such complications can be avoided if everyone involved in the project communicates with each other appropriately (one the key the concepts of agile)[2].

Similarly it is suggested that developing software using this static approach protects developers from being overworked by demanding clients who continuously request new features. This may be true in the sense that at the end of a project it will be easy to evaluate success or failure based on the original requirements. However I think it is important to remember that the software developers are providing a service which clients are paying for, therefore if a client wants a new feature or a change they should have the option. If you wish to make changes to an order you placed in a restaurant the waiter will not deny you as long as you are willing to cover any extra costs. Similarly, the responsibility for any substantial budget or timescale penalties due to changes falls directly on the client.

Conclusion

In conclusion, developing large scale software projects is an extremely difficult process. Such projects have a history of failure[3]. However this poor track record is certainly not a reason to abandon the use of agile development techniques. If anything it is actually a reason to avoid traditional techniques as much as possible as they have proven to be problematic. The criticisms of agile development made by the author of “Agile Methodologies in Large-Scale Projects: A Recipe for Disaster” are understandable however after some thought these concerns can be disregarded. Unlike agile development itself which may not be perfect but at least it makes a conscious effort to make a step towards a world in which project failure rates are significantly lower. For this reason alone it should at least be considered as a methodology for any software project large or small.

References

1. http://www.agilemodeling.com/essays/agileDocumentationBestPractices.htm
2. http://agilemanifesto.org/
3. http://en.wikipedia.org/wiki/List_of_failed_and_overbudget_custom_software_projects

Engineering large scale software successfully is an extremely difficult task, as such the majority of projects result in failures [1]. Recent times have seen the desire to offset this obvious imbalance through the emergence of Agile software development. The popularity of Agile design methodologies is ever increasing with more and more companies developing software using agile approaches such as scrum and crystal every day [2]. Despite this the success rate of large scale software engineering remains relatively low and big project failures still crop in as news stories every so often [3]. One of the most notorious examples of a very large scale agile development project failure has been the UK governments Universal Credit [4].

Universal Credit, that sounds boring!!

It is, Universal credit is the UK governments attempt to simplify the welfare system in some way by replacing six of the key mains tested benefits (Job Seekers Allowance, Child Tax Credit etc) with one payment [8]. The reform will require the support of a large scale IT system. It is hard to tell exactly what will be required from this system, ironically some areas of gov.uk were not accessible when I wrote this. However, I believe it will include some kind of web support for calculating eligibility and payment amounts. In any case the exact requirements of the software are not important for this article which is concerned with the development process used to build it.

Why single out Universal Credit?

There have been other high profile failures of large scale government engineering projects so why focus on Universal Credit. The theme of failure in Government IT projects is all too familiar, they have a history of running overtime and over budget. However, the government envisioned a different fate for Universal Credit through the use of agile development [5]. Unfortunately though, the development of Universal Credit has induced feelings of déjà vu by sharing two key characteristics with some of its predecessors: huge cost and late deployment [4].

Is Agile to blame?

The failure of Universal Credit has been very public and the reasons for it have been the subject of widespread debate. Agile development has been at the root of much of this debate in fact some people have claimed that Agile is at the root of the failure and should never have been used for this project [7]. While others suggest that Agile is not to blame as it was the victim of misuse and misunderstanding by the people involved with Universal credit [6].

Both of these viewpoints certainly have merit however I feel as though the first is really just a manifestation of the second. It is clear from reading numerous articles related to the agile development and the failure of Universal Credit that from the start there was confusion about the concept of agile development. For example, despite the intention to use Agile development the project was still constrained by elements of big design up front. This major contradiction to the principles of Agile development demonstrates an alarming lack of understanding. It appears as though in the context of this project the word agile has been thrown around for the sole purpose of encouraging vendors to deliver the system as swiftly as possible. In other words Agile should not be blamed for the failure of the project as it was never Agile in the first place. The reality is that the government’s attempts to use Agile development resulted in a kind of mutated waterfall model with accelerated implementation and testing phases.

Don’t trust the feds!!!

It could be said that no appropriate development methodology exists for these large scale government projects. Certainly the continued failure of projects such as Universal Credit and the even more notorious NPFIT could lead one to this conclusion. The levels of bureaucracy and public scrutiny involved in the development of such systems separates them from large scale private sector projects. Perhaps the politics involved impacts the development process to such an extent that we cannot apply the usual mainstream software engineering practices to these systems. If this is the case then there may be a need for someone to develop some guidelines or methodology for successfully delivering large scale public sector IT projects. However I feel that the development of such guidelines is unlikely as it would involve further governmental input and their track record is not encouraging.

It is also possible that the failure of large scale government IT programs is caused by a simple case of over ambition. Both Universal Credit and NPFIT attempted to provide systems which required the software to unify different approaches to problems in an enormous scale. In the case of Universal Credit there was an attempt to replace one on one sessions people had with councilors to evaluate their welfare needs with a software system. The problem with this kind of computing is that it tries to provide too much structure to something that is inherently chaotic. In some cases people interacting with people is still very important and in these situations we cannot expect software to successfully replace human interaction.

Political agendas could be at the route of the ambitious nature of government IT projects. Popularity among the general public is paramount to any government regardless of the particular party. One way in which governments can become and remain popular is if they are seen to be making significant and positives changes. This could lead to spawning of over ambitious large IT projects where perhaps a collection of smaller more subtle changes would be more beneficial. In any case it is clear the current practices process for developing government IT projects need to be assessed if disasters such as universal credit are to be avoided in the future.

References
[1]http://www.zdnet.com/blog/projectfailures/study-68-percent-of-it-projects-fail/1175
[2]http://www.infoworld.com/d/application-development/whats-wrong-agile-development-culture-people-top-the-list-213480
[3]http://www.cio.co.uk/insight/strategy/why-nhs-national-programme-for-it-didnt-work/
[4]http://www.computerweekly.com/news/2240185166/Universal-Credit-will-cost-taxpayers-128bn
[5]http://www.computerweekly.com/news/2240104931/DWP-adds-agile-development-into-IT-contracts-for-2bn-Universal-Credit-system
[6]http://blog.unicom.co.uk/the-universal-credit-project-a-failure-of-waterfall-project-management-or-a-failure-of-agile-methods/
[7]http://www.computerweekly.com/news/2240187478/Why-agile-development-failed-for-Universal-Credit
[8]https://www.gov.uk/universal-credit/overview