s1038803 – SAPM: Course Blog

Beta is the new launch (and why it fails)

There is trend that is becoming popular in software development these days. A lot of companies adopted the agile development model, with one of its slogans ‘release early, release often’. While there are many wildly successful products released using this methodology, this idea is often applied incorrectly, leading to unsuccessful software (as defined by [2]). The point of this article is to show how and why this process can lead to software failure, especially of commercial software, whereas the software is a final product that the end user pays for.

Good old days and how its changed

When you think of a tangible product, say, your favourite coffee table or a new blockbuster movie, they all have one attribute in common – they are finished products. Once you purchased one of them, thats it. The product is not going to be upgraded or get new features – it is not going to be maintained (and if the design of the coffee table is updated or a DVD with extras is released, it’s sold as a new product). Software is different in that regard – it can be updated, patched up and so forth.

In the old days, most software got distributed like any other tangible good – on CDs. Any updates and fixes would usually go into the next release. Now, when most software is distributed through the internet, the mindset has changed. Sending out updates is quick and easy. Fixing small annoyances or critical bugs, adding extra features – there is nothing that can’t be done after release, if needed.

A paradise

Every company wants to be the market leader. There are many possible ways to achieve that, but they usually all boil down to:

a) Have the best product

b) Be the first on the market (find one or be innovative and create a market for some product)

The first one often spans from the second one. Ideally, by being the first on the market, you can capture the majority of the userbase (all of it, if nothing even remotely similar exists)[3]. Users, in turn, provide you feedback on your product, on how to make it better suit their needs. If suggestions are followed through, users will not have much incentive to switch to a competitors product and you’ll be able to dive like Scrooge McDuck.

So, if you want to be the first person on the market, it is often better to release your product as early as possible, before any competing product is created. Even in existing markets, when some interesting development occurs in the field, it if often beneficial to be the first company to implement that feature. This approach worked out quite well for a lot of companies, for example, almost all Google products are released in beta stages and they quickly gain popularity as bugs are swiftly fixed and updates released[4]. Another great example of releasing before the product is completely polished is twitter: there was nothing quite like it on the market, so, despite its shortcomings (fail whale anyone?) still gained traction and no competitors come even remotely close to taking market share from twitter.

It seems that releasing a product before it’s perfect is the perfect solution, as users are becoming more lenient towards execution if the software product is innovative enough.

So, what could go wrong?

Trouble in paradise

The idea of updating post release is often taken to the extreme and we getting products that are just not ready for the market. This trend is currently most apparent in video game industry. Whats worse, is that these software prototypes are sold to the end user at full price, with promises of providing a complete product at some point in the future.

One such example is Steam Early Access programme, whereby you can purchase unfinished games (usually by indie developers / small companies) and help the developer test them. One can sugarcoat it as much as he likes, but a customer is essentially paying to do a job that people are usually paid to do. This is a gray area in terms of software failure. On one hand, software is buggy and it does not satisfy quality requirements of the intended end product. On the other hand, people who purchase early access know that they are getting an unfinished product and should expect bugs and crashes.

Lets assume for a moment that users really liked the previous works of the company and want to support it in its next software endeavour. Another issue arises – feature creep. The software is in this almost-released state, the number of stakeholders, who invested their time and money into the product, increases. Each stakeholder wants to pull the project towards slightly different direction. Whereas previously a developer could decide which features to leave and which to cut from the final release, now the line is blurred. It is now impossible to please everyone who already bought the product. Another thing to consider, is that users can be terrible at giving feedback, and generally don’t know what they want until they are presented with a solution. In this case it is possible, that listening to solutions from the small group of uses can lead to making software that is undesirable to the majority of your target audience. In either case, this leads to a software failure condition where the software does not satisfy users needs in terms of scope.

Lastly, the issue of development time arises. Quite often the periods in which the software stays in a ‘beta’ stage can last for months or even years, with no clear deadline for the finished product defined. Better, finished products (maybe not as feature rich, but very stable and usable) could have appeared on the market by then. When such competitor product appears, the initial software can be considered a failure in regards to being delivered in time.

To release or not to release

In general, releasing early to get feedback is beneficial to a software product. It just has to be done right. Feedback from users should be a supplement to your software development lifecycle and not be the only driving force behind it. Otherwise the aims of the project will expand indefinitely and never be achieved. Early access should also not be sold to users at the full price of the finished product. Testing your software is a favour that the users provide you and not the privilege they have to earn. Finally, set a release date and release your software then, don’t drag the beta-stage forever.

[1] The Art of Agile Development http://books.google.co.uk/books?id=g_ji7cRb–UC&lpg=PA208&ots=vmQwDuYF3y&pg=PA208#v=onepage&q&f=false

[2] SAPM, Success http://www.inf.ed.ac.uk/teaching/courses/sapm/2013-2014/sapm-all.html#/6

[3] First-Mover Advantage http://www.pearsoned.co.uk/Bookshop/article.asp?item=312

[4] The Beta Principle: Skip Perfection & Launch Early http://99u.com/articles/6313/the-beta-principle-skip-perfection-launch-early

Success for all software projects*

(*success not guaranteed, side effects may include going over budget, over time and delivering the wrong product)

This is a response article to “How to make Software succeed?”

In the original article the author identifies that, in addition to some of the more traditional reasons for software failure, there are issues that can lead to software failure post-delivery. He then discusses a few major software failures (Google Buzz, Universal Credit, Pac-Man), common reasons for failure and proposes solutions that would make software projects successful.
Just to remind – a software project is considered successful if it is delivered on schedule, development costs were within budget and software meets the needs of users, in both scope and quality.

I disagree with some of the solutions proposed by the author as those methods do not guarantee success nor are they applicable to most organisations. The author also did not address all issues he mentioned in the introduction and in his examples.

Failures/Solutions discussed in the blogpost

Project runs over time

To begin with, I agree with the proposed solution of having the core of the project implemented before writing additional features. This way even if feature creep occurs later on in the project, the core functionality will still be present and unfinished features can be dropped or scheduled for the next release, ensuring the project is delivered on time.

However, I do not think it is an easy task to “make a reasonable schedule”. First of all, there are various stakeholders in each software project and they want the project to be released as fast as possible. Only a few select companies have the luxury of releasing a product ‘when it’s ready’. For the rest of us, it is not the question ‘When can our developers finish this product?’, but rather ‘Can our developers create this product before this specific date in the future?’. It is the role of a manager to estimate whether the question can be answered positively. Even then, most companies think their schedule is reasonable at the beginning of the project, only to be proven otherwise later on.

Next, the author proposes conflicting ideas of “carefully design at the very beginning” and “give up some bad idea, before it is too late to do so”. While the former statement proposes a carefully planned design, one which is most often employed in the waterfall model, the later one suggests releasing a prototype quickly in order to see if the idea of the project is feasible at all (more common to agile and other modern methodologies). To “carefully designing a piece of software at the beginning” is already in itself a large commitment, where the project has been agreed upon and backing out would have negative consequences.

Project runs over budget

The author claims “be realistic, make the budget based on the full requirements”. Again, the same argument as before can be applied. The developer team has to decide if they can produce the software given a certain budget. Projects run out of budget because the estimates made at the beginning seemed reasonable enough, not because the team knew they were going to use twice the budget, but still decide to do the project.

The claim “work with team which has record of delivery within budget” means that either the team is very efficient or that they are good at estimating the cost and do not accept projects they estimate they can’t complete. This further assumes that there is a team which has record of delivery within budget, but it is not always the case, especially for new projects.

Poor communication

I do agree that the team and the customers have to understand each other. I would like to add that is not only important to talk to your customers early, but to continually get feedback, as further clarifications can be needed, and software scope can change ever so slightly as time goes on.

Never reviewing project progress / Inadequate testing / Lack of quality assurance

I grouped these together as they all relate to the same problem – product does not meet the users needs in terms of quality. The author proposes that the solution to not reviewing project progress, not testing and not writing documentation is to actually do all of those things. While it is an obvious solution, the problem is in itself a little different. It is not enough to schedule some time for all of these actions somewhere in the future, testing and writing documentation have to be done continuously, as the code is being written, not sometime later. If the project is already behind schedule, leaving testing for later can lead to testing being dropped as the project is being rushed through the door (same for documentation). Furthermore, you simply cannot test after “product has been released, before you actually get feedback or a complaint from the user”. If bugs are severe enough, the project will now either be way over time or it will not have met users quality needs. In either case, the project will be a failure under our definition of project success.

Failures not discussed

The author mentions that public opinion and competition can cause the product to fail after it has been launched, but does not pursue these problems further or give possible solutions to them. In the case of Google Buzz, they provided a product similar to the one that was already on the market and fairly popular. They did not, however, add much additional value for the users (some would argue it provides less value), and therefore users had no incentive to switch. Since they did not have the first-mover advantage, they should have made a product that is in some regard ‘better’ than the original.

Similarly, having the wrong idea for a product from the start will cause it to fail, no matter how brilliant it is in regards to technical aspects or how well known your company is. There is no solution to this issue, you can only mitigate the cost of failure by releasing a prototype early and based on the initial interest, continue or move on to the next project.

Conclusion

In this article I have discussed the proposed solutions to causes of software failure made in the post “How to make Software succeed?”. I do not claim to know how to make every large scale project succeed (no one does, really). However, some solutions proposed by the original author are just not applicable in a real world scenario or do not tackle the cause of failure appropriately I have also expanded on the point made by the author that projects can still fail even if all technical aspects are in order, due to external circumstances.

On scripting languages and rockstar programmers.

The article “Scripting: Higher Level Programming for the 21st Century”[1] by John K. Ousterhout discusses the role of scripting languages in software development and expresses hope for a bright future of said languages. His hopes have certainly been fulfilled. The scripting languages are still going strong today, with major companies opting in to use them for development of large scale systems (for example YouTube[2], EVE Online[2], Dropbox[3], etc.). What I am proposing is that is not only possible, but oftentimes faster and cheaper to create a medium to large scale project using one of high-level scripting languages as a core of the system.

So, what are scripting languages?

These are typically high-level (do not directly map to machine instructions), interpreted (quick to modify without recompiling), dynamically typed (types can be interchanged to easily combine different objects), garbage-collected (memory allocation is taken care of) languages, such as Python, Perl and Bash. These languages use a collection of useful components that can be written in other languages. They are ideal for manipulating data arriving from a variety of sources, joining it for further processing.

Blasphemy! That can be done faster in C!

An age old debate, occurring every time something new comes along. Assembly language came to replace machine code, systems programming languages came to replace assembly. Scripting languages are just another layer of abstraction, sacrificing some performance for a more concise code that produces same outcomes. The speed difference is hardly noticeable for most applications.[4] Any significant inefficiency often comes from using an incorrect data structure or not applying so.

This means that developers can write code faster, allowing for more time to fix bugs or code extra features. My point is: at this point in time software developer time is expensive, while computer time is cheap. If you need extra power you can spin up 100s of instances of compute servers on the cloud at a click of a button for barely any cost at all.[5] Hiring more developers on a project just because the system is overcomplicated is just a waste of resources better spent elsewhere.

A true rockstar programmer can code in C just as fast as he can in Python!

A rockstar programmer most likely would. The problem is, most software developers are not rockstars, they are average.[6] Now, with the widespread availability of computers, it is quite easy to start programing. You can pick up the very basics over a weekend. On a scale of [copy pasted ‘hello world’ from a tutorial website] to [can code up a general purpose operating system in a week] most programmers are somewhere towards the middle.

Writing software in low level languages, however, is hard. Your middle of the road programmer might not have had extensive experience with pointers or manual memory allocation. It is a lot easier to write code that is even less efficient than code in scripting languages if you don’t know what you’re doing. Not to say that average software developers can’t learn and are doomed to be mediocre forever, it’s just that the project would take longer to complete.

Python, on the other hand, has a much shorter learning curve and your average programmer will be able to contribute to your project in a short period of time. It is easy and fun, quick to prototype with, quick to recover from failure and to keep programing.

Why not just hire the best developers then?

If you want to get the cream of the crop, you’d better be prepared to pay enormous amounts of money or provide other good benefits to your employees. Whether you’re a medium-sized company or a startup in the process of creating the next big thing™, you have to ask yourself this: Why would a top software developer come to you instead of one of the giants (Google, Microsoft, Facebook, Amazon)? It all comes down to the attractiveness of your company, really.

So, should we just do everything in scripting languages?

Of course not. There are applications where systems languages reign supreme. Applications where every millisecond and every kilobyte counts and require precise control over memory (embedded systems with very small amounts of memory, operating systems programming) can only be efficiently written in system level languages.

Sor scientific simulations – it varies. As mentioned before, scripting languages are usually ’the glue’ that joins other languages. It is possible to have C backed libraries (SciPy, NumPy) that achieve C-like performance while having a nice wrapper in Python. So, scripting languages are not the silver bullet that can be applied to every problem, but it is good enough for most applications.

Final verdict

In the end, should we use scripting languages to create large projects? As with any question in computer science, the answer is always the same: It depends. If you have a good team, the one that understands the concepts well, the project deadline is so far you can afford to spare time to train less experienced developers and you need it to work really fast, then consider a systems language for your next large project. Otherwise, a scripting language may suffice.

If you have anything to add or would like to propose a different perspective – leave a comment down below. 🙂

[Note: I have picked C and Python for most comparisons in this article , but a lot of points can be applied to other languages as well]