Test-Driven Development – How Applicable is it?

Test-driven development (TDD) is a development process for software that attempts to simplify designs through the writing of tests.

It involves the developer writing tests with an improvement or new functionality in mind and then writing code so this test can be passed. This forces the developer to concentrate on the requirements of the software before writing code, as they need to know what functionality to test. Once this simple code passes the test, it can be refactored to reach the standards acceptable to the development team.

However, how applicable is this developmental process? Can it really be applied to every development situation?

I will be exploring these questions in my post, looking at the benefits of a TDD and problems experienced when dealing with legacy code, small projects and also at how much testing really should be done under a TDD.

What is TDD?

Firstly, in order to be analysed, TDD needs to be correctly specified.

Defined by Kent Beck in 2003, [1] TDD is built upon the concepts of TFD (Test-First development), and can be described simply as:

“TDD = Refactoring + TFD” [2]

A developer who is programming under the TDD approach never writes a new feature or functionality, until there exists a test that will fail because of a lack of implementation.

After the test is passed, the programmer will refactor this simple code to bring it up to the quality of the team standards (this step turns TFD into TDD).

Simply it can be summarised in the following cycle: [See diagram below]

  1. Add a test.
  2. Run all tests and see if the new one fails.
  3. Write some code (causing the test to pass).
  4. Run tests again.
  5. Refactor code.

Diagram 1

Benefits

But are what are the benefits of this practice?

Naturally, due to the nature of TDD, more code has to be written due to the extensive testing carried out.

A model (designed by Muller et al.) compares the development times of startup projects, one developed under conventional methods and one under TDD and the results show that the total code implementation time is, in fact, reduced. [3] (See Image below)

Development Time

Due to the extensive unit testing being carried out, there’s also a greater likelihood that defects are caught early in the development lifecycle, saving time and money had they not been found till later.

Lech Madeyski, showed the benefits of the practice empirically over a more traditional development approach where testing was left to the end of development.

He found that the quality of the unit tests created under TDD was greater, as they scored better on branch coverage and the mutation score indicator than the tests from the traditional approach. Branch coverage and mutation score indicator can be used to show the thoroughness and helpfulness of tests for detecting faults in the software. [4]

Silver bullet?

So we can see the benefits of the conceptually simple TDD in both the speed of development and quality of tests but can all software development projects really be developed under TDD?

Code that has already been developed from a different development style, called legacy code, can be brought up to the standard of TDD, in that small tests can be written to demonstrate the functionality of the program but for a large codebase this can prove tricky and begs the question: is it really worth it?

Having a consistent development process within a team is a reason for converting legacy code to TDD, as it would mean developers who were more comfortable developing under TDD wouldn’t have to switch to unfamiliar practices to work with the older code, likely resulting in less mistakes being made.

Nevertheless, repetition of test creation would almost certainly occur, as the legacy code would already have tests in it. These would either have to be redesigned or ignored in order to fit into the TDD style.

Shihab et al present an approach of such an adaptation, coining it “Test-Driven Maintenance”. It enables teams with limited resources to test legacy systems efficiently by producing a prioritised list of functions that updates as unit tests are written and passed. [5]

As there exists an approach to bring a legacy system under a TDD-like development process (TDM) and because of the positives of having a TDD, it’ll be worthwhile to convert legacy systems, regardless of the repetition that might occur.

The positives are even greater when you see that companies tend to spend over 90% of their total software cost in the maintenance and evolution phases and so making sure these stages are efficient and as well developed as possible will help to save a lot of money. [6]

We need it quick!

Some people argue that the upfront cost, in terms of time, of constructing tests is not worthwhile for a small prototype program rushed to market.

While this might make sense in concept, the consequences of not following TDD are not quite so obvious.

In reality, TDD tends to be very useful in smaller projects, rather than overkill. This is because on this smaller scale it’s easier to be disciplined enough to continue to write tests and follow the practices of TDD. Whereas, on larger projects, developers tend to get complacent due to time constraints and the complexity/number of features that need to be written.

If this small project were developed under TDD and proved to be successful, the extra time sacrificed to create tests would pay dividends in its future development. If, however, the system was developed under some other method it would probably continue with it and so we will end up with another legacy system. Converting this system to a TDM would take a lot of effort, much more than if TDD was followed from the start.

So, even for small projects, I think that TDD is a development style that should be practiced as even though the upfront costs of the tests may slow down progress at the start, time spent here will prove to be invaluable later on.

More tests?!

Many sceptics of TDD complain that while testing is useful for ensuring successful software, it spends far too much time concerned with testing very basic functionality, which they believe doesn’t need to be tested. Also while testing might ensure that the developers are confident that their code works, they don’t get paid to test, this is the job of the testers. [7]

However, redundant testing is just what non-practisers of TDD think it’s all about.

Here is what, Kent Beck, the father of TDD says:

“I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a level of confidence… If I don’t typically make a kind of mistake… I don’t test for it”

This shows that even in TDD you don’t have to test every single function and that common sense should be applied.

If you’re concerned you might get it wrong, write a test for it.

In larger projects this philosophy will help keep the total number of tests down, which will mean less time spent maintaining them. It also means that developers won’t spend too long writing tests for the software, while still being able to reap the profits of the TDD development style.

Conclusion

Test-driven development is definitely a development process that needs to be seriously considered more by software development companies.

As we’ve seen it shines in a variety of situations, from legacy code to quick prototypes and even in large startup developments. The positives are shown through the overall shorter implementation times, better quality unit tests and greater chance of catching defects early in development.

The father of TDD, Kent Beck, also dispels some myths about TDD, revealing that tests only should be made when YOU think they should and until YOU are confident with your code.

For all these reasons I think that TDD needs to be something that development teams need to reconsider.

References

[1] – Kent Beck, “Test Driven Development: By Example”, Addison-Wesley, 2003.

[2] – Scott Wambler, “What is TDD?”, 2013, http://www.agiledata.org/essays/tdd.html#WhatIsTDD [Accessed on: 1st March 2014].

[3] – Matthias Muller & Frank Padberg, “About the Return on Investment of Test-Driven Development”, 2012, http://www.ipd.kit.edu/KarHPFn/papers/edser03.pdf [Accessed on: 1st March 2014].

[4] – Lech Madeyski, “Test-Driven Development – An Empirical Evaluation of Agile Practice”, Springer, 2010.

[5] – Shihab et al, “Prioritizing the Creation of Unit Tests in Legacy Software Systems”, 2010, http://sail.cs.queensu.ca/publications/pubs/shihab_spe2011.pdf [Accessed on: 1st March 2014].

[6] – Len Erlikh, “Leveraging legacy system dollars for e-business”, 2000, http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=846201 [Accessed on: 2nd March 2014].

[7] – Assaf Stone, “10 Reasons to Avoid Test Driven Development”, December 2011, http://www.softwareandi.com/2011/12/10-reasons-to-avoid-test-driven.html [Accessed on: 2nd March 2014].

Pair Programming, Best Development Practice?

This discussion is a response to s1369981’s post, “Why Pair Programming is the Best Development Practice?”

In their post, the author tries to convince the reader of the benefits of pair programming, highlighting the positives that this agile developmental style brings, such as, increased knowledge sharing and project ownership, stating that:

“Pairing is about collaboration and teamwork, thus it will shine… [in] knowledge sharing and project ownership…”

Although, despite the author’s arguments, I’m not completely convinced of their stance, mostly due to the, exaggerated, claim that pair programming is the overall best development practice in all scenarios of programming development.

I think that while there are certain situations when it can be beneficial, (between two novices) extending this to every programming situation is not significantly justified.

The author doesn’t see this because they neglect to address:

  1. The economic downfalls of the practice.
  2. Developers’ natural opposition to this working style.
  3. The fact programming might not be so mundane, if you enjoy where you’re working.

These counter arguments that I present will show that the author has failed to consider these important points in their article and that they’ve been too general in their argument that pair programming is the best development practice for all situations.

But Work is Boring?

In the post, s1369981 makes certain claims, that I’m not particularly in agreement with, such as:

“… most programming tasks in a software company will focus on boring and repeatable tasks with very few of them being creative and challenging.”

This pessimistic view of what the programming world after university is like tends to suggest that the only hope for a programmer to have an enjoyable time is to pair up, therefore distracting you from your “boring and repeatable tasks”.

This solution of improving your enjoyment at your job would only ever be a temporary one, as the novelty of pair work wears off.

Finding a more exciting company according to your personal tastes in programming would help you to enjoy you work more, without needing the distraction of a partner to making it bearable. Also, by simply increasing your communication amongst members in the team, working on different projects, would increase team spirit and cooperation and make it feel much less like you’re working on your own.

I’m stuck!

Speaking from personal experience, while on my internship, I found that instead of any pair programming scenarios, the newcomers (or contractors) to the team sought out the help of more experienced senior developers when stuck, rather than pairing up with them while programming.

This practice produced similar benefits of a senior developer working with a novice, in that the more experienced developer could pass on valuable knowledge and use their expertise without feeling restricted by having to effectively babysit this new employee.

This also left the senior developer with time to apply their invaluable knowledge elsewhere by programming solo, where they would be able to maintain their high productivity. [1]

As mentioned before, having a pair programming situation amongst two novices or a novice and someone who is competent would be helpful because, on their own, they’d undoubtedly have a low production levels but together they can boost their learning levels and this allows new recruits to get up to speed quickly. [1]

Economics

Something not mentioned in the author’s article is the economic viability of mass pair programming, as the team would need to have more employees to manage the same amount of projects.

In controlled studies it was found that it wasn’t economically viable as only for simple systems was a significant decrease in development time found and no significant difference in correctness of solutions. [2]

In fact, in this large empirical study, Arisholm et al. found that the results did not support the general consensus and that:

“… pair programming reduces the time required to solve tasks correctly or increases the proportion of correct solutions.”

Instead, they discovered that, in general, there is an 84% increase of effort required from the programmers to perform the tasks prescribed correctly, where effort (or cost) is the total programmer hours spent on the task.

These empirical results give us a more concrete measure of the benefits of pair programming amongst a variety of levels of programmer and I believe this evidence to be more reputable than remarks from people who’ve tried out pair programming, as this is open to bias.

The findings back up the reasoning that for a team to be operating at the same level as they are currently, managing as many different projects as they are, they’d have to hire more employees to maintain this level of output even when the benefits of pair programming aren’t so great.

It ain’t all fun

The author’s conclusion takes a simplified view of the situation by suggesting it should be adopted because:

“Pair Programming is the best development practice, because it is fun!”

But as suggested earlier in the article, by the author, there is a lot of strong opposition to this with people arguing adamantly against this belief. [3]

So, certain people will not work well in pairs, no matter how many statistics or studies you throw at them and I believe that if it is going to be used in a team, it should be tried out for a certain period where productivity can be monitored.

As mentioned and described by s1369981, people should be also be educated in how to properly undertake the pair programming developmental process if they’re going to be working with it and this can help to eliminate common mistakes and incorrect assumptions made about the practice.

Once the practice has been carried out correctly, the management can get feedback from it both empirically and from the developers who tried it so that they can make a reasoned decision on whether it is a viable option for your team.

Here, developer input should be considered closely because regardless of whether it makes your programmers more productive, making them uncomfortable in their work environment will cause some people to quit.

Conclusion

There are some points in s1369981’s article that I agree with, such as, the fact that pair programming can increase knowledge sharing and project ownership in a team.

However, the application of pair programming to all forms of development is an overstretch due to the economic downfalls, some developers being opposed to paired work and the argument that only pair programming can make your job enjoyable.

I do believe that it still has its place e.g. between two novices in a company or for complex tasks, as it can help to improve correctness of code, but bear in mind that this comes at a price: overall increased effort. [1] [3]

Therefore, any adoption of pair programming should be evaluated on a case-by-case basis to see if it really is the “best development practice”.

References

[1] – Derek Neighbors, “Should Senior Developers Pair Program?”, November 2012, http://derekneighbors.com/2012/11/should-senior-developers-pair-program/ [Accessed on: 26th February 2014]

[2] – Erik Arisholm et al, “Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise”, 2007, http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4052584 [Accessed on: 26th February 2014]

[3] – Matt Ervin, “Pair Programming (give it a rest)”, November 2013, http://peniwize.wordpress.com/2013/11/17/pair-programming-give-it-a-rest/ [Accessed on: 28th February 2014]

Are We Secure In Our Software Life Cycle?

Software security is an oft-forgotten part of the software development life cycle and as a result it often gets left as an afterthought. To combat this, a penetrate-and-patch approach is used, where problems are patched when they occur in the live software and this is how security flaws in programs are fixed. However, this methodology is flawed, as it leads to more patches on released software, due to the security holes that could have been resolved earlier and for much less cost, if it was done before release. [1]

Solution?

Gilliam et al [2] propose a solution to this, arguing that security should be an integral part of development and integrated into the software life cycle.

They advocate using a Software Security Assessment Instrument (SSAI), which will be incorporated formally into the software life cycle, in an attempt to improve the overall security of the software. The SSAI is composed of a variety of components that catalogue, categorise and test what vulnerabilities / exposures exist in the software and picks out those that can be exploited.

Specifically, in this article [2], Gilliam talks about the Software security checklist (SSC), part of the SSAI, which aids organisations and system engineers in integrating security into their software life cycle and allocating their software a risk level rating, which will be useful for re-usable code. In addition, the SSC should also provide a security checklist for external release of the software.

Gilliam claims that in order to improve the security of software it will require “integrating security in the software life cycle… [as] an end-to-end process…” and this is something that I’m not in 100% agreement with. Using an SSAI and SSC for each stage of the development/maintenance life cycle is one which is too heavy on the developer and I believe that a less involved process should be used instead and this is based on certain beliefs and experiences I’ve had.

Experience

During the summer, I had an internship at a large financial institution, working on producing corporate applications for iPhones and iPads. Naturally, due to the nature of the content/information being handled, security was an important part of my team’s work.

However, the use of a security checklist as part of a larger SSAI, as suggested by Gilliam et al, was not the approach that was taken, at least, not completely.

Instead, developers were left to work on developing the functionality of apps that incorporated the in-house APIs, already developed, that were known to handle data securely. This saved an awful lot of time than if this were to be done for each separate app’s (or program’s) life cycle as suggested.

This approach is more efficient as it gives time for the developers to develop functionality, rather than worrying about checking off a security checklist. The accuracy of results from checklists are also doubtful, as items may be ticked without thorough investigation if deadlines were being rushed. This is even worse than having unsecure software, as management believes it’s secure!

Get ready to scrum

The rise of agile development practices has come about due to the realisation that the waterfall development model is fundamentally broken.[3] This means that the involved “end-to-end process” suggested by Gilliam would not be well suited to this current environment.

I experienced this first-hand during my job, as my team were developing in an agile-like development manner. I can’t see how such a security checklist, part of a SSAI, could fit into agile development style, except perhaps through consistent use of it on a daily/weekly basis.

If used in that way, I believe, it becomes a hindrance to development and will likely result in developers forgetting / not bothering to carry it out and leaving it till the end of development and then it’s not much better than the current approach of penetrate-and-patch.

Don’t worry, someone else will do it

Don’t get me wrong, I believe that software should definitely be tested for security before being released, however, I don’t think this should solely be the task of the developer but rather an external party.

This belief is founded upon my time at the company, where before the release of an app, an external party was brought in to test the code for security faults and vulnerabilities. They carried out an intensive week of software testing that, in my mind, is a much more viable way of validating security in programs. These teams were specialists on security vulnerabilities and much like the SSAI, have specific tools (test harnesses) that probed the software.

Feedback from the tests would be relayed to the development team and changes would be implemented in the program. If the software proved to be far too unsecure the external party would be brought in again to run tests after major changes had been made to the software.

If this had been done in-house, tools to realise the functionality of the SSAI would have to be brought in and run by developers of the software being tested. This approach would probably prove to be more costly in terms of price and hours, than if an external company had been brought in.

Don’t look at me, I’m the new guy!

If anyone who is part of the team on a temporary basis (contractors), they would need to be brought up to speed on a large amount of security procedure if it was heavily embedded into the software life cycle. This takes away valuable time that could be spent otherwise utilising the programmer’s capabilities.

I felt that during my job I didn’t need to worry about how I was coding in terms of security, which I would have had to if the SSC had been in place. I would be fearful that every line I wrote was incorrect as I hadn’t dealt with secure programming before, whereas, in reality, I was much more relaxed and able to program to the best of my ability.

Smashing stacks

This year I have been taking part in the secure programming course, which aims to encourage us to build new software securely by using techniques and tools to find and avoid flaws found in existing software. [4]

The way this is achieved, normally, is through common sense i.e. by not reproducing code that was found to be unsecure, rather than the formal approach described by Gilliam et al.

I think that this formal approach is perhaps an idealised attitude as to what should be happening and, in fact, for the majority of software life cycles, teams are more concerned with getting the bulk of the work done before focussing on how secure the product is.

But look, I’m secure!?

The security rating that could be provided by using an SSAI with an SSC could be very useful, as it would allow users of software to gauge how secure any data they input is and enable them to compare the security of similar products.

However, the consequences of this rating might not have the desired outcome. This is a similar problem to that which was seen before in the SAPM lectures [5], where companies would produce more features for their software in order to tick boxes, making it seem like they had the better product. However, in reality, the features weren’t desired by the users and only existed to make it appear like the software was better than its rivals, as it “ticked more boxes”.

Why does it matter?

But should we really care about software security in the software life cycle? I say yes, very much so.

As pointed out by Gilliam et al, several studies can be found showing that neglecting security in the software life cycle can have very negative impacts, both financially and in terms of image. [2]

They recommend that integrating security into the life cycle of the software can counteract this, but I disagree with them in terms of how much involvement it should have at each stage.

Their endorsement that this integration should be “end-to-end process” is one that is not carried out by an organisation who is heavily involved with secure programs, not best suited for the agile development style (that is rising in popularity) and is an out-dated and idealised view of how security can be integrated.

From my experience, I’ve decided that software security is best handled by external companies, who attack the software in order to identify weaknesses (ethical hacking). These can then be sealed / fixed with minimal effort (hopefully) and without the developers having to become experts at using security tools or looking for exploits.

In essence, leave security to the professionals.

References

[1] Gary McGraw & John Viega, ‘Introduction to Software Security: Penetrate and Patch Is Bad’, November 2001, http://www.informit.com/articles/article.aspx?p=23950&seqNum=7 [Accessed on: 4th February 2014].

[2] David P. Gilliam, Thomas L. Wolfe, Josef S. Sherif & Matt Bishop, “Software Security Checklist for the Software Life Cycle”, 2003, http://dl.acm.org/citation.cfm?id=939804 [Accessed on: 4th February 2014].

[3] Mark Baker, “Waterfall’s Demise and Agile’s Rise”, May 2012, http://www.modelmetrics.com/model-viewpoint/waterfalls-demise-agiles-rise/ [Accessed on: 5th February 2014].

[4] David Aspinall, “Secure Programming Lecture 1: Introduction”, January 2014, http://www.inf.ed.ac.uk/teaching/courses/sp/2013/lecs/intro.pdf [Accessed on: 6th February 2014].

[5] Allan Clark, “Introduction”, January 2014, http://www.inf.ed.ac.uk/teaching/courses/sapm/2013-2014/sapm-all.html#/15 [Accessed on: 6th February 2014].