Software testing: an overlooked process during software development

On July 4, 1996, a cluster, powered by the Ariane 5 rocket, was ready to launch on its first journey. Thirty senven seconds after firing, the rocket went off the track and ended in self-destruct as shown in Figure 1. According to Dowson, this accident, known as one of the world's most extravagant failure, costs more than 370 million dollars.[1]

Figure 1: The Ariane 5 explosion

Analysis shows that the rocket system crashes due to an overflow when trying to convert a 64-bit floating number into a 16-bit integer. The inertial reference platform of Ariane 5 is directly reused from the one on Ariane 4 whereas the flying conditions are relatively different between these two models. In this case, the higher horizontal acceleration of Ariane 5 rocket resulted in the overflow. Sadly, due to lack of analysis and tests, the error was not discovered before launch.[2] In other words, there might be chances to avoid this tragedy, if more robust software testings are designed and performed. In fact, the lack of testing is not only a problem in large projects cooperated internationally; It is a habitual behavior that existed deep in the bone of most programmers which stops them producing code with better quality and design. This article will re-emphasize the importance of software testing and introduce the steppingstone of this approach, unit testing.

Definition

Software testing is a process or quantifiable way to identify the robustness, correctness and other aspects of a piece of software.[3]
It mainly verifies:

  • if software behaves as expected
  • how well it performs

Motivation & Benefits

Upon finishing a piece of codes, try to answer the following two questions first:

  • How to make sure that the program works?
  • How to prove its correctness to others?

Rather than directly compile and run it on your personal computer and tell your friend to watch, there are better ways to demonstrate it: software testing. With this approach, it is possible to reduce the number of errors, increase the efficiency for debugging and moreover, not only the quality of the program improves, huge amount of time and efforts are saved from finding hidden bugs which eventually makes all people's lives better.

Different Types of Testing

Rather than rush into the design and development phase, software testings should be carried out at the beginning of the project as well. Instead of perform a single test with the hope of finding and fixing every bug in the end, it is encouraged to carefully design and frequently perform testings during the project. Sometimes, spotting and fixing hidden bugs in earlier phases is significantly easier than doing that in the final stage and by continuously perform testings, it is much cheaper to fix a bug.

General speaking, tests should be applied individually, in groups and eventually to the whole system. That is, software testings could be typically divided into:

  • Unit testing
  • Integration testing
  • System testing
  • Stress/Load testing
  • Acceptance testing

In this article, unit testing, the steppingstone of all software testing techniques, are particularly introduced.

Unit testing

Testings designed for an individual component, such as a method or a module of a software, are called unit testing. By directly verifying the low level behavior of the program, unit testing could easily isolate certain part from the whole program and check its correctness. Ultimately, with this approach, we are able to fully concentrate on a small piece of code and try to identify any potential error or bug affecting the program. Especially when implementing complicated algorithms, this approach is extremely helpful.

Since unit tests focuses on the low level of the program, sufficient understanding of the program codes is needed. In most cases, these tests are implemented by programmers themselves.

Example

This section gives a rough idea about how unit testing are performed using in Java. Please notice unrelated codes such as package importing are omitted. Given a class, named Target, declared as follows:

public class Target{
    public int multiply(int n1, int n2){
        return n1 * n2;
    }
}

It has a method called multiply which calculates the multiplication of two input integers. In order to check the correctness of this method through unit testing, the following Java code are implemented:

public class testTarget{

    @Test
    public void testMultiply() 
    {
        Target t = new Target();
        int x  = 1 ; 
        int y = 1;
        assertEquals(1, t.multiply(x,y));
    }
}

Here, the testMultiply method is implemented to check the correctness of the method multiply. Declaring both integer variables x and y equal to 1, it will try to evaluate whether the calculated result of method multi is also 1. After running this test in JUnit, result are presented as shown in Figure 2.

Other cases can also be considered, such as:

  • x = 1, y = 0
  • x = 1, y are declared as null
  • x = 0, y = 0
    Figure 2: The result of unit testing

Advantages and Disadvantages

One of the big advantages of embedding unit testing early in the development cycle is that hidden bugs can be traced easily and is relatively cheap to fix. Then, by keep designing new tests for both existed and new code, to a certain stage, a complete testing environment can be built. At that point, if anything has changed, the program can still be easily re-evaluated with old testing suite. By using test-driven approach, programmers tend to write test-friendly code which improves the design of the program and make the whole development more efficient.

However, there is no silver bullet in software development. The same is true for unit testing. Focusing only on one piece of code, it is unable to catch errors between integrations among different modules or classes. In this case, Integration tests should also be performed as a complementary practice. Moreover, designing and implementing tests need time, especially for large project. In some cases, the time and efforts needed for designing and implementing tests are often more than coding the actual project.[4]
Alternatively, tools like Junit and CPPUnit can be used as assistants for generating testing code.

Conclusion

To all software components, code quality is considered as one of the key factors which can be increased by performing software testing.
Among different testing techniques, unit testing is the one that directly check on the low level of the program, often implemented at the early stage of the development cycle and gives the programmers confidence on their codes. However, time and efforts are required for implementing well-designed tests. It is important to find a balance between developing the actual program and implementing testings.

References:

[1] Dowson, M. (March 1997). "The Ariane 5 Software Failure". Software Engineering Notes 22 (2): 84. doi:10.1145/251880.251992.

[2] Nuseibeh, Bashar (May 1997). "Ariane 5: Who Dunnit?" (PDF). IEEE Software 14 (3): 15–16. doi:10.1109/MS.1997.589224.

[3] Myers, G.J. and Sandler, C. and Badgett, T.. The Art of Software Testing. Wiley. 2011 http://books.google.co.uk/books?id=GjyEFPkMCwcC

[4] Cramblitt, Bob (2007-09-20). "Alberto Savoia sings the praises of software testing". Retrieved 2007-11-29.

One thought on “Software testing: an overlooked process during software development”

Comments are closed.