Comparing the Defect Reduction Benefits of Code Inspection and Test-Driven Development ,done by s1340691 Eshwar Ariyanathan

Introduction to key terms :

1)Test Driven development(TDD) : It is a agile software development methodology where the test cases are written first before the actual code gets written. This enables the developer to re factor the code and find bugs in the code in the initial stage of the development process.

2)Code Inspection(CI): It is a process of software inspection where experts analyse the source code to look for any bugs or errors in the source code and then rework is done by developers to correct the source code.


Wilkerson, J.W.

Sam & Irene Black Sch. of Bus., Pennsylvania State

Univ., Erie, PA, USA

Nunamaker, J.F., Jr. ; Mercer, R , “Comparing the Defect Reduction Benefits of Code Inspection and Test-

Driven Development,” IEEE Transactions on Software

Engineering, vol. 38, no. 3, pp. 547-560, May-June

2012, doi:10.1109/TSE.2011.46


1) The previous research work shows that the code inspection has been researched on a lot of scenarios and industrial applications and has been shown to have superior results in terms of testing practices.

2) The scientists working on agile methodologies have a claim that Test Driven Development is a better approach of testing.


To find out whether Test Driven Development(TDD) is a better approach or Code Inspection(CI) is a better approach of testing.


The experiment was based on a programming assignment on spam filter in Java which was done by 40 undergraduate students. The students were divided into four groups for testing namely TDD group, CI group, TDD+CI group and last group which neither used TDD not CI for testing.

The result on final analysis taken into account had only 29 students and the others left before the assignment was over.

Code inspections were also performed by the same group of students using online collaborative tool. The students were given training for coding using Junit before the experiment was conducted.

The total defects found by TDD group and CI group were compared and analysed for producing the experimental results.


*Code inspection was better at reducing defects when compared with Test driven development

*Code inspection + TDD was producing better results but not statistically .

*Code inspection was slower than TDD approach in finding the defects.

*TDD has to clearly defined as the procedure takes in a lot of comparisons and variations everyday.


I have a lot of disagreeing points regarding the experimental results and the method employed for the experiment.

Firstly , the experiment concludes that Code inspection is better than TDD in terms of defect reduction.

I do not agree with this point because it has been given with just one programming assignment which is less than 600 lines of code and there are no sufficient justifications as to why Code inspection is better.

Secondly, the experiment is done with students of a university with varying levels of ability.

So my claim is that how can we accept a result from people who are neither experts in the field nor do they have done sufficient research to prove the validity of the experiment.

Thirdly, the time taken for the experiment is a week (less than a month in research standards.) When the time taken for an experiment is very less then jumping to conclusions is not at all acceptable.

My next point is that regarding test driven development the students who performed the experiment were just beginners.

They were just given some tutorial and lectures regarding JUnit before the experiment was performed. Clearly the students involved in the experiment were not experts in testing using JUnit.

My next argument is that the result states that Code inspection was able to detect 23% defects compared to 11%defects of TDD. But they have not mentioned as to what type of defects were being found out by the TDD group or the Code inspection group.

The other important point to be noted is that the students involved in the experiment were of varying levels of ability with respect to programming and methods of testing. So the deviation in results produced might have also happened because the students doing code inspection were better

testers compared to that of those who did Test driven development.

So the results from a small experiment done with people who were not experts and done in very less time without any background and without any clear set of parameters cannot be considered as a valid result.

But this experiment could be considered as a spark so that industry could take on with further research in this field to prove the effective approaches of testing.


My conclusions are that the research working regarding this testing using should proceed with testing experts both in TDD and code inspection and should be done in controlled manner with sufficient amount of time.

The future research should also take in to account of different parameters involved with testing and on varying lines and complexity of code and then come to conclusion in finding a superior method of testing.

So as far as this research paper is concerned i am not agreeing with either the approach or the results obtained.


[1] G. Tassey, “The Economic Impact of Inadequate Infrastructure for Software Testing,” technical report, Nat’l Inst. of Standards and Tech nology, 2002.

[2] B. George and L. Williams, “A Structured Experiment of Test-Driven Development,” Information and Software Technology, vol. 46, no. 5, pp. 337-342, 2004.

[3] E.M. Maximilien and L. Williams, “Assessing Test-Driven Development at IBM,” Proc. 25th Int’l Conf.

Software Eng., pp. 564-9, 2003.

[4] D.L. Parnas and M. Lawford, “The Role of

Inspections in Software Quality Assurance,” IEEE Trans. Software Eng., vol. 29, no. 8, pp. 674-676, Aug.


[5] F. Shull, V.R. Basili, B.W. Boehm, A.W. Brown, P.

Costa, M. Lindvall, D. Port, I. Rus, R. Tesoriero, and M.

Zelkowitz, “What We Have Learned about Fighting Defects,” Proc. Eighth IEEE Symp. Software Metrics, pp. 249-58, 2002.

[6] M.E. Fagan, “Design and Code Inspections to Reduce Errors in Program Development,” IBM Systems J., vol. 15, no. 3, pp. 182-211, 1976.

[7] N. Nagappan, M.E. Maximilien, T. Bhat, and L.

Williams, “Realizing Quality Improvement through Test Driven Development: Results and Experiences of Four Industrial Teams,” Empirical Software Eng., vol. 13, no.

3, pp. 289-302, 2008.

Using Social Networks to Analyse the Stakeholders of Large-Scale Software Projects

1- Introduction

1-1- Stakeholders?! Mmmm I heard this word before!
Whatever your software project is, you have stakeholders! If there are no stakeholders, then how and why to do it?! All the people who are engaged in a software project are called “Stakeholders”. Stakeholders are the ones who affect and be affected by the actions in the system we are building. Employees, Customers, students are all examples of stakeholders. The importance of stakeholders comes from their knowledge of the processes we are working on.
1-2- Ok? So What’s the point?
A lot of software projects fail just because people ignore or neglect the role of stakeholders. Based on reports [1], ignoring stakeholders is the most common reason for failures in software projects.  I don’t want to spend the whole time convincing you how important stakeholders are. JUST BELIEVE ME and let’s continue.
1-3- So stakeholders are important.. then?
It is once said, “Not all your fingers are the same!”. The problem isn’t really just about ignoring the role of stakeholders, but it is about dealing with them all in the same way. They aren’t the same! How can I deal with the manager of the organisation just like I deal with the concierge?! Yes they both interact with the system but not in the same manner and not with the same influence. The main idea is about building a social network where nodes are the stakeholders and links between them are weighted recommendations.

2- Using Social Networks to represent stakeholders and their importance: More details about the approach

2-1- Ok! So it is about building a weighted graph? How to put the weights?
Yes! We will represent the stakeholders and their influence by using a graph. Weights will be assigned by allowing each stakeholder to recommend other stakeholders.
2-2- I just can’t get the idea! Why to build a social network?
When we build a social network we can simply use all the Graph properties and the social network measures.
2-3- Well! Things will be clearer with an example!
Let’s take a simple example! We have 4 stakeholders, Alice is the manager of the system, Bob is a programmer, Carl is a post-graduate student and Dave is an undergraduate student. Every one of them recommends the people with whom he/she interacts. These recommendations take the form:
<Stakeholder, Role, Salience> where:
–         Stakeholder: his/her name
–         Role: the role of the supervisor
–         Salience: ordinal number represents the importance of this person (Let’s say it’s between 1 and 5 where 5 is the most important).
For example, Alice recommends both Bob and Carl as:
<Bob, programmer, 4>
<Carl, PostGradStudent, 2>
And so on for all others. After writing the previous triples for all we build the network.
2-4- So? We have a cute social network! But how to use it?
Simply, we will priorities our stakeholders using social network measures such as:
          2-4-1- Betweenness Centrality: it ranks a stakeholder based on its role as a “broker” between various stakeholders. It counts the number of shortest paths in the network that go through this stakeholder. For the previous example, Carl is the most central one as both Alice and Bob use him to go to Dave.
         2-4-2- PageRank: it is Google’s base algorithm for ranking search results. By simple words, the importance of a node (stakeholder) is determined by the importance of the nodes referring to it. In the previous example, Alice and Dave have a link with the value (5) going in. However, Alice is more important in this measure because it is recommended by Bob who has a higher rank than Carl who recommends Dave! (since Bob has a higher value of recommendations going into him!) The same as in the web, where the importance of a website is really determined by the importance of the websites that point to it!
         2-4-3- Other Measures: I don’t want to turn my blog post into a graph measures lecture! So I won’t list all of the things you can do and measure but some other ideas are related to the total number of arrows and their values (degree of a node, in-degree, out-degree, etc…).
2-5- mmmm.. that’s cool! But which measure to use?
Actually, it depends on the problem we are solving and on the domain. We may use a mixture of these. I don’t want to go into the details of this but it is interesting! Lots of papers are about this. Just google it!
2-6- Has anyone tried this method practically?
YES!! StakeNet is a tool defined and used by people from the University College London (UCL) with a LARGE-SCALE SOFTWARE PROJECT with a 30000 user [2]. Identifying and prioritising stakeholders are done by this tool for the new UCL access control project. StakeNet performs better than other usual methods to determine the importance of stakeholders.(More details in the paper[2] about the evaluation process).

3- Conclusion

Well, in my eye using a social network to show the stakeholders and their relations is extremely useful. The first reason is that you won’t ignore them and the second you will give them priorities. Another important point is allowing all the stakeholders to have an opinion in determining other people importance.  Using this representation for large-scale software projects will be a great thing for all these reasons.

– The example and image idea are from the reference [2]. I re-draw it in a more “cool” way 😉


1- D. C. Gause and G. M. Weinberg. Exploring Requirements: Quality Before Design. Dorset House Publishing, 1989.
2- Lim, Soo Ling and Quercia, Daniele and Finkelstein, Anthony, StakeNet: using social networks to analyse the stakeholders of large-scale software projects, Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 1, ACM, 2010.