Developers and testers: friends or foes? Using fuzzy set theory to decide on the correct ratio of developers to testers in agile teams

1. Introduction


Developer and tester are two essential roles in the development of large-scale and long-term software. In the past few years, there are a lot of discussion of developers and testers. These discussion reflect that software industry develop very fast and the importance of these two roles. However, fight and cooperation never stop between developers and testers in the real world.  In this article, we will talk about developers’ and testers’ work, and how to decide on the correct ratio of developers to testers in agile teams.


2. What a developer’s and a tester’s day really like?


Everyone has their own impression of these two roles in their mind according to their experience. Even talking about the same role, it usually differ because of the size of the company which they work for, the tasks they should finish and so on.


2.1 What a developer looks like?


There is no simple answer to describe a developer’s life. It depends on what levels you are and the environment of you work (the size of the company you work for, academia vs Industry and so on). Talking about different levels of a developer, there is an interesting article talking about “eight levels of programmers” [1] . This may give you an impression of different developers. In this article, we will focus on the average developers in industrial companies.


2.2 What a tester looks like

I am going to give you a brief description of testers based on the experience of my friend who work as tester in his real life.

I have a friend who work as Software Development Engineer in Test in Google. When asking the first impression of his work, he said it is very cool and “luxury”.

Talking about cool, he has opportunity to use the most advanced technologies in Google, such as Google Glass. He thinks his job is “luxury” because of his work environment. He has two advanced computers in his office. He has two 21-inch LCD monitor, one is for debugging and the other is for displaying the interface of product. If your task is testing mobile phone, congratulations! You will have many latest mobile phones and tablets from different companies for your testing, and these can occupy all the space in your table if you want. He thinks these devices help him improving the efficiency of his work, taking into account a variety of complex configurations and simulating customer environments when using their products.


Someone always ask him that as a tester, you do not need to write codes, right? His answer is: of course I need to write code. He used to test thousands of use cases for a module, and it is impossible to finish the testing by hand before the deadline. He need to write a program for testing automatically.


2.3 Good management of developers and testers


Good management of developers and testers in the development of large-scale and long-term software is like manufacturing good pair of spear and shield. Testing is spear and developing is shield. If we know the strengths and weaknesses of spears and shields in the early stage of manufacture, we can make the weaknesses become strengths.

figure 1: figure from [6]

3 How to decide on the correct ratio of developers to testers in an agile team


There is a long-standing problem in the software development world: what is the reasonable ratio of developers to testers in an agile team? In the real world, the developer to tester ratios range from 1:1.5 to 10:1, including organizations, such as Microsoft, which have ratios that are 1:1 [3]. You can see that if you have an agile team to develop a project, you cannot just choose the ratios from other companies. What you should do is analyze your own situation and think about what ratio is best for your team. However, the question is: what criterias you need to use to analyze your situation?


3.1 Johanna Rothman’s theory


Johanna Rothman gives us an answer.[2] He believes that you need to consider:


  1. Product(the requirements, product size, and complexity of the product)

  2. Your project and its process (how your organization develops products, and your customers’ tolerance for defects and ship delays)

  3. Your people (development and test staff’s abilities and responsibilities) [2]


This is the general aspects you need to consider of your product. Furthermore, he gives us a table to evaluate  three different projects as follows:

figure 2: Contrast Process and Project Attributes[2]


We think this is a very useful table with detailed criterias to evaluate a project, and it gives us important information to decide on the Correct Ratio of Developers to Testers in agile teams. In this table, Johanna uses ü to describe a certain company has the issue or not. For example, LineChecker has to define and inspect the product architecture,  but BussyApp and DataCrunch do not have to.  He believes every company will get a score as a result according to their answers of this table, which decide their ratio of developers to testers in their agile teams.


However, we believe that although the criterias Johanna provided are very reasonable, situation in the real world are more complex. You cannot easily say that a project has or has not a certain criteria, such as low customer tolerance for ship delay. The real world situation is that our answer is not just yes or no, we have different levels of our answer, such as: very positive, positive, moderate, negative and very negative. To make the result more reliable, we will use a theory called fuzzy set theory.


3.2 What is fuzzy set theory?


Fuzzy sets [4] were introduced simultaneously by Lotfi A.Zadeh and Dieter Klaua in 1965. What Zadeh proposed is very much a paradigm shift that first gained acceptance in the Far East and its successful application has ensured its adoption around the world. At the beginning of twenty –first century, based on the fuzzy set theory, variable fuzzy set theory [5] is proposed by professor Chen. With the development of fuzzy set theory, it is meaningful to introduce fuzzy set concept, models and approach into data analysis of complex situation.Fuzzy set theory is an important tool in mathematical modelling, which allows for the explicit consideration of professional expertise and judgment. It will be necessary to analyze the mutual relationship between fuzzy measures and their applications. Based on the main concepts underneath fuzzy set theory, we can make the ratio of developers to testers in agile teams more reasonable.


3.3 How to use fuzzy set theory to decide correct ratio of developers to testers?


Since the data collected from Johanna’s criterias are not in a perfect quality, using fuzzy set theory are capable of representing uncertain, non-exact, diverse and inaccurate data or information. We do not want to use many fancy formulas of fuzzy set theory to scare away our readers, so we try to explain the use of fuzzy set theory in this situation as simply and clearly as possible. We redesign the table of criterias which presented be fore as follows:

figure 3: table for evaluate your own project


You can see from the table above that for each criteria, you can have five choices: completely agree, agree, moderate, disagree and completely disagree. Each option has its corresponding score. After using this table to evaluate your project, you will get a much reasonable result compare with the old version(just yes or no answer).


We will make it clear that this ratio is not stable. It can be changed according to the performance assessment. You can add different criteria to evaluate your developers and testers.


Maybe someday in the future, testers and developers will become one role, and there is no obvious distinction but each person’s tasks will be different.


4. Conclusion

In this article, we talked about the developers and testers in the large-scale and long-term software developing team. We provide a new idea of using fuzzy set theory to decide on the correct ratio of developers to testers in agile teams. To explain how to use fuzzy set theory, we showed a example of table including different criteria to evaluate readers’ project.


[1] The Eight Levels of Programmers, April 3, 2009 Jeff Atwood

[2] Johanna Rothman on Jan 1, 2000 in Articles

[3] Cusamano, Michael A. and Richard W. Selby, Microsoft Secrets, Simon and Schuster, Inc., New York, 1995.

[4] Zadeh. Information and Control. 1965.

[5] Chen, Shouyu. philosophical basis of variable fuzzy set theory. s.l. : The Journal of Dalian University of Technology , 2005. 53-57.

[6] Stock Photo – Old shield with sword isolated over white