Combining quantity and quality in large-scale projects

Introduction

Most software managers agree that small groups of experienced and really capable programmers are far more effective than huge groups of programmers working on a project. They also claim that they can easily coordinate a small group of efficient programmers while in big groups communication is always a big problem. However, some large software projects are too large to be completed by a small group of programmers, even if this group contains some of the best programmers available in the industry. This article focuses on one of the most important dilemmas in Software Engineering: quantity or quality is more important in large-scale projects? Is there any solution that could combine both of them and bring the optimal results?

Summary

Fred Brooks in his book “The Mythical Man-Month”-Chapter 3 [1] presents a study conducted by Sackman, Erikson and Grant about programmers’ productivity. This study proved that there is a significant difference between good and bad programmers in terms of productivity. A good and experienced programmer can be up to 10 times more productive than a bad or inexperienced programmer, even though the salary paid to an experienced programmer is usually double the salary an inexperienced programmer gets. Therefore, managers would prefer to hire a couple of really good programmers and allocate tasks to them, as this would be more cost effective.

However, large-scale products require thousands of work hours and cannot be completed just by small groups of excellent programmers. So, hiring more programmers, even if they are mediocre or inexperienced, is absolutely essential for large projects. But this poses a new challenge for projects managers. They have to find solutions for combining good and bad programmers in an efficient way. They also have to coordinate a large number of persons and a large number of groups. Finally, communication and decision making between groups and their members is also a difficult problem for managers.

Harlam Mills proposed a solution [1] about such large projects. According to Mills all large projects should be divided into smaller tasks. These tasks should be allocated to small groups of roughly 10 persons which have a specific hierarchy. Mills states that these groups should operate like a surgical team where a surgeon is surrounded by anaesthesiologists and nurses, each of them having a specific role. The structure of this team should be the following:

 

  • The surgeon or the chief programmer. He is the team leader and the person who designs the program, writes the code, runs tests and writes documentation. He should pose versatile skills and significant experience.
  • The copilot. He is the surgeon’s assistant. He should be able to do surgeon’s work and he has similar skills but slightly less experience. Copilot can also offer advice to the surgeon but the surgeon in the only person who takes all crucial decisions.
  • The administrator is the person who handles human and material resources. The surgeon should typically take decisions about personnel, money or machines but he rarely has enough time to do it. Therefore, the administrator is responsible for these issues and also he is the person who is in contact with other groups.
  • The editor is the person who writes the documentation. The surgeon usually prepares a draft of the documentation and then it is the editor’s job to criticize it, rewrite it and provide references.
  • Two secretaries. The administrator and the editor should have a secretary. Administrator’s secretary usually is responsible for correspondence.
  • The program clerk is responsible for keeping technical records of the team. According to Mills proposal all files are visible to all team members. So, the clerk collects all input and output files and stores them in a chronological archive.
  • The toolsmith is responsible for interactive services such as text editing, file editing and debugging. He should reassure that all this services are working properly as they are essential tools for the team.
  • The tester is the person who runs tests and provides assistance in code debugging. The surgeon is the person who writes the tests but a tester is necessary to run these tests and provide feedback about the correctness of the code.
  • The language lawyer is a programming language expert. He is responsible for finding an efficient way to use the programming language in order to solve demanding and tricky problems. He should be a system designer and usually two or more groups share one language lawyer.

 

According to Mills this structure has two main advantages compared to an ordinary team of two programmers. Firstly, in an ordinary team the work is divided so each programmer needs its own disk space and access. However, surgeon and copilot work together thus saving material and time [1]. Secondly, in a team of two programmers all disagreements have to be solved by discussion until they reach in an agreement or compromise; but this is often time consuming. On the other hand, the surgeon is higher in the team hierarchy so he is the one decides and takes the responsibility of decision-making. This hierarchy is much more effective in software projects [1].

Discussion

Mills proposal has many positive ideas if examined in an abstract way. The main point of Mills idea is hierarchical structure. This structure is nowadays a commonplace for almost all projects. In each organization or group there should always be a person who will always take a decision.

Non-hierarchical structures are not efficient as group members always express different opinions and proposals. This is reasonable as each person has a different understanding of the problem and a different solution. But in such groups one solution should be decided so that everyone should work on it. In a non-hierarchical structure decision making should be made under discussion, cooperation and team consensus. However, this is ineffective, time consuming and in some cases impossible. Therefore, a team leader is always essential in order to coordinate and team members and take a final decision when it is necessary.

According to Brooks [1] when more people are added to large-scale project overall productivity falls especially due to bad communication. Mills proposal gives a solution to this problem. Mills proposed structure takes advantage of small groups efficiency in order to deal with large scale projects. Combined with the benefits of hierarchical structure mentioned above communication between groups is much easier following this model. Chief managers can communicate and give instructions to group leaders, the surgeons, and they can also coordinate their group giving appropriate instructions. Moreover, following this proposal the best and more experienced programmers are in the key positions designing and writing the code, while less experienced or bad programmers take the editor, clerk or tester role. Therefore, good programmers can focus on the important part of the project –design and coding- and at the same time they do not have to waste time in other tasks.

However, Mills proposal is not always effective if examined in a more detailed view. Hierarchical structure is considered essential for all large projects but the question is how can we divide personnel into smaller groups and how we can organize these groups. Mills’ surgical team is not suitable for all problems. Some projects may require teams with more programmers and less auxiliary personnel such as clerks, testers or editors. According to Hans Van Vliet’ book [2] groups can be divided according to people’s specialties. For instance a group could contain more than two software engineers, web developers or network administrators working on different parts of a specific task, under the guidance of an experienced chief programmer.

To sum up, Mills proposal offered an excellent approach on diving large-scale projects to smaller tasks undertaken by small groups of programmers. It also demonstrated the importance of hierarchical structures and showed an efficient way of using experienced programmers without wasting their time on time consuming tasks. However, groups’ organization could be different as more programmers may be required. Therefore, Mills plan should be the basis for all large-scale projects but group division should be decided according to the needs of the problem.

References

[1]. Frederick P. Brooks Jr. – The Mythical Man-month: Essays on Software Engineering – Chapter 3- 1975

[2] Software Engineering: Principles and Practice – Third Edition – Hans Van Vliet – Chapter 3 – 2008