Chose the right Software Architecture if you want your Open Source project to succeed.

While open source is by definition different from closed-source projects in terms of governance, I believe that the right architecture needs to be chosen if an open-source project is to receive uptake by users and long-term contributions from these same users. The architecture needs to be as modular as possible, and the developers must be able to take different approaches to solve the problems that each new feature aims to solve.

Why is software architecture so important to Open-Source collaborative projects?

While in proprietary software, the developers and users are mostly separate sets of people (except developers of development tools maybe), in Open-Source software the main driving force behind the project is that the users can themselves further develop functionality in the software and contribute it so that other users can benefit from the work. Since now users and the developers may be one and the same, the architecture that the software uses is of great interest to the user. This idea is also presented by Eric Raymond in his 6th rule [1]:

6.Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.

If the software that is being used is very modular in design, it should be easier for a user to add her/his required functionality to the program, without having to understand the rest of the code other than how his new feature will interact with the rest of the software.

Another important aspect in my opinion is how easily the feature can be implemented in a way that is independent from the rest of the software. That is, how free is the new contributor to choose whatever solution is suitable for him to implement the feature, keeping within the design rules set out initially?
For a new user of the software that finds a missing feature, would it be easier for him to add the functionality he requires, or would it be better for him to use another solution, or develop the solution from scratch? The learning curve needs to be kept minimal for the user to actually contribute. It may also be that open source projects will be competing between them when they have similar uses and audiences.
These are the same factors that Baldwin and Clark [2] use in their attempt to use game theory to show that architecture is important in limiting the free riding of open source software, and how architectural decisions help promote the contribution by users. According to the model, having a modular and option value (ability to implement new feature in the easiest way for the developer) is optimal in allowing exchange of valuable work among the developers. And when the resources for contributing to open source software are already limited [3], we really want to minimize the hurdles that such developers encounter to enable them to easily reuse or contribute new code.

Do developers chose this architecture beforehand, or does it come about because of the nature of the collaborative software?

Although the architecture may be crucial, most open source projects that are based around collaborative input by many voluntary developers usually do not start out with the final scale in mind. Moreover, since many open-source projects use agile development methodologies, having a clearly defined architecture from the start is not the norm. However, the argument here is that to facilitate contribution, the modularity of the code should be kept as a high priority to keep the project going more easily. This may come at a cost of increase workload, since refactoring continually will increase workload, and may also make it difficult to keep track of all the modules and how they interact. However, in their research [4], Capra et al. show that the good quality present in open source projects is not just a coincidence. They argue that design quality is both an enabler for the type of governance present in such projects, where everyone can easily collaborate and contribute, as well as being an effect of it. Since the contributors have a sense of ownership of the code, the design quality is given greater importance in the work, possibly at the cost of prolonging the time between releases.

Is this any different from what proprietary software companies try to do?

Within closed-source projects, the design is always kept internal, and the company knows that it has a number of developers that it can commit to the project, whether the developers like it or not. This means that there is less risk of the project being abandoned because it is too hard to contribute to, and the architecture will not influence the number of developers that are contributing to it. However, such companies may take a leaf out of this book, because developers do have feelings and opinions, and making their life easier can only help to improve the quality of the code that they generate for the company. We also know that once a new developer is assigned to a new project, there is a learning curve associated, which can be greatly reduced by having a modular architecture, where the developer does not have to learn how the whole code base works before they can start contributing to the project. This would cut down the cost of personnel moving between projects. It would also make the software malleable, so that new features can be added more easily, and code reuse is also much easier.

This difference in architectural choices is shown in the work of MacCormack et al. [5], where they compare the source code of Linux with that of Mozilla when it was first published in the public domain. The lack of modularity in the Mozilla code is clear, and in fact the Mozilla project underwent a lot of initial work to be modularized, so that it could function better in the open source world.

Interestingly though, all this is arguably contradicted by the empirical study carried out by Paulson et al. [6], where they claim that the results do not support the claim that open-source software is more modular than closed-source. However, the metric they chose to use to verify this hypothesis was correlation between the number of new functions introduced to the number of changes to previous functions. They claim that the number of changes to older functions increased with the number of new functions added, implying that the introduction of new functions may be requiring changes to other functions, indicating that the code is not as modular as claimed. I feel that this is a very simplistic metric, since it simply assumes that in a good modular system functions are only added and no refactoring is required. I believe modularity should be measured by the dependencies between modules, rather than the changes being carried out. Another contributing factor to this result, which the authors point out themselves, is that the projects chosen were mature projects (namely Linux, GCC and Apache) so maintenance may have contributed to the high amount of modifications to functions. Given all this, I still believe that the argument for modularity in open-source projects holds, since we can clearly saw the benefits of such a system, even if they were not currently used as Paulson et al. claim.

So What?

So in conclusion, when we stop to think what architecture the software project that we are about to start should have, whether the project is an open source project or not should be a major factor to consider. Assuming that we do want the project to not be abandoned if the lead developer (us) happens to take a break.

References

[1] E. Raymond, “The cathedral and the bazaar,” Knowledge, Technology & Policy, vol. 12, no. 3, pp. 23–49, 1999.
[2] C. Y. Baldwin and K. B. Clark, “The architecture of participation: Does code architecture mitigate free riding in the open source development model?” Management Science, vol. 52, no. 7, pp. 1116–1127, 2006.
[3] S. Haefliger, G. Von Krogh, and S. Spaeth, “Code reuse in open source software,” Management Science, vol. 54, no. 1, pp. 180–193, 2008.
[4] E. Capra, C. Francalanci, and F. Merlo, “An empirical study on the relation-ship between software design quality, development effort and governance in open source projects,” Software Engineering, IEEE Transactions on, vol. 34, no. 6, pp. 765–782, 2008.
[5] A. MacCormack, J. Rusnak, and C. Y. Baldwin, “Exploring the structure of complex software designs: An empirical study of open source and proprietary code,” Management Science, vol. 52, no. 7, pp. 1015–1030, 2006.
[6] J. W. Paulson, G. Succi, and A. Eberlein, “An empirical study of open-source and closed-source software products,” Software Engineering, IEEE Transactions on, vol. 30, no. 4, pp. 46–256, 2004.

“Do you have Testers?” – “Yes…”

This is a response article to “ Do you have Testers?” by Clemens Wolff
Wolff’s article proposes that the role of the tester is dead, supporting this argument by observations carried out during his work experience rather than academic publications. While, I agree with this view, I would extend the argument that rather than the tester role being dead, it has been reassigned to the user.
Removal of tedious bug hunts
Prior to starting my current MSc, I was working for a company that only did a small amount of testing, and that was being done by testers in the traditional role. By the time I was leaving to continue my studies, we had started to use some unit testing, and I can attest to the improvement in the development life cycle and code quality if one were to use unit testing and automated testing throughout the development. In light of this, I do agree that the traditional role of the tester will diminish to only serve  certain projects that cater for high risk scenarios. From this same work experience, I can also agree that getting the “right it” rather than “getting it right” gives you an advantage on your competitors, not only because it allows you to infiltrate the market early, but it also sets you up as one of the players in the field, proving that you do have the know how to develop such software. However getting the right software out there depends on the feedback from users.
Users as the new testers
At my previous workplace, we used to send programs to the sales team for their feedback on a feature they or the clients had requested, only to find out that their specifications were ambiguous and had been misinterpreted, or even that a feature that we believed would be helpful to the customers, turned out to not be worth the time and effort of development. These errors were minimised by sending the software to some users for their feedback. Since the emphasis is on getting the “right it”, the users are testing whether the software is the “right it” or not.
I strongly feel that the traditional role of the tester has been split up into two, as also hinted in the video posted at the end of Wolff’s article. What was mundane testing has now been covered by automated testing techniques, and what was feedback about the program itself has been shifted to the right crowd with the correct skill set to assess the software : the user. This means that instead of the developer throwing programs at the testers for feedback, they are now throwing it directly at users (with automated testing acting as a filter).
In viewing the user as the new tester, I feel that more attention should be given to the release of the software in more detail. Just like we were careful in choosing the right testers to join a team, and making sure that the process gives the right feedback to the developers, we need to make sure that the right users are the first to see a software (hopefully the most forgiving users), and that the right medium for them to send feedback is made available. Whereas previously the testing department was an internal customer of the development team, and in return the development team was an internal customer of the testing team, we can observe a similar symbiotic relationship between the developers and the users in this post-Agile world.
So in conclusion, I believe that the dialogue should be : “ Do you have testers?”- Yes, thousands… we call them users.

The effectiveness of agile practices for embedded software projects according to industry

Why Embedded software?

As ubiquitous computing pushes the Internet of Things revolution, embedded software development is starting to be placed under more pressure and is facing a new challenges. We are seeing impressively short time to market life cycles, with products that pack ever more computing power and with higher demands from the customers that seem to be ever changing. The scenario is one we would clearly associate as requiring an agile software development method, however embedded software has been know to be reluctant to embrace this rather recent approach to development as opposed to more traditional approaches which it inherited from the hardware product life cycle.

What are the problems for embedded software development to go agile?

Embedded software faces specific hurdles that limit the application of agile practices, including the difficulty (if not outright impossible) to apply release of upgrades or fixes to embedded software, the fact that the new release of software must still be compatible with a certain range of hardware that has already been supplied to the customer, and well as the close ties to the actual development of the hardware. It has been suggested that due to these limitations, embedded software companies would have to make adjustments to the agile practices in order to better accommodate their needs. However, one pitfall in such an attitude is that the easy practices are adopted, and those that would have really brought a change to the company are not adopted using the excuse that they need customization [1].

What is the feedback from industry ?

In the paper [2], O. Salo and P. Abrahamsson present the results of a survey carried out across 13 embedded software development companies that have shown interest in adopting agile practices. Since the sample is know to be biased in favour of adopting agile methods, the authors present this work not as a survey of the whole industry, but rather as a means to better understand the usefulness reported by such companies of their experience of agile. The survey investigated the actual implementation of two agile methods, Scrum and XP, and the perceived usefulness of the methods amongst project managers and software developers of large, medium and small companies.

Results

One clear point that emerges from the survey is that Extreme Programming (XP) practices were more popular than Scrum, with 54% of the responses stating that the practices were systematically, mostly or sometimes applied, as opposed to 27% for Scrum for the respective range. However, a clear point to note is that knowledge of Scrum practices seems to be less widespread than XP, with the percentage of “never” and “I don’t know” for scrum practices being much higher than that for XP.

Another possible reason for this discrepancy is the difference between XP and Scrum, where XP is presented as a set of 12 practices that can be implemented somewhat independently of each other. On the other hand Scrum is a whole “process framework”, and is not easily implemented successfully as separate parts, making adoption a more disruptive move. By definition Scrum should NOT be implemented in separate parts, as this undermines the whole framework. [3]

The feedback regarding the experience was positive in both XP and Scrum, however even here, the experience of XP seemed to have more positive feedback (90%) rather than Scrum (70%).

One particular comment by the authors that struck me as strange was that they claim that the feedback regarding actual experience proved more positive than the predicted positive experience by those who had not yet implemented agile techniques. They use this to argue that agile techniques are better than perceived before adoption. I find it hard to draw this conclusion, when this could be clearly biased, since people who have negative expectations of a technology will probably not take it up, resulting in the higher negative prediction. For all we know, these companies may be justified in their belief since the practice may not be directly effective for the company/project.

Comments about the research.

What appears clearly from the research is that companies implement some of the agile principles more than others. While this may work for XP, it is strange for Scrum since it should be implemented as a whole, rather than just parts. (It was split according to scrum meetings, which may not make sense using just a few of them). However this may be a clear indication that the agile techniques need to be modified to cater for the embedded software development, since some principles seem to be more beneficial than others.

Another point to note was that Test-Driven Development was one of the least techniques that was implemented from the XP techniques. I find this surprising, since Test-Driven Development would greatly aid the identification of problems very early in the implementation stage, and has been identified as a means of increasing the reliability of code developed. I wonder whether this percentage remains through today, 5 years after this survey was carried out.

I also find interesting the range of risk that the respondents identified. Although the authors did not discuss this in detail, I believe that the risk assessment of the software may greatly influence the way and extent that agile software development is implemented. In fact in this survey, most projects involved software that could potentially cause loss of discretionary funds or just loss of comfort. Only 2 projects involved potential loss of life or more. I believe that it would be of interest investigating the correlation between the risk involved and the way the company adopts agile techniques, as I speculate that it would be very hard for companies that produce high risk equipment/embedded software to change long standing development methods with stringent stages and control to a more flexible and less documented agile technique.

Conclusion?

I believe that due to the advancement in both software development methods, as well as the rapid increase in smarter devices that surround us, it would be very interesting to carry out this research once again now, 5 years down the line, and observe current trends, with clearer indication of the complexity, scale and risk associated with the embedded software. However, I believe that this paper gives enough proof that projects that used agile techniques for embedded software, usually considered conservative, proved useful and a positive experience.

References

[1] B. Boehm, “Get ready for agile methods, with care,” Computer, vol. 35, no. 1, pp. 64–69, 2002.

[2] O. Salo and P. Abrahamsson, “Agile methods in european embedded software development organisations: a survey on the actual use and use-fulness of extreme programming and scrum,” Software, IET, vol. 2, no. 1,pp. 58–64, 2008.

[3] K. Schwaber and J. Sutherland. (2011) The scrum guide–the definitive guide to scrum: The rules of the game.