Why don’t we use Continuous Integration?

1. Introduction:

While reading some articles or blog posts ([1], [2], [3]) on Continuous Integration it’s easy to come to the conclusion that this approach ensures your code quality, reduces risks and sets programmers minds at ease. It is not dependant on any particular software development methodology. If it is so, I thought that everybody should use it for the projects they carry out. However, it is not a case – there are many companies that still haven’t started development with CI and I question myself “why?”. This article aims to answer that question.

2. Continuous Integration itself:

Continuous Integration [1] is a software development practice which assumes constant integration of code under development. Developers check-ins to the integration server should be made few times a day. Process of code submission is not done until entire compilation, tests and inspection is carried out, proving no problems. Otherwise instant measures should be taken by the developer, to fix any issues. Most important outcome is that there is a great probability that software is functional and the newest version is available to whoever wants it.

3. Background:

Some of the issues that I raise in this article are backed up with examples taken from my experience in a company I worked for. Therefore a brief introduction to it is necessary. The firm is a small start-up project, which manufactures home automation system devices. The software developed there is to be considered as long-term projects and large when compared to the number of people working on them. Main software developed there is:

  • low level programs for field devices (sensors and actuators);
  • applications for controllers running embedded (mainly Linux) operating systems or for mobile devices. This software was the biggest one, constantly extended with newer modules;
  • installation configuration system for Desktop computers.

When I joined that company, there was only one experienced programmer working for it. After some time, he left leaving me as the only permanent one, with others being partially hired for some tasks, when there was a need for that. Don’t take the impression of the company, as working in terms of anti-patterns – I mention here its bad aspects only, since the article requires so. I believe there are many companies like this one, especially small ones, that work the same way.

4. Why CI has not been implemented yet:

Although many of its advantages, CI doesn’t make its way to all the projects being developed. Most people seem to know its advantages and the outcome it gives to the project. It sounds like a panacea to many of the issues concerning software development – it enforces programming good practices and forces developers to be consequent. However, many aspects such as human state of mind of both managers and programmers, developers’ habits and abilities or the company targets (e.g. money first, quality later) stop CI from being the part of their projects.

4.1. Commits frequency:

First of all, I find that basic assumption of CI could be problematic in some circumstances – very frequent code check-outs. To realize it, programmer commits are to be small what poses a difficulty of dividing work into really tiny chunks [2]. Even though developers tend to modularize and divide their work into manageable pieces, it’s often hard to have such granularity of modifications that each is ready in few hours. Every feature being developed requires full life-cycle consisting of consideration, planning and implementation. Especially, if the feature is to be tested before check-in, it’s problematic to reduce its development time.

Moreover, incremental committing just doesn’t work sometimes. This could happen, for instance, if software architecture was not well thought. That kind of issue may be even more frequent in Agile development. Let’s say, one needs to modify part of the code, to adapt it to some changed requirements. If multiple modules are dependent on that change, the tests may fail if the modification is not made in its entirety. Thus bigger change is to be made and it could take more time, before the program works again in as a whole. Then, in order not to break the build, the commits may not be just checked into the main repository for even few day, whatever is needed.

The above example may be a little exaggerated but there is even more to it – human mentality. Programmers, who start working on project parts in the beginning of the day, naturally tend to first analyse the problem and after solution is chosen, implement that strategy. That means most of the commits could occur in the end of the day, which makes integration even more troublesome – they have to get into the queue to the integration server and that takes time.

Increasing commits to the high frequency expected by CI requires significant mind shift. Thus, if there is no mentor, who may help in such process (e.g. experienced manager or some senior developer, who has done CI before) this transition may become a barrier not to be crossed.

4.2. Mainline only:

Another aspect, which seems counter-intuitive and may result in turning CI down, is requirement to keep every code in a mainline. This seems even more insane, when CI assumes that broken build gets highest developer’s priority. The problem here is that it is not uncommon that programmers work on some feature in their spare time, between other, more important works. That feature may not be usable or functional for a longer period, and thus incorporating it would very probably break the build.

In my company, there were many projects (mostly extensions to the controlling projects), that were started because there was a need for such functionality. However, there was no need for them to be completely finished or polished (e.g. easily configurable as the rest of the software), because they were meant to be used just in a few instances. Once proof-of-concept was working, there was no additional time devoted for finishing them and making a cohesive part of the entire project. The reason for this was, of course, limitation of resources. If the need occurred, they were meant to be continued, but until then there they had no appropriate tests and they were not acting as a cohesive part of the project. Thus they were held as a separate branches, which according to [1] is against the CI idea.

The main reason why some developed modules should not be on the mainline is lack of resources devoted by managers to it. If it is not finished or tests are not complete, and other duties have higher priority, committing it to the main code is not what CI expects.

4.3. Automate your tests:

Creating automated tests for every part of the project, when introducing CI, may impose another difficulty. As mentioned, automated tests are very important part of the CI process and they should cover the code pretty well to reduce the risk of potential bugs [4]. Even though it’s a common practice to write tests, some of them are very difficult to automate.

In our team we didn’t have people who were specialized in tests. This led to situation that testing of more complex modules like those responsible for configuration generation (writing configuration validation was as very difficult) or some GUI components was very challenging and time consuming. Given limited number of developers and time, some of them were carried out manually or by the group of people who used applications (like beta-testers). In the second case, if some bug was detected, first its harmfulness was estimated and then fix priority set. So the “small” bug might be there left for appropriate time to come, which is again against the CI ideas.

One may argue, that difficulty of automated tests writing is not a point against Continuous Integration. CI may still be used, just with small coverage or no code tested at all. However, in such a case it is not that efficient (only compilation correctness is proved) and convincing the manager of CI usability is much more difficult. So, when decision about taking the track of CI is to be made, there are less advantages to speak out.

4.4. Appropriate timing:

One important aspect that should be addressed is the time in the project lice cycle, when CI is to be introduced. It is much easier to have the project running with CI, if it was started with CI in mind. Then, all the components of the project (such as tests, scripts, even its architecture) are prepared so that it’s compatible with the idea and tools used for CI.

It is not easy, though, if the project is already ongoing (this could be the case of long-term projects). In this case it could require too significant modifications or time-consuming additions to be made, to consider CI as a good solution (of course this approach is short-sighted). In such situations, it should be estimated, what are the advantages and costs of switching to CI – the latter ones could be too high, so even if there is a team will, managers don’t agree.

4.5. Direct costs:

Not to be forgotten obstacle of introducing CI in the company may be the cost of integration servers and time cost, required to run these machines [5]. There are many open source projects such as Cruise Control [6] that could be used for that purpose. However, there is still the need for some training (especially, when no one in the team has done that before). The hardware cost depends on the type of project being tested. If it’s small and tests are not resources consuming, some old computer may be used [7]. Otherwise, newer machine should be used. In cases of more complicated processes, where build time exceeds 10 minutes limit [1], the build should be divided into few steps over multiple computers. Again, this may pose significant costs, especially for smaller companies. Larger ones, probably won’t see that as a drawback, since in comparison to money they pay as salary to the software engineers is much more insignificant cost.

4.6. Project ROI:

Last thing, not connected to large projects, but worth mentioning while CI is discussed is return of investment [5]. In case of small programs with short life cycle period, especially for those that long term maintenance is not assumed, the burden of setting things up could be too high.

5. Summary:

In my opinion, presented issues are the most important ones that prevent software developer teams from  introducing Continuous Integration. It is worth for a team considering using CI to detect if these apply to them. If so, they should not be discouraged but try more step-by-step approach. Continuous Integration has significant advantages which are especially obvious for projects where more than one developer work on. It is worth the effort in a longer run, since it reduces integration risks, gives better overview of the project state, encourage developers to work proactively and not to procrastinate. It is a really good way to achieve high quality product just by sticking to some general rules.

6. Bibliography:

[1] http://www.martinfowler.com/articles/continuousIntegration.html
[2] http://www.jrothman.com/blog/mpd/2011/12/is-the-cost-of-continuous-integration-worth-the-value-on-your-program-part-1.html (also 2nd and 3rd parts);
[3] Paul Duvall, Stephen M. Matyas, and Andrew Glover. 2007. Continuous Integration: Improving Software Quality and Reducing Risk (The Addison-Wesley Signature Series). Addison-Wesley Professional
[4] http://www.ibm.com/developerworks/java/library/j-ap11297/index.html
[5] http://stackoverflow.com/questions/214695/what-are-some-arguments-against-using-continuous-integration
[6] http://cruisecontrol.sourceforge.net/
[7] http://www.jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html
[8] http://callport.blogspot.co.uk/2009/02/continuous-integration-should-we-always.html

2 thoughts on “Why don’t we use Continuous Integration?”

  1. Nice post. I think you make the point that CI is not the obvious choice for many groups quite well.

    Story time…

    I just finished integrating CI for a handful of my projects about 6 months ago. The environment is extremely similar to what you described in your post. Small company and dev team, limited resources etc etc. So I can completely relate to the reasons you have outlined, they make a lot of sense too me. In large part, they are why I resisted CI for so long.

    I like to think that what we have built out now is “CI – lite”. We have the source control, the build server and the test runner, but the tests themselves are very limited. They only test core functionality where we absolutely cannot afford regressions and other problems (firmware updates, daylight savings time transitions, and other important data transfers).

    And to add to your post a bit, doing a firmware update takes about 20 seconds. The aggregate across all the device types is quite large. There is no way a battery of unit tests will take under 5 minutes. As you mentioned, technically, this is not appropriate for CI.

    Up to this point the largest gain from CI is organization, stability and confidence across all projects. Because of varying reasons, development pace is very high and often times quality is only average – and that is the expectation. As developers we are assigned to something, reassigned… reassigned again, then back to the original assignment. It’s horrible. As this cycle repeats over and over, disorganization sets in and you get everything from software rot too completely broken builds. I have found that CI helps keep the projects “on the rails”.

Comments are closed.