Response article: “Do you have testers?”

This is a response to the article “Do you have testers?” by Clemens Wolff.

Test is dead?

  • “Build the right it” first, “build it right” later.
  • “Test is dead”.
  • “Do you have testers?” – No, and neither should you (probably).

What these sentences quite radically describe is the recent twist in software development practices: Agile-inspired ways of creating software and fading of traditional, “testers in the basement” approach. This is the main theme of the mentioned article, and I would like to comment on this phenomenon and claims I agree with and the ones which I consider wrong.

Agile and Post-Agile era

First of all, I have to agree that software development greatly benefits from Agile-inspired approach. Extreme programming and its concepts like Pair programming, Test-driven development and Continuous integration have remarkably contributed to the whole software engineering world. They helped to reduce the ratio of failing software projects and made managing them much easier. Also, I consider the Joel Spolsky Test a really simplified but extremely accurate and useful metric. It has been written in 2000, although still is widely recognized and delivers an important message to every software team.

However, as Alberto Savoia puts it in the Test is dead speech, we are now in Post-Agile era. And one of the consequences is abandoning the Quality Assurance as we know it. Author of the article supports the claim that for most kinds of software projects, we do not need dedicated software testers in a team anymore. I would like to disagree with this idea and share some of my experiences and thoughts in this matter, as I have been fortunate to work in a software company of a similar class to Amazon. Our team was identically scored in Joel’s test, except the fact that we did have dedicated software testers.

Strict pair programming, unit and integration tests, having whole team testing new feature before its release, shared code responsibility. I totally agree that these practices are extremely beneficial for the project. They also make a job of a developer easier and less stressful.

– “Everyone in the team has a rough idea of what is going on. There is less of a chance of knowledge silos developing”.
– “There is a strong incentive to produce high quality code as your colleagues are going to be directly affected by the mess you cause”.
– “Automated tests give instant feedback”.

But why stop here?

These properties make a perfect software project, so why do we want to limit its perfection and remove testers from the team? Why not let them stay and help us even more?

I think that most importantly teams have to abandon the old-fashioned approach of having testers “in the basement”. A mythical people from the other side of the wall, processing our code changes and returning reports, silently hunting for bugs. Dedicated software testers have to work closely with the development team, preferably seat next to them and communicate constantly. They cannot be considered as enemies hunting for developers’ errors, playing hide and seek. As Spolsky puts it, they have to work there with the team and provide feedback, as soon as possible.

According to the author, as a consequence of foregoing testers the team develops stronger sense of ownership of the product and abandons “(…) just implementing some specification, passing it on to testers (or later business analysts) and never looking back…” practice. Well, this is not the way Quality Assurance should work, this is the old-fashioned approach and is inexistent in a team with healthy developers-testers cooperation. And a team should develop strong sense of ownership of the code regardless of having dedicated testers, just to make the collaboration more efficient.

Build the right it

I would also like to comment on “Build the right it first” idea. This is a great advice one could give for a software company, but… what kind of a company? According to Savoia, mainly a start-up. In his talk, he gives a handful of examples of perfectly-engineered, thoroughly-tested projects who failed to attract any customers. “Build right it” first, “build it right” later concept aims to highly accelerate product development, in order to introduce it into the market as soon as possible. As the real market is the only measure that verifies the real value of a product, not its codebase quality.

However, we are trading time for quality here. Assuming our start-up idea is successful, we end up with a significant technical debt. This is not a trivial problem to overcome. Even Savoia, when asked during his Test is dead speech about this issue, could not provide any silver bullet. And there is not. You might find your users forgiving, as in famous Twitter’s Fail Whale case, or you might loose a significant amount of customers.

Also, all agree that we should thoroughly test aeronautical or medical software. So the “Test is dead” movement communicate between the lines that for non-life-critical projects we should produce lower quality software. This might work well for start-ups, but what is an impact on long-established products with trusting user base? We do not have to ship optical discs to customers any more, yes, but does the fact that software is on the web and easy to update allow us to deliver buggy product? As Spolsky puts it, the “My customers will test the software for me” idea is not so right, giving example of Netscape story. Also, a bug in a product on the web is still a bug and requires the same effort to fix it and carries the same threat of introducing regression into the system.


And where is the truth? Well, probably somewhere in the middle. What I think every development team should try, is aim to strike a balance between trustworthy and effective Agile-inspired methodologies and techniques, and well-established Quality Assurance practices. There is certainly a small revolution happening in Software testing right now, however, the quote “Test is dead” is in my opinion much more of a metaphorical wordplay than a fact.