Plans are nothing; planning is everything[1]

[1] Dwight D. Eisenhower

 

We already know that a lot of IT projects fail, and they have been high failing rate for a long time.  I recently had a discussion with my colleague who worked at Initrode, and albeit the company was successful and a major player in the world market, he was perplexed about how his manager handled the team. It seemed that his managers core task was to estimate the time for each tasks and then track it and adjust it if the task took a different time, and that act of adjusting was which perplexed my colleague the most:

Why would you do something, if you knew you would have to change it later?

Plans are Nothing

Schedule for getting back on schedule is a little behind schedule

Indeed, by that definition in continuously changing and intangible environment like computer science and type of plan is pointless. We learned that estimating is difficult and requires a lot of knowledge and feedback to improve the quality of our estimates, however there is inherent danger in estimating cost and task in IT, because the domain of computer science keep changing continuously, which invalidates our knowledge of the problem.

One significant example is TAOCP, which keeps getting updated instead of publishing new volumes. Although this example might be superfluous in the context of developing and managing large scale software, it is a perfect example, how a technical dept can arise from the advance of the environment, not caused by the decisions during development.

If I know I will change it later, why do it now?

Planning is Everything

Putting deadlines aside, if we were to divide development methodologies based on amount of the whole plan needed in advance we would get the two following extreme cases (form lectures):

  • figure out how to do it the best way, then do it
  • just do it, and figure out if it is right later

The first choice aims to indicate any possible “future” technical dept and tries to overpay it in advance, hoping that once the dept arises it will be enough. The second acknowledges the dept and delays paying it while hoping that, once the dept arises, the resources available will be enough.

I will not argue which side of the spectrum is better as it mostly depends on the project in question and it a topic for a whole another blog post. I just wanted to make a simple observation:

Just as doing is important for a startup, planning is important for established projects.

And by planning I do not mean plans (hint above). I would like to call it “the ability to change plans based on current situation”, but then I would be describing a principle of Agile methodology; the truth is planning is a more fundamental action that that.

Proper planning increases the knowledge of the current state of the project and improves the estimates about the projects; proper planning is somehow orthogonal to all software methodologies — all of them require it, they just make different assumptions about their estimates and evaluate them in different order.

Therefore now matter what methodology you use and what you do, go back to your plan,  re-evaluate it and make sure it properly reflects the state of your project. Possibly as often as you can, even if it is just adjusting  the tasks times for your team.

Reply to “On scripting languages and Rockstar programmers” by s1038803.

I agree with the author that Rockstar programmers are rare. However, firstly I would like to point that there is a subtle difference between a programmer and developer, as the original author uses the term interchangeably. Although there are many definitions for both of the above circulating on the web I will choose a simple distinction between those terms.

  • Programmer is someone who writes code. Therefore a good programmer is someone who writes good code in a good way.
  •  Developer is someone who creates software. Therefore a good developer is someone who creates good software in a good way.

The above definition is deliberately vague to point out the main difference: the programmer is evaluated based on the code he writes, while the developer is evaluated based on the software he created. Neither good code does not imply good software nor good software does require good code (I do realise those things are highly correlated); the author put a lot of emphasis on the efficiency when writing code, a trait often associated with a good programmer, whereas it might not be relevant to creating and maintaining large scale projects.

On his blog, Scott Hanselman defined a Rockstar developer as someone who makes complex problems disappear, and then he made an argument that instead of Rockstar developers, there are Rockstar teams. The main reason behind it, that only a well interplaying team will make really complex problems just disappear, not a single programming diva.

On the other hand, the blog post I am replying to made a certain generalisation towards the “average” developer with which I disagree: he did not take into account the knowledge and experience when them.

It is true that scripting programming languages are more expressive and allow to write more succinct programs, however, when comparing someone’s performance while using a language, you should always take into account if that person knows the language already and if she or he has previous experience with it.

The main reason for that is even though it might be more difficult to do some task A in a systems language rather than a scripting language, an average developer might will do that task on schedule and within the budget provided that he has sufficient knowledge and experience with the task. Again, he might not be able to deliver the same task in an unfamiliar scripting language, even though the language itself might be simpler than the systems language. The knowledge of the tools you are using is extremely important, but bear in mind that there are certain habits developers get into, that make it really difficult to transition between different tools.

For instance, a developer used to C might struggle with Python’s dynamic typing — he will not check if he is passing proper arguments to methods because compiler was doing that for him. This case might seem silly, but it occasionally occurs to me or my colleagues when prototyping something in Python. Proper testing would probably check if everything is fine, but testing the arguments for types all the time is a significant drag and results in unmaintainable code.

Given all of the above and I assume that every developer is a programmer, but not every programmer is a developer, which I will not discuss in depth, because it is a topic for a whole another blog post. Therefore Rockstar developers are even harder to find than Rockstar programmers and I my opinion they are a myth.

Hack and Slash

My recent experience with a horribly overdue project, which consisted of
probably every software planning fallacy possible, gave me an interesting insight into
getting a substantial part of it back on track. Some statements made in this
blog post will be exclusive to creating web applications, nevertheless
I would like to present it, since the way it relates to the grand scheme of
designing and maintaining large scale software is particularly interesting,
especially when designing components.

The Problem

Vast numbers of software projects still fail (as in go over the time or budget), even though there are various software methodologies that aim to address the problem of intangibility in software projects and increase the chance of success. Furthermore, even applying the currently most hip methodology or anything-driven development we cannot even estimate if that does improve our chances of delivering the project on time within budget, as there is no single silver bullet.

Just Hack (and Slash)!

The decision which allowed the mentioned project to get back on track was to (surprisingly) remove people associated with the project. The mythical man-month stated that increasing the number of people will increase the time and cost associated with project, but claiming the opposite would surely seem preposterous.

Yet, After slashing (figuratively!) around half of the team the project was reevaluated. While keeping the same same functional requirements, the code base was reduced. General and heavyweight components were replaced by smaller ones, allowing to control the behaviour of the application not by tweaking and maintaining big components, but by connecting smaller components in a controlled way. Even though from the outer perspective there is little difference, the effect on the source code was astonishing — the code base was reduced to 20% of the original size.

Large, because it was… big?

Finally, the team started to get things done faster, even though it consisted of a smaller number of people. The code base was smaller, meaning that there was less maintenance required — but it also meant that there were less thing to test, so there was an additional increase of productivity, because implementing new fully tested features required writing even less additional code and allowed focusing more on productive programming (less tests -> lest new tests -> less code). In the aftermath, the original project was suffering under its own size, which was caused by the use of large components, which were required to allow a larger number of people to work on the project.

Conclusion

It is repeatedly stated that there is high percentage of projects failing to be delivered on time and within the budget, even though we — developers — have gained some insight about planning and executing such projects. We realise that certain decisions, like throwing people and money to projects at the latter stage are clearly pointless. However, maybe those relations hold in the opposite way as well. One way to ensure that a large project will not fail would be to limit the size of the project in the first place — agile methodologies address seem to address the problem in a similar way.

Finally, the idea of reducing the project size to prevent a large project from failing by reducing it size seems fairly straightforward and not exactly innovative, yet why the result of doing so felt so surprising?