The Three R’s of Software Evolution

1. Introduction

Software evolution and maintenance is a major part of the software development process. For large scale software projects the maintenance costs are often much more than the development costs. In the software evolution process, three techniques are often considered: Reverse engineering, Reengineering and Refactoring. These three techniques are widely used throughout the software evolution and maintenance phase to ensure the software is maintainable and useful throughout its lifetime. In this blog post, I will discuss software maintenance in relation to the three techniques and provide my opinions on them.

2. Change is inevitable

Changes in software are inevitable, and these changes result in maintenance activities that can be classified into several categories [1]:

  • Corrective maintenance: concerned with defects or problems found in software after deployment.

  • Adaptive maintenance: adapt to changes in requirements to ensure the software product meets the user’s needs.

  • Perfective maintenance: support the addition of new functionalities on top of existing system architecture or improve the existing system’s performance and reliability.

  • Preventive maintenance: concerned with bug fixes to software after it is deployed to prevent faults occurring in the software.

 

As a result of these changes, we must continuously maintain the software to ensure it still remains useful and do not end up becoming a legacy system that is difficult to evolve and maintain. Furthermore, Lehman proposed a set of laws of software evolution [2]. These laws describe issues which usually apply to large scale real-world software which are constantly evolved and are used by many people.

3. Reverse engineering

Reverse engineering is the process of analysing a software system in order to gain an understanding of the system in terms of the design and specifications. The aims of this process are to generate detailed documentation for a legacy system to aid code understanding or to learn about design decisions during the initial development of a project. Once the system is reverse engineered, the design and specifications can be used as feedback to improve the existing system or as ideas for creating a new system.

 The process usually starts off with manual code reviews or the use static analysis tools on a software system to extract information which is then stored in a datastore. The information from the datastore is then reviewed to generate relevant documentation which then allows us to understand the structure of the system.

Figure 3. Overview of the process of reverse engineering [3]

I have had some experience in reverse engineering, most notably from the system design project. The system design project was a special case of software development: once the project is finished, the development ended and the codebase was not maintained. The result was that it became a legacy system that was difficult to maintain and lacked documentation for future groups to reuse components from the system. In our group we used the idea of pair programming in XP to analyse what the code in different parts of the system were trying to achieve. In some cases, we had to use our initiative and knowledge to understand the components due to badly written and confusing code.

In my opinion, the development of the system design project relates well to Lehman’s second law of software evolution: Increasing Complexity. For example, the idea of adding new functionality to a robot (for a better chance of beating other robots in a football game) means the software becomes more complex over time thus requiring additional work to maintain and reduce the complexity of the system.

4. Reengineering

Reengineering is the process of creating a maintainable system from a unmaintainable, legacy system, it is often seen as a large scale version of refactoring. The process of reengineering involve tasks such as: source code translation (porting the software to a different platform which uses a modern high level language); restructuring or rewriting of software components to aid understanding and documentation. A few advantages of reengineering include:

  • Reduced risk: applying reengineering to the existing system rather than re-developing the software from scratch means we don’t need to define new requirements or specifications which could be ambiguous if they are not clearly defined.

  • Reduced cost: reengineering is a lot cheaper than re-developing software from scratch with a reason that staff do not need to be reallocated to do different tasks or tasks they are not familiar with which results in additional training.

 

However, reengineering does not apply to all situations. Back to the example of the system design project, we had looked at the different components of the codebase such as: communication – how the bluetooth communicator sends commands to the robot; I2C Multiplexer board [4] – controlling the speed and direction of the motors for the robot and the various vision libraries that can be used to provide orientation for the robot on the pitch. Since there was a lack of documentation and that reading code is more difficult than writing our own, we decided to re-write our own component for the I2C Multiplexer. Another reason for this was that we had used different style motors which required additional tuning of speed constants that differed from the previous year’s group. The result of re-writing the component saved us time and effort rather than having to completely restructure the original component.

 5. Refactoring

Refactoring is the process of cleaning up code to improve the internal system structure without changing the functionality or behaviour of the system. The result of refactoring has many benefits such as: improved readability and understanding of code; increased flexibility of a system allowing new functionalities to be added easily.

However, in some software companies where they are under pressure to deliver a piece of software on time, the aim might be to “do the simplest thing that possibly works” which often result in refactoring being ignored with arguments such as “we don’t have time to refactor” or “refactoring might break the system”. My response to the former argument is that refactoring is worth the time and effort which pays off in the long-term maintenance of the project. Without dedicating effort to refactoring, we may have a case in which a lead developer leaves the company causing the entire project to be scrapped – no one else in the team is able to understand the internal structure of the system as layers upon layers of functionality have been added (making the system more and more complex) resulting in eventual failure of the project. For the latter argument, the use of fully developed test suites will ensure that when we refactor the code, the behaviour of the system is preserved. This is done by running the tests on the system after each modification to ensure the restructuring hasn’t affected the system.

 

From my experience of developing an Android app for my final year project, I realised that refactoring lays the foundation for building a good quality application. I often had to refactor code in order to build more complex features which otherwise led to increased coupling and duplication of code. As an example, during the development of a LocationManager component [5] to detect changes in the user’s location, the functionality was initially added to all other components which required access to location data. This introduced lots of duplicated code for each component that included this functionality which made it difficult to maintain – when the LocationManager component was updated the rest of the components all required updating. After refactoring, the LocationManager component was implemented as an interface allowing other components to implement its features as required.

6. Conclusion

In this blog post I have discussed the techniques of Reverse engineering, Reengineering and Refactoring in general software development and in relation to my experiences in both the system design project and my final year project. Reengineering is especially useful in large-scale software development where the risks and costs of re-designing a system are high whereas Refactoring should be done as good practice in all areas of software development.

References:

[1] Wikipedia. Categories of software maintenance: http://en.wikipedia.org/wiki/Software_maintenance#Categories_of_maintenance_in_ISO.2FIEC_14764

 

[2] Lehman’s Laws of Software Evolution (Section 2.2.2):

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4659256

 

[3] Software evolution, Reengineering and Reverse engineering: http://opencourseware.kfupm.edu.sa/colleges/ccse/ics/swe316/files/2_17_Software%20Evolution,%20Reengineering%20and%20Reverse%20Engineering.pdf

 

[4] I2C Multiplexer: http://www.inf.ed.ac.uk/teaching/courses/sdp/i2c-motor-multiplexer.pdf

[5] Android LocationManager: http://developer.android.com/reference/android/location/LocationManager.html

Response article: “How to make Software succeed”

Introduction

This is a response to “How to make Software succeed” post by s1263235 [1].

The author initially presents a few cases of software failures ranging from complete failure with the entire project scrapped and resulting in a discontinued service – Google Buzz to project going over budget and delayed delivery – Universal Credit. The author then describes some issues that result in the possible failure of a software project and expresses their view of possible ways to resolve these issues to prevent or reduce the chance of software failure.

The three main criteria that a software project is often judged on in terms of its degree of success are: whether the project was delivered on schedule; whether development costs are within budget and whether the project meets the user’s needs in terms of scope and quality. My opinion is that when any one of these criteria are not met, the software is considered to have failed.

In this response post, I will offer my opinions on some of the main reasons of software failure listed by the author and discuss other potential problems that leads to the failure of a software project.

Project delivered late

The author stated that the delay of project delivery is a reason for the failure of a software project and possible ways to prevent this include careful design of the project; giving up on ideas which are infeasible to implement or following a reasonable schedule. I agree with this reason, as I stated above, this does contribute to the failure of a project. However, I would like to elaborate on the point of “following a reasonable schedule”. In large-scale software development, the schedule is often set by the project management team or even the stakeholders themselves. They usually do not have knowledge of the details of the software project and could set an unreasonable deadline which makes delivering the project on time difficult. A way to solve this problem to is to provide adequate training to the project management team so that they have an understanding of the software project and will then be able to do careful planning at each stage of the project to draw up a more realistic schedule.

 Project ran over budget

I agree with the author’s point that a project which ran over budget is another case leading to software failure. The author states that estimating the budget that should be allocated to a project is a difficult task. To expand on this, since the estimation of the size and effort of a project is very difficult – estimating the size and effort of a project at the early stages is inaccurate (where estimation is actually useful) whereas estimating near the end of the project becomes fairly useless. This means it is often difficult to estimate an accurate budget in relation to the size and effort of the project. The overall result of this inaccurate estimation causes the project to run over budget.

There are some additional points that help reduce the chance of a project running over budget [2]:

  • Limit or reduce the scope of the project: we should concentrate on the main requirements of the project and limit the focus on adding additional features or functionality. However this requires careful planning as reducing scope of the project may not satisfy the user’s requirements.

  • Regular budget financial forecast: by having a budget plan before the start of the project and reviewing the plan on a regular basis during the project reduces the chance of the project running over budget. After conducting budget reviews, if the budget is constrained we may have to take out some functionality and reduce the scope of the project.

  • Resource allocation: when the budget is constrained, we could consider allocating resources to a part of the project that is less resource intensive.

Poor communication

The author also points out that poor communication between the customers and the development team as well as the developers within the development team [3] leads to project failure. I agree with this point, and my opinion is that it is crucial that the developers themselves are clear of their role in developing the software project, they have to work in unity and aim towards the goal of meeting the initial requirements of the project. Good communication allows them to resolve any misunderstandings and ambiguities in technical requirements as well as identifying other problems along the way.

An example that aid the communication between developers is the concept of pair programming in Extreme Programming (XP) methodology. This practice allows developers to work together and understand problems and specifications better. It is also helpful when a new member joins the development team, they get to know the project details when explained by another experienced developer.

Frequent communication between customers and developers are also important in order to capture requirements early on in the project lifecycle. In the XP methodology, the on-site customer practice means a customer is always with the development team.  This means the development team can be notified of any changes to the requirements immediately and plan changes accordingly. The customer can write user stories to allow developers to gain further understanding of the functional requirements.

 

Other possible reasons

I will now provide a few reasons in addition to the author’s reasons of causes of project failures [4].

Inability to cope with a project’s complexity

Development teams that target towards new technologies or a specific industry are often troubled by a software project’s complexity. This is mainly due to inadequate knowledge of the field. The developers may not have any experience developing software for that particular area and the customers may not know exactly what they want from the software which results in incomplete requirements and specifications. This leads to the possible failure of the project.

A possible way to prevent this issue is to draw up a contingency plan which includes possible delays in the project; increase the budget allocated for the project should it require additional resources. However, this will only be helpful in the short-term to compensate for additional costs and delays but will not prevent catastrophic failures.

Use of third-party software components

The problem occurs when a development team uses third-party software components as part of their software architecture in the project. The third-party software components may be untrusted; not fully tested and have poor code quality. Using these components may result in security vulnerabilities and introduce bugs into our system. The poor quality of code will often result in developers spending a large amount of time understanding it rather than writing their own. Maintenance is also a problem, especially when the third-party software components are updated, all software components in our system that depend on those also have to be updated.

Conclusion

In this response article, I have highlighted some reasons of software failures discussed by the author; provided my opinions on those issues as well as added a few other causes of failures. Overall, I agree with the main point discussed by the author that software failure is very common in large-scale projects. However, with some careful planning and following the points described in this article can greatly reduce the chance of software failure.

 

References:

[1] https://blog.inf.ed.ac.uk/sapm/2014/02/14/how-to-make-software-succeed/

[2] http://project-management.com/what-to-do-if-your-project-runs-over-budget-and-how-to-prevent-it/

[3] http://blog.azoft.com/communication-within-software-development-company/

[4] http://blog.azoft.com/preventing-software-development-project-failure/

 

 

 

 

Architectural patterns for Mobile Application Development

1. Introduction

Mobile application development share common software design principles as desktop application development. However, typical mobile applications follow the mobile application lifecycle management process [1], [2] that differs from desktop applications.

Mobile applications often require more user interaction compared to its desktop counterpart, usually waiting for an action from the user (such as a button click) before responding to the user with an updated UI showing the requested information. We will discuss a software pattern that focusses on the interactive aspects of mobile applications as well as a more general high-level decomposition pattern and discuss their uses in mobile application development.

2. Model View Controller (MVC)

The Model View Controller (MVC) pattern [3], [4] is a very popular approach for the development of a mobile application. We observe that most mobile applications’ core operation is to retrieve data from a data store and update the user interface with the newly requested information based on the user’s inputs. The logical sense is to tie the user interface components with the data store components. However, since the user interface components are updated regularly to accommodate the changing user requirements and new technologies – more so than the data store components, we introduce extra coupling in our system. The aim of this pattern is to separate the components of user interface (View); core functionality and data (Model) and the response to user inputs (Controller).

2.1 MVC in the context of Android applications

I will first provide a brief overview of the components of the Android system. The main components [5] are:

  • Activity – represents a single user interface class. Activities are usually packaged together to form the UI components of the application.

  • Service – allow tasks to be executed in background threads (such as networking operations) without affecting UI components.

  • Content Provider – enable data to be stored within the application with the use of SQLite database or SharedPreferences (data stored in an XML file on the device).

  • Broadcast Receiver – responds to announcements from the system (such as low battery warning) and provides notifications to the user.

 

In my experience of developing Android applications in the SELP third year project and my final year project, my opinion is that the suitability of the MVC pattern depends on the context of the application in question.

My project is based on a tourism application consisting of: UI components (update a list of nearby points of interest and displaying these to the user); service components (location listener for tracking user location using GPS and detect changes, background running tasks to fetch data from other services); data components (for storing and retrieving user preferences); widget components (buttons, edit text, text views for handling user clicks and inputs).

The UI components is made up of a package of Activities that resemble the View component of the pattern where each Activity updates and modifies the UI throughout its lifecycle. The service and data components represent the Model component of the pattern in that it monitors the behaviour of the data and core application domain. It also handles requests from the View and Controller for information about its state and instructions for changing its state respectively. The widget components are similar to the Controller in that it waits for user inputs (such as a button click) then interprets those inputs and notifies the Model or View of the change.

In the high-level overview of my application, it seems that MVC patterns fits well with the Android ecosystem with each application component more or less mapping to the Model, View and Controller. However with closer inspection of the details, things are not so straightforward. An example is the Activity component that forms part of the application. Each Activity consists of a Java file and a corresponding XML layout file. We define the UI layout of the Activity in the XML file and implement the functionality and behaviour in the Java file (separation of user interface with application logic). When we use widget components in our Activity, we first have to define the widgets as XML elements in the layout file to instantiate the widget and its attributes. The handling of the behaviour of the widget in response to user actions is then implemented in the Java file. However, we often have to access the widget’s attributes from the XML layout while implementing its functionality and this violates the concept of separating the View from the Model/Controller in the MVC pattern.

A possible solution to this is to use an Observer pattern within MVC. We could make the Activity components (View) implement an Observer interface to receive updates and notifications of changes to the application logic (Model’s state). When a change occurs, the application logic notifies all its Activities (observers) of the state change. This eliminates the coupling between the Model and View.

Figure 2. Using Observer within MVC [4]

3. Layered Abstraction

The Layered Abstraction pattern [6] consists of a hierarchy of layers. At each layer, we have components that work together within the same level of abstraction, with the layer below providing functionality for the layer above.

The Android architecture [7] follows this Layered Abstraction pattern. It consists of four layers of abstraction with direct communication only between a layer and the layer directly above or below it:

  • Applications – this layer consists of a set of core applications as well as developed applications.

  • Application Framework – layer which contains the main components of the Android system (Activity, Service, Content Provider, Broadcast Receiver) as well as other system components.

  • Libraries – consists of core Android system libraries as well as a lightweight relational database management system – SQLite.

  • Linux Kernel – contains device drivers which provides communication with the device’s hardware.

Figure 3. Android system architecture [8]

I will now provide an example scenario from my tourism application that follows this software pattern. When a user uses the application to determine their current location, a background running task is executed from the Service component which sends a location request from Applications to the Application Framework layer. The location manager sees the request and provides periodic updates of the device’s geographical location (latitude, longitude coordinates) with the help of the core system libraries in the Libraries layer. The system libraries communicate with the WiFi and GPS device drivers in the Linux Kernel which interact with the device’s hardware to receive GPS signals. Once the signals are received the notifications are propagated back up the layers to the Application layer which updates the user with their location information.

4. Conclusion

In this blog post, I discussed the interactive MVC pattern and the high-level decomposition Layered Abstraction pattern and their relation with mobile application development.

In terms of the MVC pattern, I think for applications with large UI components it might be worthwhile following this pattern of design as the constant update and modifications of the user interface due to the rapidly changing nature of user requirements will result in less work maintaining the core business logic of the application.

With the Layered Abstraction pattern, since the Android system follows the abstraction layer architecture as discussed above, we can say that this pattern forms the foundation for all Android applications that makes request to services and receives notifications from them.

 References:

[1]http://www.theserverside.com/feature/Mobile-ALM-Guidance-Andoid-and-iOS-development-is-not-business-as-usual

[2]http://searchsoftwarequality.techtarget.com/Mobile-ALM-FAQ-Answers-to-mobile-development-questions

[3] http://www.inf.ed.ac.uk/teaching/courses/sapm/2013-2014/sapm-all.html#/463

[4] http://msdn.microsoft.com/en-us/library/ff649643.aspx

[5] http://developer.android.com/guide/components/fundamentals.html

[6] http://www.inf.ed.ac.uk/teaching/courses/sapm/2013-2014/sapm-all.html#/443

[7] http://en.wikibooks.org/wiki/Android/Introduction#Architectural_Overview

[8] http://elinux.org/Android_Architecture