How to find underlying reason from user feedback to make your software better

1. Introduction

Everyone knows the importance of collecting user feedback of your software product. The simple reason is that user feedback reflect their satisfaction and user experience of your software product, and these can have big impact of sales of your software product.  Many software products failed because their developers have not done a good job of collecting useful feedbacks from their users. This article is going to talk about different ways to collect user feedback, analysis the advantages and disadvantages of these ways, and how to improve the effect of user feedback. I will propose some ideas to help us find underlying reason from user feedback to make our software product better.

2.Existing ways to collect user feedback

2.1 Using feedback form

The most common way to collect user feedback is feedback form. Companies post some questions on their mobile phone applications, websites and so on.

Figure 1. Mobile Feedback Form [1]

44.png

Figure 2. Website Feedback Form [2]

Advantage of feedback form is that it does not cost much money and you can collect feedback directly from your existing user. Disadvantages of using feedback form are very low rate of collecting user feedback. Although some companies try to use some reward to attract their users fill in their survey, this method can lead to another problem. Many users just fill in their survey very quickly and randomly to get the reward, this can make the result of feedback forms wrong and cause the company greater losses.

2.2 Using third-party service

Some company uses third-party service to collect users feedback. For example, Google Analytics[3] is one of the best free analytics applications for getting basic traffic and visitor data. You can use Google analytics to monitor the user’s activities of your website or mobile apps every day, such as number of visit, where your users from and so on.

Advantage of using third-party service to get user feedback is that you can get every user of your software product anonymously, and the result of the feedback is more reliable.  The disadvantages are you need to pay extra money if you need premium service, and service can not be customized by yourself if third-party company does not provide this service. Of course, data security is another issue you have to consider.

2.3 Eye tracking technology

Academia and business have big interest of eye tracking technology. Eye tracking is used in research as a means for insight into human behavior [4]. For example, Tobii Mobile Device Testing [4] solution lets you study how users experience mobile websites and apps or how they consume mobile ads on mobile devices and tablets [5].

55.png

Figure 3. Mobile Device Testing by using eye tracking technology[5]

Advantages of eye tracking technology include it is a very trendy technology and the future development of this technology are very broad. Disadvantages of eye tracking technology include it just give you answer of what area users are watching, but it does not give you why user look at this area. Correctness of this technology should be tested and verified.

3.How to find underlying reason from user feedback

One challenge of collecting user feedback is that we need to make sure the representativeness and accuracy of data. When we ask our old users why they are no longer use our software product, they always use “no time” and “no money” as a reason to get rid of us, but the real reasons are always unknown.  How could we know it is the real reason that our user like or dislike our software product? We need to find underlying reason instead of superficial reason from user feedback.

3.1 five why’s

Five why’s is invented by IDEO [6], which is a global design firm that takes a human-centered, design-based approach to helping organizations in the public and private sectors innovate and grow. Idea of this five why’s method is very simple: asking your user five questions about an issue continuously to get the deep and progressive answer, and this method allow your user express their deep thoughts and reasons. Here is an example of five why’s method from IDEO as follows, and this example shows the true reason of user exercise:

6.png

Figure 4. an example of five why’s method from IDEO [6]

When we test our software product, five why’s method can become as follows:

 
Game developer: “Why don’t you log in our  game?”
User: “I have no money and no time.”
Game developer: “Do you play other games which are similar to our game recently?”
User: “Yes, but I am not very interested in it.”
Game developer: ”What kind of game you like now?”
User: ”I guess some sports game.”
Game developer: ”Why you like sports game now?”
User: “I feel very exciting when I can create my own role in the game, and I feel myself play in the court.”
Game developer: “What differences do you think between our game and sport game?”
User: ”I think the sport game soundtrack are much cooler than your game, their 3D effect really exciting and I can play with my friends online. Your game does not have these functions.”

After these five question, game developer finally find the real reason of user does not play their game.  User play other games because they like better game soundtrack, visual effect and multiplayer game.

3.2 Behavior Analysis

When we use user feedback survey on the internet, it is difficult to ask questions to get deep reason. It is hard for user to answer the why they behave like this instead of that, but it is easier for them to answer with their behavior. Furthermore, it is more reliable to answer questions about behaviors. There are some questions we don’t need to ask our users directly to get the deep reason, instead, we can just analysis their behaviors to get the underlying reason.

3.3 Indirect comparison of other related products

Sometimes users are unaware of the real reasons behind their actions. We may not get specific reasons if we ask questions directly. So we can use indirect comparison of other

related products to inspire users to dig out the underlying reasons.

I will show you an example as follows:

Interviewer: why don’t you use our premium service?
User: “I have no money.”
Interviewer: “Are you still use other premium service?”
User: “Yes. I use premium service from X company.”
Interviewer: ”Why you use X company’s premium service instead of our premium service?”
User: “Because I think their service are more useful, and I have more privilege.”
Interviewer: “What privilege? Could you give us an example?”
User: “Unlimited downloading”

This use case shows us it is not because user have no money that he does not use our service. It is because our premium service is unattractive to users, and user want unlimited downloading privilege.

4.Conclusion

In this article, we talked about different ways to collect user feedback which are used by today’s technology company. However, after analysis these different ways, we find they all have their disadvantages. So we proposed some ideas to improve the accuracy of user feedback to make the result more reliable. These ideas could be rules to follow in the future if technology company want to design their method to collect user feedback. These ideas can help software company find the underlying reason why or why not the user like their product, and these more reliable user feedback could be guidelines to follow if they want improve their existing product or develop new products.

Reference

[1] Mobile Feedback Form. 2014 Neemware Inc.

http://www.neemware.com/products/mobile-feedback-form/

[2] Preparing for a Site Redesign. March 20th, 2013. Bourke Design.

http://www.bourkedesign.com/2013/posts/preparing-for-a-site-redesign/

[3] Google Analytics. 2013.

http://www.google.com/analytics/

[4] Tobii. 2013 http://www.tobii.com/

[5] Tobii Mobile Device Testing Solution. 2013 Tobii Technology.

http://www.tobii.com/en/eye-tracking-research/global/products/solutions/mobile-device-testing/

[6] IDEO http://www.ideo.com/uk/