Learn Software Development » User

Getting team members to attend user feedback sessions, usability being very important (Part 4)..

ashish — Tue, 01 May 2012 03:41:38 +0000

In the previous post (methods for getting team to view user testing), I had talked about the ways in which a development team can view the results (rather, the actual process) of the user testing (sending somebody to the actual location; getting a real time broadcast of the event; running the videos later). However, in this post, I will try to get into more detail of what the user testing actually reveals, and how it more than just a video of the session. Interpreting user testing does require a fair degree of professionalism, and to get the best value out of the user testing, it is recommended that you have an expert in the field of user testing to run through the entire process.
Why do you need an expert to run the user testing process ? Well, because if you don’t run the process properly, you get results that may not reflect the sample of the population you are looking for as customers, and you can make changes in your application based on the results you get, which means that you are now going off on the wrong path, and that can be very expensive for the application or even spell doom.
A user testing expert will work with the experience designer and the product manager to determine the questions that need to be tested, to break up the queries in a form that can be made into test cases for the users, such that you can have a query for the test users who can then take a series of actions that answer the query (something like, for an imaging application: “If you want to remove the red eye from a photo using application xyd, what would you do ?”). In such a query, you would find out how the users react, which menu do they try, do they look baffled or start trying out things and get to the required tool within a few seconds, and so on. The key in this is framing the questions in such a way that the user can understand it quickly, and yet in a way that the response meets the objective of the user testing process.
One of the biggest areas where the user testing expert starts out with is about ensuring that the right people are selected for the user testing. This is not so easy as it sounds. The conditions for selecting the right sample need to be enumerated based on discussion with the Product Manager (and these could be multiple people from the same type of sample, or a broader sample to try to represent more sections of the desired user set (and keep in mind, since user testing can be pretty expensive, in the tens of thousands, you cannot in most cases go for large sets)), and then the right set of people are selected. Part of this is whether the tests are needed to be carried out at the location of the users, or in a controlled location. Using the user location gives the background of the user environment, and can be helpful in determining the amount of time and focused attention that the user gives to the application. To be contd ..

A Common Sense Approach to Web Usability	The FlowingData Guide to Design, Visualization, and Statistics	Designing with the Mind in Mind

Will continue this in Part 5 …

Getting team members to attend user feedback sessions, usability being very important (Part 3)..

ashish — Thu, 26 Apr 2012 19:41:46 +0000

In the previous post of the series (Usability testing and its importance to the team), we talked about how to get the team to understand the value of user testing of new concepts and workflows, especially through the process of taking the team members through cases where the team members are shown an example of the users finding workflows difficult, and the team expected this workflow to be a bit hit.
In this post, I will talk more about how the team can actually get to experience the process of user testing. Here are some methods that depend on the techniques that the company may have adopted for this purpose:
- Actually being there for the user testing. This is easiest when the user testing takes place in the same location as where the engineering team is based. If they are there in the same location, then it is fairly easy to take the team to the room or location where the user testing is happening, and putting them where they can observe the user testing but not disturb the actual place of user testing. I have seen teams watching the proceeding from a place with a glass through which they can see inside but the people inside cannot see outside.
- When the user testing is happening in a different geographic location from the engineering team. In this case, the attempt should be to ensure that as many of the team members can be sent for the actual user testing. This gets difficult when the team members are in a different country and the user testing is in a different geography. Given the need to ensure that the team members understand the importance of user testing, it does make sense to try and make the budget for this trip, and get the people who are the most involved in the design aspects for the new workflows to make the trip.
- When the team members are not able to make the trip (or most of them are not able to do so), the next best option for the remaining members of the team is to view the user testing on a real time basis. This is through using some sort of broadcasting situation where the user testing is broadcasted through the user of a solution such as Cisco Web broadcasting or Adobe Connect, such that the team members can see the user testing. With modern networks, the clarity of such videos can be pretty high, and with proper placements of cameras and audio, the team can see it similar to what the user testing researcher sees.
- However, there are situations where all of the above situations cannot happen. When the team is in a different geography, especially when the team is based in India, the time zones are very different and it would be hard for the team to actually get involved in viewing the presentation real time. In such cases, the videos of the user testing should be shown to the team, and in a setting where the entire team watches the presentation, and are led by either the Product Manager or the User Experience Designer who can walk through the critical parts of the actual user testing.

A Common Sense Approach to Web Usability	The FlowingData Guide to Design, Visualization, and Statistics	Designing with the Mind in Mind

Will continue this in Part 4 (link) …

Getting team members to attend user feedback sessions, usability being very important (Part 2)..

ashish — Tue, 24 Apr 2012 18:51:24 +0000

In the previous post (User feedback and teams), the post was about the importance of getting feedback from users where you are evaluating changes in the workflows in your product, or modifying some of the features. In this post, I will write more of this topic, specifically with reference to the team members and their exposure to the user feedback sessions.
In today’s world, a lot of teams are spread geographically and hence getting all of them to the same location where the user testing is happening is not possible or feasible (it can be done if the budget is there, but given tough economic times and a lot of stress on the budget, it just does not seem feasible to ensure that the team members can all make it to the actual physical location where the user testing is done). In our case, we had team members in New York, in Romania and in Bangalore, and the actual testing was done in London.
So what are the alternatives that can be employed ? We will talk about some techniques that can be used by the team members to view these user testing sessions, but for now, we need to also ensure that these team members see the value in viewing these user testing sessions.
This is a difficult task. How do you get team members to see the benefits of being more involved with user testing, viewing how users evaluate the workflows that have been built or modified. One easy way of getting them more interested is taking one of the most radical feedbacks from the user testing and having the product manager present this feedback to the team (this could be something such as the users not finding the entry point to a feature which was supposed to be the most useful features for users). This should present a huge challenge to the team, since it would not be easy for many of the team members to understand how users could react in such a way.
This is the perfect point to get the team involved, by challenging them through such methods. If the user session has been video recorded, then at this point the product manager or the user experience designer should show the video from this point onwards (after first displaying some information about the user which shows how the user is part of the target population, and if possible, showing their setup where they use the application including the times of the day when the user typically uses the software). An important part is to emphasize the point where the users face issues in the workflow, and with modern techniques, it is even possible to monitor the movement of the eyes of the user. Being able to see this gives a very good insight into how the users react, and can be very surprising for the team in terms of the way that the users react.

A Common Sense Approach to Web Usability	The FlowingData Guide to Design, Visualization, and Statistics	Designing with the Mind in Mind

Will continue this in Part 3 (link) …

Getting team members to attend user feedback sessions, usability being very important (Part 1)..

ashish — Sat, 21 Apr 2012 21:22:49 +0000

In the previous post (Needing user feedback?), I was talking about a case where there needs to be a balance between having user sessions to get feedback about new proposed plans, vs. the case whereby a lot of new stuff is such that users may not have thought of and where getting feedback may not be the right approach.
This post however takes the classical case whereby the team is looking at some changes in some workflow in the product and wants to get these validated by a cross section of users. Consider whereby the team has got a lot of feedback about some of the features in a product release and has decided to try and make some changes in the feature set and the workflows. There is a certain conception when such changes need to happen, and the product manager and the experience designer would have figured out how to modify the workflows in order to incorporate the user feedback. However, such changes can be tricky to get right. Even earlier, when the workflow was designed, it would have been done to optimize user workflows, and the fact that users had more feedback means that getting the workflow right is not as easy as it seems.
The problem is that when teams start thinking about a new design and get involved in the discussions, there is a strong tendency to get so involved that you start to feel that whatever you are designing is the right way of doing things. It needs interaction with the users to ensure that the team members keep an open mind and keep on thinking about whether the design will work for them or not. You would be surprised to discover some of the outputs from usability testing, whereby users tend to spend a lot of time on workflows that you would have thought were very intuitive, and also seen other areas whereby the users found a particular feature so interesting, but on which the team had not really given a second thought.
The problems comes in terms of budgets, whereby when there is a need for usability testing, it is typically the product managers and the experience designers who get to spend time in terms of usability, and who get to interact with the users. So they tend to be more understanding of the areas where the users may get stuck, but the average team member does not really have this kind of understanding. For this purpose, what is required is to ensure that a cross section of team members is exposed to this usability testing, and spend as much time as possible with the users on whom the usability testing is being carried out. As much as possible, there is a need to make budgetary allocations for the same, and always have some of the team members being able to observe the usability testing. Doing so ensures that team members get a lot of experience with how users actually think (in many cases, I could see that the team members were getting a rude shock in terms of how they could see the users interacting with the software application, and this in turn caused some changes to the way in which team members thought.

A Common Sense Approach to Web Usability	The FlowingData Guide to Design, Visualization, and Statistics	Designing with the Mind in Mind

Will continue this in Part 2 (link) …

Getting user feedback on new designs, but do you go totally by user feedback ?

ashish — Thu, 19 Apr 2012 19:46:11 +0000

This is a tricky topic. Conventional wisdom says that when you are planning a new product, it is very important to get user feedback through the process, multiple times. The logic: When the team is working on a new application or software product, they tend to get too involved with their own thoughts, their rationale and seek to believe that these are exactly what the users want. There are a number of products that were developed without user feedback and testing, and a large number of those do not really set the user community on fire, many of them are downright failures, since the users really are not looking for the workflows that the team thought would be loved by the users.
To try and ensure that the people involved in the application design and development remain close to what the users want, and also get a chance to vet their design and prototypes, typically a restricted user testing is done. A sample of users are taken, and depending on the current state of development of the application, either screen shots, or prototypes, or the working product is shown to the users and they are encouraged to use the application or evaluate whatever has been given to them. Viewing the interaction of users with the software is invaluable to the design team, since it gives them a fair idea of what the users like, what they dislike, and whether they are comfortable with the workflows that are being shown to them.
So far so good; however, over the past few years, there is another thought that is gaining attention. A number of new hardware and software applications have become successful where the application is meeting a need that users may not have thought about, and in such cases, user testing and feedback may not be the sole criteria on which to go by. It would be very much possible to think that some of the great apps on the iPhone, such as Instagram or others were not really designed and launched after user feedback.
Instead, you had a committed set of application developers who knew what they wanted, and were passionate about what they were doing. If some users did not like it, this would not have shaken them. In such cases, what happens is that even very honest user feedback would not have been the right way to go about; and this is the concept that is now shaking development companies around the world. When you are trying to build something in the area of the web / tablets / phones / Facebook app, a lot of stuff that is being worked upon is new, or atleast the workflows being developed are new. In such a case, the companies do not even go in for user feedback, instead working on releasing the software, seeing the amount of user acceptance and then tweaking (or dropping) the application, and go on. This makes life difficult for companies that have worked the traditional way, and which depend on user feedback as a way of validating new stuff that they are doing.

Some great books from Amazon on usability and user feedback:

A Common Sense Approach to Web Usability	The FlowingData Guide to Design, Visualization, and Statistics	Designing with the Mind in Mind