In the previous post (methods for getting team to view user testing), I had talked about the ways in which a development team can view the results (rather, the actual process) of the user testing (sending somebody to the actual location; getting a real time broadcast of the event; running the videos later). However, in this post, I will try to get into more detail of what the user testing actually reveals, and how it more than just a video of the session. Interpreting user testing does require a fair degree of professionalism, and to get the best value out of the user testing, it is recommended that you have an expert in the field of user testing to run through the entire process.
Why do you need an expert to run the user testing process ? Well, because if you don’t run the process properly, you get results that may not reflect the sample of the population you are looking for as customers, and you can make changes in your application based on the results you get, which means that you are now going off on the wrong path, and that can be very expensive for the application or even spell doom.
A user testing expert will work with the experience designer and the product manager to determine the questions that need to be tested, to break up the queries in a form that can be made into test cases for the users, such that you can have a query for the test users who can then take a series of actions that answer the query (something like, for an imaging application: “If you want to remove the red eye from a photo using application xyd, what would you do ?”). In such a query, you would find out how the users react, which menu do they try, do they look baffled or start trying out things and get to the required tool within a few seconds, and so on. The key in this is framing the questions in such a way that the user can understand it quickly, and yet in a way that the response meets the objective of the user testing process.
One of the biggest areas where the user testing expert starts out with is about ensuring that the right people are selected for the user testing. This is not so easy as it sounds. The conditions for selecting the right sample need to be enumerated based on discussion with the Product Manager (and these could be multiple people from the same type of sample, or a broader sample to try to represent more sections of the desired user set (and keep in mind, since user testing can be pretty expensive, in the tens of thousands, you cannot in most cases go for large sets)), and then the right set of people are selected. Part of this is whether the tests are needed to be carried out at the location of the users, or in a controlled location. Using the user location gives the background of the user environment, and can be helpful in determining the amount of time and focused attention that the user gives to the application. To be contd ..
|A Common Sense Approach to Web Usability||The FlowingData Guide to Design, Visualization, and Statistics||Designing with the Mind in Mind|
Will continue this in Part 5 …