Assignment 6
COMP 171

Assignment 6: Usability Testing

Next you will perform a critical analysis of your interface using the "think out loud" methodology, with at least 5 users.

Prepare a list of tasks for the test users to perform This is a new, more extensive list, not the one you used for the task-command analysis in Assignment 3. It should take no more than 10 minutes for an experienced user to perform these tasks (it may take longer for your test subjects). Start them off with your program already running; assume we are not testing that aspect of the interface in this test.

Now, get five friends (one at a time) to perform the tasks on the list using your system. They need not be expert users, and they should not be from this course. To the extent that it is possible, they should match the target demographic you chose for your system. Give them the list of tasks, provide your "getting started" documentation if any, start up the system, and ask them to perform the tasks, while thinking aloud, that is, verbalize what they are thinking as they try to figure out and use the system. Watch quietly and take notes on problems they run into and possible solutions. In addition, ask each user to fill out a debriefing questionnaire asking how they liked the system and what problems or misunderstandings they had.


For Assignment 6, each team should hand in:

Role Questions

Engineering: What are the advantages and disadvantages of usability testing by in-person observation as you performed for this assignment? Are there any important metrics that would be more reliably collected by an automated software integration designed for user task analysis? Form a summary of your findings, and use it to make a recommendation on what types of tools you think should be used if your team was to pursue this route for further usability testing. Check out this article ( ) for an overview of usability testing tools currently on the market.

Product Management: How would you convince the stakeholders that the data you collected from five people is worth investing in? What insights can you get from these 5 people that you can't get from a quantitative study that may not only collect more sample size, but also give a definite data insight to act upon? Also, what will be the data that you look for as a product manager once you get a usability study report and why?

UX Research: This article challenges the demographics that were traditionally used by User Researchers: From reading this, what do you think is the best way to recruit? Also, if the findings you find from doing a qualitative study with 5 people contradicts from what is found by another team that ran a quantitative study involving thousands of people, how would you convince the other team that your data is important?

Product Design: Name 3 different methods of user testing, and a thorough scenario/context in which you would use each.