In this tutorial we conducted and participant in user observation and think-aloud evaluation based on three websites.
What kinds of information and insights did it give you about the usability of the prototype?
1.1 User observation review as user (My test website: Sydney Opera House):
There’re two types of user observation, one is real-time observation, that the evaluator observe the user doing the task and, at the same time, take down notes. The other is that the evaluator watch a video taken already about the user doing the task.
In the second way there’s not much interaction. Although the user knows that his action is being recorded, he may less likely to feel nervous or behave unnaturally. On the other hand, as a user in a real-time observation, I may get some hint from the evaluator’s facial expression and writing that whether he thinks I am performing well or not.
On this website, Sydney Opera House, I found the task easy to complete. I personally think this website is beautiful and well-constructed. Only in details does it have some flaws that the visual-weight of some buttons were not appropriate.
1.2 Think-aloud review as user:
From my own experience, user think-aloud feels like trying to guide myself passing through the task. More importantly, by telling what I thought and what I wanted to do, I learned about my own mental model of this prototype.
2.1 User observation review as evaluator( officeworks.com):
The useful forms helped me, the evaluator, to evaluate how much frustrated the user was about the prototype. User’s behaviors and facial expressions directly shows his satisfactions and dissatisfactions. However the nature and personality of the user will impact on his performance.
2.2 Think-aloud review as evaluator:
Because the user is speaking out his ideas and descripting his actions, as an evaluator I can easily get information about good and bad points in the prototype. So there’s no need for me to “guess” the user’s true thoughts through observation. For this reason think-aloud has become one of the most useful and powerful evaluating tool.
What aspects of the technique worked well or were frustrating?
- As user. Almost every worked well. The only frustrating thing was that in think-aloud, I had to keep talking and describing my actions on the website. For some times it bothered me because I couldn’t use the website frequently as I was able to for the reason that I had to do language-organizing at the same time.
- As evaluator. The most frustrating part came when the user misunderstood the task. The task introduction was designed carefully in a way that will not lead the user. It only tells user what to do but never tells them how to do. So sometimes it’s very difficult in think-aloud when the user can not understand the task properly. And I can not give him hints so as not to lead him.
Nevertheless, taking down notes quickly and correctly is another challenge, both when observing and conducting a think-aloud. Using video recording is a good way to avoid this issue.
Overall, I think think-aloud can give me better evaluation result while user observation is more convenient to conduct and also more convenient for user to participant.