1.What kinds of information and insights did it give you about the usability of the prototype?
As user (task1 Sydney Opera House)
In the task one, it took me a long time to find the event but then I book tickets quickly. In fact, I was a little embarrassed when I couldn’t locate the event and I eventually got it using searching with observer’s reminding. Therefore, I thought I must have some different expression on my face, like frown, lack of patient. However, my observer commented that I just had a poker face. In this case, I think although non-verbal behavior is a important part of usability testing, but not quite useful with the person who is not good at express their emotion and physical behavior superficially.
It is a quite common way to evaluate usability. When I was doing the think-aloud, I tried my best to keep talking and verbalize what was happening internal to me, like ‘I am looking at menu’, ‘I think I need to scroll down to see if I can find the event’, etc. But it was not easy to describe everything, since my thoughts always went faster than my words.
Observing and note-taking simultaneously is quite a challenge for me, especially when users were facing difficulties to move on and felt confused about what they need to do, they would have different posture and facial behavior. Therefore, during these period, I was prone to miss some non-verbal feedback while I was marking on the sheet. Fortunately, video recording helped me a lot to review and recheck the results. In addition, I think it is important to be familiar with the form before user observation, so that I can pay more attention on observation.
It is easier for me to do note-taking for verbal comments than facial expression. My users seemed to be really experienced about think-aloud, which means they kept talking all the time and gave comments appropriately. During think-aloud, in order not to confuse the user, evaluator also need to give concrete sub-tasks at right time.
2.What aspects of the technique worked well or were frustrating?
User observation like non-verbal behaviors are useful when user facing difficulties, we can easily observe their expressions like frowning, grimacing, and we can also recognize whether they have some different posture. But it is hard to observe when users don’t have enough expressions and postures.
Think-aloud requires user to keep talking, which is sometimes hard for the participants to really adhere. When user forget to say something important, observers can’t recognize what the user is thinking about.
However, both techniques are easy to conduct to receive a large amount of feedback without high cost.