1) What kinds of information and insights did it give you about the usability of the prototype?

As a user
I was performing task 2, and it was relatively easy to complete the task. Verbalising my thoughts made me become aware of the little problems I was facing. If I did not vocalize my thoughts, I may overlook on these problems and would think that my experience was quite perfect. My physical expressions came quite naturally while I was in the process of figuring out the task, so I was not able to notice them.
As evaluator
User Observation: I watched one of my group members perform task 1, and I noticed that he had several expressions while verbalizing his thoughts. Most of his physical expressions were in line with his verbalized thoughts, but there were a few expressions that were contradicting. His body language provided some insights into his actual feelings about the usability of the product. Based on think-aloud results, the user found the website difficult to navigate with a number of distractions from other unrelated events. This is reflected in most of his physical expressions, where he was observed to be surprised and focused on trying to complete his task. Although he found the task hard, he was smiling in some instances, so this was hard to interpret. Overall, the website (product) in task 1 seems not to be user-friendly.
Think-Aloud: In the think-aloud exercise, I observed another group member verbalize her thoughts for task 3. At first, she mentioned that it was quite hard to locate the faculty name in the website. She was not able to find it even through typing in the search bar. It took a while for her to mention that she found the faculty name under the “Science & Technology” tab, and clicking it directed her to the Google Scholar page link, where she said she found the page easily. Overall, the website (product) in task 3 seems to be user-friendly in general, but could be further improved.
2) What aspects of the technique worked well or were frustrating?
As a user
I guess being able to express myself freely through verbalizing thoughts and facial expressions was able to make me more aware of my feelings towards using the product. However, as I am thinking about the task at the same time, it is hard for me to be fully aware of all my thoughts and expressions. I could only have a general sense of how I felt in the whole process.
As an evaluator
User Observation: This technique was useful in interpreting the actual feelings of the user. However, as the user mostly had his head facing downward to perform the task on his mobile device, it was hard to capture all of his physical expressions. It was also hard to note verbal and non-verbal behaviours at the same time, as I had to focus on writing some notes and also try catching all of his expressions as much as possible. I guess having a recording of the user’s behavior would be best to really capture the whole user experience process and review, so that any information is not missed out during the observation study.
Think-Aloud: Generally, it was easy to note down the verbalized thoughts of the user. Sometimes, I had to prompt the user to say something when she went quiet while using the website. A frustrating thing was that I had to imagine what location/interface part the user was mentioning in the website, as I was not able to observe her screen and jot down my notes at the same time.
Advertisements