What kinds of information and insights did it give you about the usability of the prototype?
Observing the physical behaviour of a user as they are undertaking a task was extremely useful as it made me attend to non-verbal cues. Physical cues rarely lie, and can be more telling than verbal behaviour. Physical expressions can reveal pain points for the user which they may not necessarily articulate.
As the user being observed, it became apparent that I was showing frustration in my face that I was not necessarily voicing. Therefore attending to physical cues during a user evaluation can give you critical insight into what’s frustrating about the prototype, or what’s pleasurable to use about the prototype.
As the user, Thinking Aloud was extremely helpful because it helped me to clarify my own mental processes and the models that I used to navigate a website, for example. The types of information it gave me included whether the prototype was easy to use, whether it cohered with my expectations, whether I thought there was anything missing, whether I understood the function of everything that was presented to me.
Listening to another user verbalise their own thoughts was similarly helpful in gaining insights as to the mental processes they went through in trying to accomplish the task. This gives you an insight into the underlying assumptions and mental models they have in their head, which they have gained with past experiences. This gives the tester useful information about affordances and consistency.
What were aspects of the technique that worked well or were frustrating?
As the user, thinking aloud can sometimes be difficult because it can be hard to clearly articulate your thoughts and impressions. You have to get over a certain level of self-consciousness. There are many things you take for granted, which you may not feel you need to voice. Or you may have too many thoughts about what you see, and you are unsure which ones are strictly relevant to the tester.
It was difficult to record non-verbal behaviour in real-time because it is so dynamic. Therefore taking a video and playing it back may be the best method to accurately capture this data. Also it is unclear to what extent observing non-behaviour would be useful without the aid of thinking aloud. Using behavioural data to supplement thinking aloud might be the best practice.
Recording the data in the format provided allowed us to quantify the data, which actually gives us quantitative data — this can give us the data upon which to base our design decisions.
Overall, the experience of evaluation reinforced the value of user testing and showed me how useful it could be in the iteration process. Conducting user evaluations is a critical source of feedback. Doing it with “expert” users like fellow classmates was a vital source of feedback as they can give you highly informed critiques.