This was a super fun exercise that was easy to get into, because the tasks were so specific and explicit.
As the user, I felt that it took effort to translate my thought processes into words that the evaluators would find useful. This was because I am not usually very…conscious of how I engage with objects or interfaces. In trying to be a helpful user, it taught me that the heuristics we bring to our everyday interactions are active decisions, even if they are subconscious and difficult to isolate.
Even though speaking my thoughts aloud wasn’t very easy, it definitely made me aware of object interaction. For users like myself, the non-verbal recording of my reactions was a good gauge of my response to the task.
I enjoyed being the evaluator more. It was easier to analyze the usability of the prototype without having to be the one using it.
As the evaluator, I could see very clearly where our user was experiencing obstacles in her usage of the STAPLES prototype. It helped that she spoke aloud out her confusion — we were able to prompt her to elicit her assumptions: she assumed the ‘Filter Results’ would be on the right side, and was thus disoriented when she couldn’t find it there.
We couldn’t always decide on how to fill in the questionnaire, e.g. “how many times did she frown? how many times did she squint at the screen? how many questions and frustrations did she have?” This was a tricky technique to work with.
As with many observations, evaluation is a subjective process. Perhaps that is why multiple evaluators are a good idea: so that their impressions can be aggregated.