An effective starting point for designing new technology is to identify an existing problem or user need. Finding a big problem or need often yields an untapped opportunity for design. Observing people can also help you build empathy and think from their point of view. It's simply to point out that if you ask people what they want as opposed to observing what they do, you can easily be led off-track. You should learn how to ask and listen to be able to find out what people's real goals are, and you learn to watch and observe rather than simply convene a focus group to be able to unearth real behaviors and needs.
The reasons for doing the observations coupled with your access to resources, such as equipment, will determine how you record your observations. Direct observation is the cheapest way of recording observations but given the choice, most people would usually opt for some form of automatic recordings, such as video, because it provides a permanent record to which you can return many times later as necessary. Furthermore, there are various tools that you can use to assist you in your analysis.
Direct observation
Individual users may be directly observed doing specially devised tasks or doing their normal work, with the observer making notes about interesting behavior or recording their performance in some way, such as by timing sequences of actions. However, direct observation is often an obtrusive method because the user may be constantly aware of their performance being monitored, which can alter their behavior and performance levels. This phenomenon is known as the Hawthorne effect, after a 1939 study of workers in the Hawthorne, Illinois, plant of the Western Electric Company.
Direct observation may also be very useful early in a project when you are looking for informal feedback, and gaining a picture of the kinds of things that users do and what they like or do not like is more important than formal data. You can also improve the quality of your data by developing a specific recording technique such as shorthand notation If you know exactly what you are looking for then a checklist may be useful so that you can record each time an event occurs. If you want a permanent record then some sort of recording equipment will be necessary such as video, audio, or interaction logging.
Indirect observation
Video logging provides an alternative to direct observation, which is much preferred for the reasons mentioned above. Sometimes the video recording may be synchronized with some form of automatic keystroke or interaction logging, which is built into the system software. Collecting several kinds of data obviously has the potential for providing a very complete picture of the HCI being evaluated. However, there are trade-offs. The different kinds of data collection techniques have to be managed and synchronized and there is more data to analyze, which as you will see, can be very time-consuming. Indirect observation also creates more distance between evaluators and users.
In many studies, several aspects of user activity are monitored by different video cameras. For example, one camera may be focused on the keyboard and screen while another is directed to the user. The second camera can record where the user is looking on the computer screen and her use of secondary material such as manuals. Users' body language can provide useful clues about the way they are feeling about using the system. However, there are some important issues to consider. You need to plan the observation, which means thinking about what you want to find out and what kind of data you need.
Analyzing video data
Although many different kinds of studies can be done using video, researchers agree that analyzing video data can be very time-consuming and that studies should be well planned to avoid wasting time. In less formal studies video can be very useful for informally showing managers or disbelieving designers exactly what problems users encounter. This can provide a strong motivation for redesign. Apart from this informal approach, two other types of analysis can be undertaken: task-based analysis and performance-based analysis. The task-based analysis attempts to determine how the users tackled the tasks given, where the major difficulties lie, and what can be done. The performance-based analysis seeks to obtain clearly defined performance measures from the data collected. The most common measures are: frequency of correct task completion, task timing, use of commands, frequency of user errors, and time is taken up by various cognitive activities, such as pausing within and between commands and reading or inspecting various areas o the screen display.
An important factor in the use of video or any observational method is the trade-off between time spent and depth of analysis. An informal evaluation can be undertaken in a few days, possibly consisting of direct observation, system logging, or video recording typical users undertaking selected tasks. If a more detailed understanding of user actions or task performance is required, the evaluators either have to collect and analyze user protocols or select relevant performance measures and play through any videotape or system log several times to extract all the measures. This greater depth of analysis can be very time-consuming, good support tools can help to reduce analysis time and several are now available on the market.
Photo by Miłosz Klinowski on Unsplash