Cooperative evaluation is a technique to improve a user interface specification by detecting the possible usability problems in an early prototype or partial simulation. It sets down procedures by which a designer can work with the sort of people who will ultimately use the software in their daily work so that together they can identify potential problems and their solutions. Participative evaluation differs from cooperative evaluation in that it is more open and subject to greater control by users. In this article, we will have a better look at the differences.
Cooperative evaluation is designed to be a low-cost technique that can be used by designers and users without specialist HCI knowledge. Very little training is needed to do the cooperative evaluation. Initially, designers identify the evaluation tasks but users are encouraged to comment and to suggest appropriate alternatives and ask questions. The aim is to make the interaction as natural as possible. A strong emphasis is placed on the need to create a relaxed and informal atmosphere before starting the evaluation. During the evaluation, think around protocols are collected using the following procedure:
One or more users is recruited and care is taken to make sure that the users are typical of the population that will be using the final product.
Tasks are selected that will be representative of the kinds of things that the users would wish to do when working with the system. (If a very early prototype is being used in the test this can have an important function in helping to restrict the number of functions that are available to be used.)
As each user works with the system she verbalizes the problems that she experienced and the evaluator makes notes.
A debriefing session is held at the end of the session so that any misunderstandings that might have occurred can be cleared up and check for common understanding.
The designer then summarizes the notes and reports the problems back to the design team.
Variations to the technique may also be used, including post-tests surveys to check users' opinions and round table discussions when there is a wide variety of users in the user population.
Participative evaluation
Participative design and evaluation share the same philosophy and many of the same techniques, and in practice, they are closely interwoven. In fact, to think of them separately would be artificial; they are basically one and the same thing. Greenbaum's and Kyng's (1991) book entitled Design at Work contains a host of examples of informal evaluations in which such techniques are used. They claim that mock-ups facilitate evaluation because they are not threatening and are accessible to everyone, which encourages user participation. They enable users and designers to trace breakdowns in the interaction by recreating them, trying solutions, and discussing issues in users' own language. These authors suggest that cooperative prototyping can be encouraged by:
establishing focus groups, which include competent representatives from the whole user group range, who will work with the designers and collaborate in the mutual learning process;
designers are prepared to coordinate the groups and work on the development of prototypes with users;
providing prototypes that are sufficiently realistic and stable to enable users to take charge of the evaluation;
ensuring that communication is maintained;
using early prototypes to focus on exploring the coupling between the social and political issues in the workplace and the technical ones
highlighting the benefits of cooperative prototyping, particularly early in design, so that the activity is well resourced.
Cooperative prototyping provides a way of inter-meshing evaluation and design so closely that they are virtually a single ongoing process. There is, therefore, a clear testing stage with identifiable data.
Photo by Antonio Janeski on Unsplash