In this article, we will discuss the techniques that are used in usability engineering. Usability engineering was defined by Tyldesley as a "process whereby the usability of a product is specified quantitatively and in advance. Then as the product is built it can be demonstrated that it does or does not reach the required levels of usability" (1988). Usability engineering has been well received by a number of companies because its semi-scientific and engineering nature provides a systematic procedure for testing the usability of a product during development.
The underlying aim of usability engineering is to engineer for improvement. The design and evaluation components of usability engineering are very closely interrelated in cycles of design-evaluate-redesign. The engineering nature of the process is reflected in this iterative development and in the fact that many characteristics of a product are specified quantitatively and in advance. The process that follows results in the building of a product that demonstrates the planned-for characteristics.
The scientific nature of the process is reflected in the evaluation setting. Testing usually takes place in a purpose-built laboratory and follows a procedure that is broadly scientific in nature. However, the nature of the testing differs from the scientific experimentation in that it is not usually possible to control variables not directly of interest. Furthermore, if you were to control all the variables you would end up changing the nature of the system being examined. Interestingly, as you will see later, compared with some forms of evaluation such as contextual evaluation and ethnography, there is a high level of evaluator control in usability engineering. In fact, it is sometimes criticized for being too scientific in nature and for presenting a distorted impression of the product's real usability in the workplace. Good et al. (1986) describe the process of usability engineering as consisting of the following steps:
Defining usability goals through metrics,
Setting planned levels of usability that need to be achieved,
Analyzing the impact of possible design solutions,
Incorporating user-derived feedback in product design,
Iterating through the "design-evaluate-design" loop until the planned levels are achieved.
Metrics and the usability specification
In this previous article, you read about the use of metrics in upgrading products. These metrics are also used to describe various levels of acceptability of key design features. More often than not they are based on measures such as time to complete a particular task, a number of errors, and attitude ratings given by users in a questionnaire.
The main emphasis in usability testing is on monitoring users' performance on carefully constructed standard tests known as benchmark tasks, which they perform in a usability laboratory or similar setting. Video cameras are positioned so that simultaneous film of the screen and the user's hands on the keyboard (or other input devices) can be obtained. In addition, keystrokes are usually logged. Typically, these data are used in conjunction to obtain information about such things as the time that it took a user to perform particular tasks and subtasks, error rates and types of errors, usage of the manual, and so on. In order to standardize the experimental method as much as possible, the basic rules of usability testing are to give a specified set of users specified tasks to complete in a controlled environment - in other words, to control as many of the variables as possible.
Users' opinions are also important and are elicited through questionnaires and interviews. Data from the questionnaires s used to produce attitude metrics, which are recorded on the usability specification.
When the design team discusses the usability specification it may be faced with having to make trade-offs which involve accepting lower standards for some attributes in order to achieve others. Priorities, therefore, must be established. One way of doing this is through a process called impact analysis. This technique involves listing the usability attributes and, alongside them, listing the proposed design decisions. Next, the percentage impact of each design solution is estimated for each of the attributes and sets of attributes as a whole. Although this process is reported to be fairly cumbersome many insights can be gained from doing it. Usability engineering has brought structure to HCI design. It has also been important in getting all contributors to the design process to work towards achieving explicit and agreed goals. In this respect, the usability specification is an important document, which represents a consensus common goals. Consequently, it provides a communication medium through which employees with technical expertise, such as programmers, and those with less technical knowledge such as marketing personnel and users, can communicate. As Nielsen has pointed out, usability applies to the total life of a product, which may stretch through as many as seven or more upgrades. Usability engineering is reported to produce a measurable improvement in the usability of about 30%. However, it is not totally free of problems. The scientific nature of the testing means that it is a fairly unfriendly environment for users and also quite different from the real world. Informal field observation may also be conducted to compensate for this because it provides a way of assessing how users use technology in the workplace and how this pattern of usage relates to the laboratory tests. Interpretive methods, such as contextual inquiry, provide another solution to this problem.