Keystroke-Level Model

Just how fast is using a unified text field and location automation field compared to the traditional approach of using multiple form fields? The best way to find out is to analyze the time it takes for users to complete each step in the process and compare each approach. To do this, we’ll use a hard science approach known in the field of human-computer interaction as the Keystroke-Level Model. KLM is a widely used method for predicting how long it takes users to do a task on a user interface. It uses estimated times for basic, routine user actions.

These predictions come from a study by Card, Moran & Newell (1980) involving extensive tests with users. All that is required for this kind of evaluation is:

• a specification of the system's functionality or at least that part of it which is to be examined,

• a task analysis, which is basically a list of all the proposed user tasks with a breakdown of each task into its components. Often these components are sequences of subtasks that require further decomposition.

Nowadays we have available several analytic evaluation methods. The differences between these methods revolve around task structure, user operations, and the combination of user and system operations. Some examples of these are:

• the task structure can vary from a simple task consisting of a single user command to more complex tasks composed of several commands,

• user operations can be evaluated with analytic methods using either extremely simple memory operations closely linked to physical actions at an interface (such as the recall of what is to be typed or selected) or more complex cognitive operations.

Two kinds of models exist single-layer models that can be thought of as having a flat representation and multi-layer models, which are more complex. One of the most widely known analytic methods is the keystroke level model. This is an example of a single-layer model and it deals with short tasks, usually just single commands, and has very simple user operations embedded in the task sequence. The purpose of the keystroke level model is to enable designers to calculate task performance times that can be achieved by experienced users, that is, it gives the designer an idea of the minimum performance time for specified commands.

KLM Operators

The keystroke level model asserts that the execution part of a task can be described in terms of four different physical-motor operators followed by one mental operator and one system response operator:

• K (keystroke or button press): it is the most frequent operator and means keys and not characters (so e.g. pressing SHIFT is a separate K operation). The time for this operator depends on the motor skills of the user and is determined by one-minute typing tests, where the total test time is divided by the total number of non-error keystrokes.

• P (pointing to a target on a display with a mouse): this time differs depending on the distance to the target and the size of the target, but is held constant. A mouse click is not contained and counts as a separate K operation.

• H (homing the hand(s) on the keyboard or other device): This includes movement between any two devices as well as the fine positioning of the hand.

• D (drawing (manually) nD straight-line segments with a total length of D(nD, lD) cm): where nD is the number of the line segments drawn and lD is the total length of the line segments. This operator is very specialized because it is restricted to the mouse and the drawing system has to constrain the cursor to a .56 cm grid.

• M (mentally preparing for executing physical actions): denotes the time a user needs for thinking or decision making. The number of Ms in a method depends on the knowledge and skill of the user. Heuristics are given to help decide where an M should be placed in a method. For example, when pointing with the mouse a button press is usually fully anticipated and no M is needed between both operators. The following table shows the heuristics for placing the M operator

• R (response time of the system): the response time depends on the system, the command, and the context of the command. It only used when the user actually has to wait for the system. For instance, when the user mentally prepares (M) for executing his next physical action only the non-overlapping part of the response time is needed for R because the user uses the response time for the M operation (e.g. R of 2 seconds – M of 1.35 seconds = R of .65 seconds). To make things clearer, Kieras suggests the naming waiting time (W) instead of response time (R) to avoid confusion. Sauro suggests taking a sample of the system response time.

Pros

The KLM was designed to be a quick and easy-to-use system design tool, which means that no deep knowledge about psychology is required for its usage. Also, task times can be predicted (given the limitations) without having to build a prototype, recruit and test users, which saves time and money.

Cons

The keystroke-level model has several restrictions:

• It measures only one aspect of performance: time, which means execution time and not the time to acquire or learn a task.

• It considers only expert users. Generally, users differ regarding their knowledge and experience of different systems and tasks, motor skills, and technical ability.

• It considers only routine unit tasks.

• The method has to be specified step by step.

• The execution of the method has to be error-free.

• The mental operator aggregates different mental operations and therefore cannot model a deeper representation of the user’s mental operations. If this is crucial, a GOMS model has to be used (e.g. model K2)

• Also, one should keep in mind when assessing a computer system that other aspects of performance (errors, learning, functionality, recall, concentration, fatigue, and acceptability), types of users (novice, casual), and non-routine tasks have to be considered as well.

Furthermore, tasks that take more than a few minutes take several hours to model and a source of errors is forgetting operations. This implies that the KLM is best suited for short tasks with few operators. In addition, the KLM can not make a perfect prediction and has a root-mean-square error of 21%.

Photo by Stephen Dawson on Unsplash

Discount Usability Evaluation

Discount usability evaluation was also developed by Nielsen in 1989. The idea behind this technique is to enable developers with few resources - in terms of time, money, or expertise - to benefit from usability testing during product design and development. In particular, the method was geared towards small companies that cannot afford full-scale usability testing like that used in usability engineering.

In Nielsen's words, "The discount usability engineering method consists of scenarios, simplified thinking aloud and heuristic evaluation and is intended to alleviate the current problem where usability work is seen as too expensive and difficult by many developers".

Discount usability engineering is thus really a hybrid of empirical usability testing and heuristic evaluation. The first part of the method uses usability testing and involves the construction of scenarios, which are tested using "think-aloud" protocols. Interestingly, often only one to three subjects take part in this testing phase. On the basis of the results obtained from this yet the scenario is changed and tested again. The second part of the method involves testing the scenario using the heuristic evaluation method described above.

The features that result in important discounts include:

• The scenarios are small and so can be changed easily - mockups are often used.

• The think-aloud method is done informally and does not require psychologists,

• Just a few key guidelines are used rather than the large number available - the nine guidelines listed above were found to be ample for the purpose,

• The whole cycle need only involve two or three testers since the number of additional usability problems found by more testers was not worth the extra effort.

An example of discount usability engineering

A Danish bank (Nielsen, 1989) wished to improve its bank account statements, so it planned to redesign them. At the beginning of the redesign a vocabulary study was performed on the original version of the statements, which identified 26 words, and 30 people were asked to rate these words according to how well they understood their meaning. Of the 26 words, 7 were difficult to understand, as the "think-aloud" method alter confirmed. The statements were then redesigned and several iterations of discount design-test-redesign were carried out using well-known usability heuristics such as "consistency". In this study, a total of 8 different versions of the account statements were tested. However, the whole study was reported to take only 90 person-hours, including the design of 7 different versions of 12 different kinds of bank statements.

Photo by NASA on Unsplash

Usage simulations

Usage simulations involve reviewing the system to find out usability problems. These reviews are usually done by experts, who simulate the behavior of less experienced users and try to anticipate the usability problems that they will encounter. For this reason, they may also be referred to as expert reviews or expert evaluations.

Ideally, these reviewers will be experts in HCI and will also have a broad experience of different kinds of systems, so they should be able to spot usability problems concerned with inconsistency, poor task structure, confusing screen design, and so on. The main inconsistency, poor task structure, confusing screen design, and so on. The main reason for having reviewers pretend to be fewer experienced users rather than employing typical users in the first place is to do with efficiency and prescriptive feedback. In terms of efficiency, a small number of reviewers can usually identify a whole range of potential problems for users during a single session. Real users would take much longer and require more facilities. Furthermore, experts are often forth-coming with prescriptive feedback about how the system can be improved and how usability problems can be put right. Often little prompting is needed to get reviewers to suggest possible solutions to the problems identified because they have experienced many systems, and the design team may benefit from this additional input. Generally, they provide detailed reports based on their use of the prototype or working system.

While reviews are relatively straightforward, it is necessary to consider the following:

• To ensure an impartial opinion the reviewers should not have been involved with previous versions of the system or prototype.

• The reviewers should have suitable experience, both of the application and of HCI. Media and creative design expertise may also be needed for some systems. Finding a small panel of reviewers with the necessary expertise may be difficult and compromises may have to be made.

• The role of the reviewers needs to be clearly defined to ensure that they adopt the required perspective when using the system. While it is relatively easy to assess both the very limited understanding of novice users and the extensive knowledge of very experienced users, intermediate categories of user experience are much more difficult to define and role play.

• The tasks undertaken, the system or prototype used and any accessory materials, such as manuals or tutorials, have to be common to all experts and representative of those intended for the eventual users.

During data collection and analysis the I'm is to obtain a common set of factors from the individual reports that address the most important problems. This aim may be achieved in three ways:

• Structured reporting reviewers have to report observations in a set way. For example, they are asked to specify the nature of the problems they encounter, their source, the importance for the user, and any possible remedies.

• Unstructured reporting reviews report their observations and categorization of common problem areas is then determined.

• Predefined categorization reviewers are given a list of problem categories and they report the occurrence of problems in these categories.

Reviews appear to be an attractive method in terms of their potential efficiency and prescriptive feedback. However, there are a few problems that need to be noted. These problems are bias, locating suitable reviewers, role-playing, and the behavior of real users.

Photo by Lukas Blazek on Unsplash