Basics of Usability and Human Factors Engineering Assessment Tools, as Relevant to Medical Device Design

Usability refers to the extent to which a product can be used by specified users to achieve specified goals in an effective and efficient manner, to the satisfaction of these users. In recent years this term has taken on great significance within industry, and terms such as usability engineering, user-centered design and universal design are now in common use. Many companies designing consumer products now have in-house usability labs. This includes some medical device companies. Much of the target of usability research has focused on human-computer interaction (HCI) and interface design, which is also relevant since there has been an increasing use of computers within the clinical environment. Practical components of a user-oriented design include a diversity of criteria:

• ease of use and learning,
• high user task performance,
• low user error rate,
• subjective user satisfaction, and
• user retention over time.

A number of usability methods that don't require a sophisticated usability lab have been described in the liturature, such as cognitive walkthrough (where evaluations/users systematically document and talk through the goals and steps involved in using the interface), and heuristic evaluation (where usability specialists determine whether each element of a user interface follows established usability principles). An example of rules of thumb for heuristic evaluation of user interfaces comes from Jacob Neison, considered one of the pioneers in the area:

• Visibility of system status
• Match between system and the real world
• User control and freedom
• Consistency and standards
• Error prevention
• Recognition rather than recall
• Flexibility and efficiency of use
• Aesthetic and minimalist design
• Help users recognize, diagnose, and recover from errors
• Help and documentation.

These rules are quite general, and as noted in a recent government-supported book on research-based usability guidelines, care must be taken to also use actual assessments of performance and iterative design procedures.

Rather than try to summarize this large field of usability testing tools (see also RERC-AMI's links page, under Human Factors/Usability Resources), two complementary perspectives are covered here that help set the stage for our approach, which has culminated in our Mobile Usability Lab (MU-Lab). The first is based in medical device safety, the second on the value of user diversity and detailed task-based performance assessment.

Perspective 1: Insights from FDA's Human Factors Engineering Program

This perspective comes from the FDA and its Center for Devices and Radiological Health (CDRH), which has the responsibility for regulating medical devices (including "durable medical equipment"). Product "safety" and "effacacy" are the key criteria for the FDA, and the CDRH has developed a strong interest in encouraging the use of human factors principles and processes in the design of medical products. This is not surprising given that medical error is said to be the eighth leading cause of death in the U.S., and that many medical products initially approved for use by health professionals in controlled settings have migrated to home use, often used by persons with functional limitations.

The FDA's Human Factors Program has produced documents and supported standards activities that emphasize a systems perspective to help companies minimize user risks and maximize error tolerance, including:

• Do it by Design: An Introduction to Human Factors in Medical Devices (by FDA's D. Sawyer, 1996)
• Medical Device Use-Safety: Incorporating Human Factors
  Engineering into Risk Management (by FDA's R. Kaye and J. Crowley, 2000)
• AAMI / ANSI HE74-2001 Standard: Human Factors Design Process for Medical Devices - a standard that includes support of the FDA

Consider the following quote, from the second of these documents (Kaye and Crowley):

" Important characteristics of user populations include:

• General health and mental state (stressed, relaxed, rested, tired, affected by medication or disease) when using the device,
• Physical size and strength,
• Sensory capabilities (vision, hearing, touch),
• Coordination (manual dexterity),
• Cognitive ability and memory,
• Knowledge about device operation and the associated medical condition,
• Previous experience with devices (particularly similar devices or user interfaces),
• Expectations about how a device will operate,
• Motivation, and
• Ability to adapt to adverse circumstances. .." (from Kaye and Crowley, 2000)

This directly follows a paragraph that emphasizes the importance of consideration of the abilities and limitations of the user (Section 3.2.2, Medical Device Users). The strong focus on risk management frequently mentions the importance of understanding such abillities and limitations. This document also emphasizes the importance of identifying "use scenarios," similar to what we try to do through our R1 (Needs Assessment) process that includes national surverys, focus groups, product surveys and other conceptual considerations of accessible and universal design principles (which culminates in R1.4, problem definition and refinement). In Section 5 of Kaye and Crowley (2000), human factors/usability approaches are divided into two types:

• analytic (heuristics which involve systematic decomposition and analysis of device use, for instance use scenarios and descriptions, descriptive functional/task analysis, heuristic analysis, expert review) and
• empirical (use studies involving evaluation of data from acutal or simulated use of the device by a representative sample of test participants, for example walk-through and full usability testing).

Our R2 (Usability Testing) project assumes that most of the initial analytic analysis has been done, and targets implementation of the empirically-based approach of product testing by actual participants, only with a special focus on including participants with a diversity of abilities and on testing within an ecologically valid environment.

Perspective 2: Integrating in Accessible/Universal Design Concepts with a Detailed Protocol for Task Analysis

While in part the purpose of R2 is to study products that are problematic or exemplary, it is also intended to be complementary with D2 (Design Activities). In most cases we target activities/tasks using devices where the performance goal or goals are clearly identifiable. Often each activity can be broken into a sequence of subtasks (e.g., stages related to positioning, mobility, reaching, manipulation, and communication). Each of these can be broken down further, and there are many such classification schemes (e.g., this is a focus of the R3 (Accessibility Metrics) project of the RERC-AMI). Here, however, is where our approach can be considered a natural extension of that suggested in the FDA and the AAMI/ANSI standard: it is clear that persons with disabilities will often employ unique and creative strategies to accomplish a use task. Thus for this RERC it becomes important to identify a user population with a diversity of abilities that include participants who represent subpopulations with known challenges, and then carefully and systematically document use strategies and barriers. Typically the results of R1 (Needs Assessment) can help guide us through the process of user selection. But additionally, we felt that there was a need to structure the process through a Protocol Manager that includes pre-activity participant interviews, user testing, and post-activity participant interviews.

The motivation is that we note that observations of performance are not sufficient for task analysis, except for very simple tasks. Sensors and observers simply cannot get into the mind of the user, some of whom may be quite frustrated. Normally the usability engineer must also employ methods that extract information from the subject/client, for instance related to their understanding of a product, the choices they made to perform a task, their opinions of a product, and perhaps their suggestions for improvment. Often this takes the form of both a structured questionnaire or interview, and an open-ended stage. We believe that such information is best collectioned both before the user testing session (e.g., to help refine the testing protocol task(s) to make sure they are appropriate) and immediately following performance of the activity. This is how MU-Lab is structured.