R3: Accessibility Measurement

Highlighted MED-AUDIT Study: Rochelle Mendonca's MS Thesis Project

Assessing the Usability of MED-AUDIT (Medical Equipment Device - Accessibility and Universal Design Information Tool)

Overview:

This study was designed to assess the usability of the two prototype versions of the MED-AUDIT (The Black Box Version and the Expert User Version). It included 38 novice participants from Occupational Therapy (N=28) and Engineering (N=10) as well as 4 experts in the area of disability and universal design. This study compared the time required to score the two versions. It also explored the relationship between the amount of universal design knowledge that participants have and the ease of the scoring the two versions. Lastly it looked at the educational potential of the MED-AUDIT by assessing if scoring the instrument bought a change in their perception of accessibility.

Objectives:

The main objective of this study relates to the R3 project. The goal of the R3 project is the development of a tool which will provide a protocol to rate the design of a medical device in order to determine its level of accessibility. This tool is called the MED-AUDIT (Medical Equipment Device-Accessibility and Universal Design Information Tool) and is being designed to assess the accessibility of various kinds of medical devices. This study was designed to assess the usability of the two prototype versions of the MED-AUDIT (The Novice User Version and the Expert User Version). It provided information on the various usability aspects of the two versions including the time required to score the tool, the knowledge requirements which increase or decrease usability as well as the educational properties of the MED-AUDIT.

Specific Aims:

• Identify the best method to collect data on the accessibility of medical devices using the MED-AUDIT.
• Compare the two versions of the MED-AUDIT using data collected on the accessibility of two different models of a blood pressure monitor (an automatic monitor and a manual monitor).
• Compare use durations for the two versions of the MED-AUDIT using time data collected using a specialized module included in the software.
• Determine the relationship between the ease of using the two MED-AUDIT versions and participants’ knowledge about medical devices and universal design.
• Obtain accessibility scores for thirteen different disabilities from experts for both models of the blood pressure monitor.
• Assess differences between accessibility scores obtained from novices before and after they score the MED-AUDIT.
• Compare accessibility scores obtained from experts with those obtained from novices before and after scoring the MED-AUDIT to assess the difference between the two.
• Obtain preliminary data to assess inter-rater reliability, content validity, face validity, administrative validity and internal validity (However, this data was not analyzed as part of this study)

Subject Recruitment:

This study included three subject groups:

• Occupational Therapy participants were recruited from the undergraduate and graduate program at the University of Wisconsin Milwaukee.
• Engineering participants were recruited from the rehabilitation engineering labs from within the Marquette University’s Biomedical Engineering Program.
• Experts were recruited from the RERC-AMI research team.

The Engineering and Occupational Therapy participants scored two models of a blood pressure monitor on one randomly selected version of the MED-AUDIT. The expert participants were asked to assign accessibility scores to the two models of the blood pressure monitor for thirteen different disabilities.

Evaluation:

• The Novice User Version and the Expert User Version were compared on various time measures to determine which was more time efficient.
  • The amount of time required per question for the two versions was compared.
  • The total amount of time required to score the complete assessment was compared for the two versions.
• The level of universal design knowledge participants had was compared with the ease of using the two versions.
• The scores obtained from novice participants for the accessibility of the two models of the blood pressure monitor was compared before and after they scored the MED-AUDIT.
• The scores obtained from novice participants were compared with those obtained from experts for the thirteen different disabilities.

Results:

• The efficiency per question was greater for the Novice User Version compared to the Expert User Version, that is time taken to score each question was faster for the Novice User Version.
• The total efficiency was greater for the Novice User Version compared to the expert User Version, that is the total time taken to score the complete assessment was lesser for the Novice User Version as shown in the Figure 1 below.
• No relationship was found between universal design knowledge, skills or experience and the usability of the two versions of the MED-AUDIT.
• There was a significant difference between novice accessibility scores for the thirteen disabilities before and after they scored the MED-AUDIT.
• Scoring the MED-AUDIT did bring about a change in participants’ perception of accessibility by showing that participants’ scores more closely approximated experts scores after they score the MED-AUDIT.

Future Directions:

The results from the study are being used to develop the two versions of the MED-AUDIT. The data from this study is being analyzed to obtain preliminary reliability and validity information. Further studies are now being developed to assess the reliability and validity of the instrument.

Ackowledgements:

Advisor: Dr. Roger Smith
MED-AUDIT R3 Team: Dr. Jack Winters, Dr. Kris Barnekow, Melinda Winter, Todd Schwanke, Melissa Lemke, Eli Gratz, and Helen Eng.

This thesis was defended in December 2005.