Recently the Office of the National Coordinator for Health Information Technology (ONC) unveiled new certification and meaningful use requirements for electronic health records (EHRs), including safety-enhanced design requirements. According to these requirements, EHR vendors must include evidence of user-centered design and user test results in their certification submission, a potentially significant amount of work.
Though providers are not required to follow the Stage 2 requirements until 2014, vendors should take steps now to position their products for success and comply with these new guidelines. Here’s a summary of what’s involved, with excerpts from the official documentation.
Where Do We Start?
The certification requires that you perform Summative Usability Testing on specific areas of the product, following a formal User Centered Design (UCD) process.
The essential document to read in detail is NISTIR 7742 Customized Common Industry Format Template for EHR Usability Testing. It outlines how the ONC recommends you document all of the results of your summative usability testing.
What Is Summative Usability Testing?
Summative usability testing is comprised of having representative users walk through a series of tasks and measuring the amount of time to complete the task, how many and what type of errors occur, and satisfaction with the task interaction.
The goal is to benchmark the current usability of an interface—as measured by things like task times, completion rates and satisfaction scores.
Summative testing is typically carried out by a user experience design professional who runs the tests one-on-one with representative users of the system, one at a time.
You will need to submit the information specified in the NISTIR 7742 template, for each and every one of the applicable eight certification criteria specified in this certification criterion and for which certification is sought.
The following information/sections in NISTIR 7742 are required for submission:
- Name and version of the product
- Date and location of the test
- Test environment
- Description of the intended users
- Total number of participants
- Description of participants: their experience and demographic characteristics
- Description of the user tasks that were tested
- List of the specific metrics captured during the testing for effectiveness, efficiency and satisfaction
- Data scoring
- Results of the test and data analysis
- Major test findings
- Identified area(s) of improvement(s)
According to the ONC, there are illustrative tables on pages 11 and 20 of the NIST 7742 document that may not need to be populated, depending on the tasks tested. All of the sections specified above must to be completed, including ‘‘major findings’’ and ‘‘areas for improvement.’’ EHR developers can perform many iterations of summative user testing. Thus, the submission that is ultimately provided for testing and certification may be the expression of a final iteration in which few areas for improvement would be identified.
What Are the 8 Areas Of Focus?
- § 170.314(a)(1) (CPOE);
- § 170.314(a)(2) (Drug- drug, drug-allergy interaction checks);
- § 170.314(a)(6) (Medication list);
- § 170.314(a)(7) (Medication allergy list);
- § 170.314(a)(8) (Clinical decision support);
- § 170.314(a)(16) (Electronic medication administration record);
- § 170.314(b)(3) (Electronic prescribing);
- § 170.314(b)(4) (Clinical information reconciliation).
The test procedures to cover these areas is explained in detail here.
How are the Results Captured?
Use the Customized Common Industry Format Template for EHR Usability Testing (NISTIR 7742). The form and format for how the data is presented for testing does not necessarily need to be according to NISTIR 7742 (i.e., an EHR technology developer could capture information specified in NISTIR 7742 without having to use the template).
What’s Out of Scope?
NISTIR 7804 ‘‘Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records” is not included in the 2014 Edition EHR certification criteria, but the ONC may consider it for future editions of certification criteria.
Which UCD Methodology Should be Used?
The methods for how an EHR developer could employ UCD are well defined in documents and requirements such as ISO 9241–11, ISO 13407, ISO 16982, and NISTIR 7741. ISO/IEC 62366 and ISO 9241–210 are also acceptable alternatives.
If you are working with a User-Centered Design consultant, ask which methodology they will be using. The ONC strongly advises EHR developers select an industry standard process because compliance with this certification criterion requires submission of the name, description, and citation (URL and/or publication citation) of the process that was selected.
This certification criterion is a first step toward formal usability testing becoming part of the culture of EHR technology development. At a minimum, only lab- based summative testing is necessary to be performed in order to demonstrate compliance with this certification criterion. However, the ONC notes that nothing precludes field- testing and formative testing from also being performed and we encourage EHR developers to do so.
For more information on how Macadamian can help, contact us at firstname.lastname@example.org.