As Meaningful Use Stage 3 (MU3) moves from being optional for physicians and hospitals in 2017 to mandatory in 2018, you may find yourself wondering what changes you can expect in this phase regarding certification criterion for safety-enhanced design.
Overall, the requirements for MU3 certification and participation in the program are significant. EHR vendors will be challenged to deliver certified products according to the proposed timelines and navigate confusing requirements for safety-enhanced design.
What should I expect in MU3 regarding safety-enhanced design?
Many of the same steps that vendors had to complete for Stage 2 still apply in Stage 3. Vendors must still include evidence of user-centered design (UCD) and the results from a final summative usability test in their certification submission.
These steps are to:
- Complete a summative usability test on specific areas of your product according to NISTIR Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records
- Document all the results of your summative usability test according to NISTIR 7742 Customized Common Industry Format Template for EHR Usability Testing
- Include evidence of your UCD process; examples of method(s) that could be employed for UCD are included in ISO 9241-11, ISO 13407, ISO 16982, ISO/IEC 62366, ISO 9241-210 and NISTIR 7741.
If you need to review these steps, Macadamian’s Didier Thizy introduced them in here in a previous blog.
Updated list of certification criteria
Originally for MU3, 17 certification criteria (7 new to Stage 3) were required to have UCD processes applied to them to satisfy the certification criteria. However, based on comments and review regarding the rule, the 17 criteria have been reduced to 12.
The five dropped criteria were: “vital signs,” “eMAR,” “incorporate laboratory tests/results,” and both “decision support” criteria. Therefore, there is only a net increase of two criteria.
The criteria included in the 2015 Edition certification criterion are as follows (emphasis added for the new criteria):
- Section 170.315(a)(1) Computerized provider order entry—medications
- Section 170.315(a)(2) Computerized provider order entry—laboratory
- Section 170.315(a)(3) Computerized provider order entry—diagnostic imaging
- Section 170.315(a)(4) Drug-drug, drug-allergy interaction checks
- Section 170.315(a)(5) Demographics
- Section 170.315(a)(6) Problem list
- Section 170.315(a)(7) Medication list
- Section 170.315(a)(8) Medication allergy list
- Section 170.315(a)(9) Clinical decision support
- Section 170.315(a)(14) Implantable device list
- Section 170.315(b)(2) Clinical information reconciliation and incorporation
- Section 170.315(b)(3) Electronic prescribing
We believe that the additional certification criteria include capabilities that pose a risk for patient harm and, therefore, an opportunity for error prevention in the UCD process. In other words, the additional criteria are meaningful and worthwhile evaluating in the process, as they are aspects that could be harmful to patients if designed poorly.
Evaluating the usability of the new criteria, Section 170.315(a)(5) Demographics and Section 170.315(a)(6) Problem List, is very similar to evaluating the medication and medication allergy lists (Section 170.315(a)(7) and Section 170.315(a)(8), respectively).
Clinically appropriate usability test tasks can be constructed for these new criteria in the same fashion as the two mentioned such as recording a new problem and updating the patient’s preferred language.
Updated requirement regarding number of test participants
The other major change from Stage 2 to Stage 3 is the mandatory minimum of usability test participants. The final rule requires 10 participants per criterion, as a mandatory minimum for the purposes of testing and certification.
Previously, there was a recommended range of participants but no mandatory minimum. This dictated minimum will be an extra burden during recruitment efforts for vendors.
However, the mandatory minimum is better than a suggested range because it helps ensure the reliability of the test results. As we noticed in previous MU stages, vendors were completing summative tests with fewer than four test participants. A usability test with so few participants is not a summative usability test. A summative test requires a higher number of participants than a formative test because summative testing is a benchmark test.
As vendors collectively sigh in relief in not having to recruit as many participants (as the suggested previous range), they should be aware that according to industry standards and best practice recommendations, 10 participants is not the norm.
As per NISTIR 7804, 15-20 test participants are suggested per user group for which the application is being designed. This is also consistent with the language used in the 2011 FDA guidance on validation testing. Therefore, vendors should always keep in mind that as a general rule, the more participants involved, the more likely you are able to identify and remedy design flaws.
We believe that the updated number of participants strikes a good balance between any potential burden for EHR vendors regarding recruitment and identifying meaningful usability problem in the product.
With the recommended minimum, research suggests that 80% of usability problems are found whereas 95% of the problems are found with more than 20 participants.
Therefore, the final suggestion provided to vendors is that they are strongly encouraged to exceed the mandatory minimum in an effort to identify and resolve more problems.
Update requirement regarding user satisfaction measure
Another update from Stage 2 to Stage 3 is that the ONC will now also permit the submission of an alternative acceptable user satisfaction measure. Alternative approaches include the Single Ease-of-use Question, System Usability Scale, or Software Usability Measurement Inventory.
In essence, a vendor could choose to either meet the NIST 7742 based approach for user satisfaction or provide documentation of an alternative acceptable user satisfaction measure.
In general, we’re happy to see the continued submission of summative usability test results, as a part of MU3. These submissions promote transparency and can foster product design competition, spur innovation, and enhance patient safety.
If you have any questions regarding the released final rule for the third stage of the Meaningful Use incentive program and the corresponding certification requirements for EHR vendors, let’s connect!
Insights delivered to your inbox
Subscribe to get the latest insights in IoT, digital health, and connected products.