Test Strategies for HIPAA Compliance

Macadamian Technologies | January 9, 2017 | 9 Min Read

If you search for information on testing HIPAA compliant applications, you’ll find a number of overviews on HIPAA, but very little information on how to approach testing software to ensure that it will be HIPAA compliant. Macadamian’s Test Strategy includes HIPAA compliance verification as an important part of its initial sanity testing and full-feature testing. We also ensure that HIPAA compliance defects are clearly indicated in our bug tracking system so their fixes are given high priority.

Note that this article only covers major software testing areas related to ensuring HIPAA compliance, and does not cover other areas such as required physical safeguards like not deploying the software on workstations with publicly viewed screens. Also be aware that testing strategies should depend on the software requirements of the application. Not all testing strategies described in this article are applicable to all applications.

TIP: Defect Tracking

A clear naming convention helps any tracking system. Consider reporting all defects found that relate to HIPAA compliance with the keyword “HIPAA” appended to their titles in your defect tracking system. Any that correspond to high-risk areas (in the Roles Matrix) should be automatically assigned a Critical severity. This ensures that these defects are immediately visible to the Project Leader and are fixed with high priority. Processes like this help reveal issues quickly and allow for easier reporting.

Conduct Initial Sanity Testing

Perform HIPAA-related sanity testing as early as possible in the development cycle to quickly discover any major defects in the application’s HIPAA compliance.

Sanity testing should cover high-level testing in the following areas:

  1. For each high-risk role/component/operation relationship (as determined in the Roles Matrix), verify the following:
    • A user of the specified role is able to authenticate successfully and is granted view, add, modify, delete, or no access to the specified application component operation.
    • Each action (user authentication, viewing, modifying, etc.) performed above is recorded in detail in the audit trail.
  2. Verify encryption in the following areas:
    • EPHI (electronic protected health information) in the database.
    • Audit trail entries.

Develop a Roles Matrix

Assuming the application makes use of RBA – role-based access (or a similar strategy), the first step in preparing for HIPAA compliance testing is identifying all the roles present in the system and their expected access levels to all components of the application. This identification is done in communication with the customer who then also approves the risk level associated with each role/ component/operation relationship.

The risk level is based on information disclosure, the frequency of use, the chance of error, and impact to the customer if an error occurs in a given component.

The table below is an example of what a roles matrix might look like. In each cell:

“X” indicates that the role has the ability to perform the specified operation (View, Add, Modify, Delete) in the application component. The color indicates the privacy or security risk level if there were a defect in this role/ component/operation relationship, where:


As you proceed to sanity testing, this chart helps you identify the risk level associated with each relationship and prioritize testing to ensure problems in high-risk areas are found and fixed first.

Create Detailed Test Cases

To aid in HIPAA compliance and provide an exact traceable record of what is tested, our test cases are written with very explicit details. The individual steps to be executed are broken down to a low-level action (with sample input if applicable) and a specific expected result. This way, test cases can be failed at specific steps, making it easier to write clear defect reports.

EXAMPLE: Test Case Details

Test #1234: Nurse Edit Patient Charting Care Plan

Here is how a test case for a nurse logging in and viewing a patient’s care plan might look.

StepDescriptionExpected Results
1 Start the application.Login screen appears.
2 Login as nurse.Nurse view of application opens.
3 Click Patients link.A list of only the nurses patients is displayed.
4 Click on the patient in the list.The selected patient’s Patient Charting record is displayed.
5 Click the Care Plan link.The patient’s care plan is displayed in the correct layout according to the wireframe.
6Press Edit button.The Edit Patient Care Plan popup opens with proper layout.
7 Clear the information in the mandatory fields. Press Save.A message is displayed saying mandatory data is missing or invalid. Asterisks and examples of valid data are beside the offending fields.
8Enter data that is too long for one of the fields.Only the maximum length of data is displayed in the field.
9Enter valid different data into the fields. Press Save.The Edit Patient Care Plan popup closes and the patient’s new care plan is displayed
10 Click Back to Patient List link.The patient is still displayed properly in the list.
11 Click Logout link.Nurse is logged out of the application

Five Main Areas for HIPAA Compliance Testing

For simplicity, we describe HIPAA compliancy testing in five main areas (there is some overlap between them):

  1. User authentication
  2. Information disclosure
  3. Audit trail
  4. Data transfers
  5. Information on correct data use

User Authentication

User authentication to the application is typically one of the following:

  • Ownership-based, for example: ID cards
  • Knowledge-based, for example: user id/password
  • Biometric-based, for example: fingerprint

User authentication testing needs to cover more than just the successful login path for each role as described in the sanity testing section. In addition to this, test the negative path and more intricate test cases such as the following (depending on the method of authentication):

  • Login failure for each of the following reasons, where applicable:
    • empty user id
    • empty password
    • invalid user id (including case-sensitivity if applicable)
    • invalid password (including case-sensitivity if applicable)
    • expired account
    • blocked account
  • Locked-out account (after repeatedly failing login x
    number of times)
  • Login success after password change
  • Characteristics of password change itself:
    • cannot reuse previous x passwords
    • forced to change after certain time period
  • Login idle timeout (user session expires after being idle for a period of time) on both workstations and mobile devices
  • Login credentials not stored in application memory (if required for security)

For applications that also execute on a mobile device with reduced code, conduct separate testing for user authentication on the actual device.

Again, make use of the Roles Matrix to ensure users are granted the correct access level to all application components.

Information Disclosure

Information disclosure is typically limited with two main strategies (but both may not necessarily be tested if they are not used in the application under test):

Role-based access (RBA)

grouping of users into logical classes which have certain levels of access to specific application components (as described in our Roles Matrix).

Patient allocation (PA)

supervisor assigns patients to a health care provider for a specified period of time (for example a shift in a hospital on a specific floor).

As discussed in the Sanity Testing and User Authentication sections, your testing strategy needs to ensure that information disclosure is limited by RBA (or similar strategy used in the application under test).

When applicable, design test cases to ensure that the PA limitations are also respected, such that application users are only able to view/add/modify/delete patient information that they are supposed to currently have access to, and not able to view/ add/modify/delete information for patients that have not been currently allocated to them.

Also under the topic of limiting information disclosure, your test strategy needs to ensure that if the application is uninstalled that all EPHI (electronic-protected health information) is completely removed/deleted from the mobile device or system. This includes checking all the locations in which the application may have written data, such as install directory (default or custom), Windows Registry, and/or user folder.

As discussed in the section on Audit Trails, also verify that all accesses and attempts to access EPHI are identified and reported.

Audit Trails

After sanity testing, you should also conduct a more thorough analysis of the audit trail. Depending on the format of the audit trail (for example, it could be log files, database entries, etc.) you might use comparison tools to verify that the entries produced match previously constructed benchmarks of expected entries.

Perform the following verification:

  • That all expected audit trail entries exist, for every operation performed on the EPHI (electronic protected health information).
    • Make use of the Roles Matrix to ensure no operations (performed by each role) are missed when writing detailed test cases that specify exactly which entries are expected.
    • Also ensure that entries are created for actions performed on all types of devices, including mobile
  • That each audit trail entry contains the following information:
    • Date and timestamp of the action
    • The user id/name of the user performing the action
    • The access level of the user
    • The patient record id (if applicable) on which the action was performed
    • The action performed or attempted
    • The specific application component from which it was performed (e.g., billing vs. patient charting)
    • The location or system id (if applicable) from which the action occurred (e.g., the hospital or clinic’s NPI (National Provider Identifier)
  • Entries conform to the software’s clarity requirements, and that the audit trail can be easily followed, if necessary for a future investigation.
  • Entries cannot be removed from the audit trail.
  • Audit trail can only be viewed by certain user accounts.
  • All attempts to breach security are recorded in the audit trail in such a way that they stand out for easy identification.
  • Audit trail is encrypted.

Data Transfers

In addition to the encryption verification performed on the database and audit trail during the sanity testing stage, also use a network analyzer tool (such as Wireshark) to ensure EPHI (electronic protected health information) is encrypted during:

  • Data access between all workstations and mobile devices on which the application is installed and the database.
  • Data transfers to an external location.
  • The movement of data to an offline storage location.

If the application under test involves healthcare business practices like the electronic transfer of claims transactions, remittance advices, enrollment and eligibility transactions, then the test strategy should ensure that the proper EDI (electronic data interchange) X12 formats are used for these processes.

Information on Correct Data Use

Also, verify that the application provides an explanation of correct data use prior to access. Depending on the application, this could be in the form of verifying there is a help page available for each specific operation that involves EPHI (electronic protected health information), or testing of a training version of the application that allows users to see how the application works before granting access to real EPHI.

Download the Guide to Developing Engaging Digital Health Software

This 90+ page guide focuses on the intersection of your user, technology choices, and the regulatory environment you face that will play a critical role in the success of your application.

Suggested Stories

Meaningful Use Stage 3 Safety-Enhanced Design Updates

Overall, the requirements for MU3 certification and participation in the program are significant. EHR vendors will be challenged to deliver certified products according to the proposed timelines and navigate confusing requirements for safety-enhanced design.

Read More

5 Best Practices for Designing in Medical Device Cyber Security Controls

There is no silver bullet for achieving security and it definitely cannot be considered an afterthought once a design has been completed, or considered at the development phase but forgotten about after deployment. Security needs to be a constant focus from product conception, design, development, deployment, maintenance, and support.

Read More

Simplifying Ontario’s New Workplace COVID Screening Requirements

Many businesses in Ontario that have begun reopening will now be required to have all employees and essential visitors complete a symptom screening survey. While this will help make employees and customers safer, it creates a few challenges for businesses to implement.

Read More
Macadamian has been acquired by Emids 🎉
This is default text for notification bar