NIST, ITL, EHR Observation - Part 3 - Supporting Statement

OMB Control No. – 0693-0043 – NIST Generic Clearance for Usability Data Collections

Project title: Study of Electronic Health Record Usability.

PART 3 – Direct observations of EHR Users

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

The Information Technology Laboratory (ITL) of the National Institute of Standards and Technology (NIST), will be working with participants consisting of clinical staff (e.g., physicians, nurses and radiology technicians) and non-clinical staff (e.g., hospital unit clerks) at study sites who interact directly with electronic health record (EHR) systems as part of their professional duties at the research site. We will select research sites based on the EHR system that they use, their geographic location, and their availability to participate in the research study. Specifically, we aim to recruit a sample of research sites that use a variety of EHR vendors and at locations within close proximity to the research team’s locations. Participants will be recruited and reminded about participation opportunities via standardized mass communication: e-mail distribution lists and recruitment posters. Copies of the e-mail invitation and recruitment poster have been uploaded into ROCIS. Participants will not be contacted by telephone or by mail to their places of residence. Participation in this study will be open to all individuals, regardless of race, ethnicity, gender, or other personal characteristics. The aim of the recruitment strategy is inclusiveness and no EHR user will be denied the opportunity to participate as long as enrollment is open. Once OMB approval is obtained, the enrollment period will run through May 27, 2012. This group is appropriate to survey because they will provide us with the necessary information to understand how individuals across a variety of professions and institutions use EHR systems.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The survey methodology was developed by the contractors and subcontractors in consultation with the team from NIST. Key personnel from the contracting organization, Wiklund Research & Design (WR&D), developed the first draft of the survey. WR&D sent the survey via email to the subcontracting organizations (Vanderbilt University, Design Science and University of Wisconsin) for review and comments. All subcontractors were given approximately two weeks to submit comments to WR&D. WR&D incorporated the subcontractors’ suggestions for improvement and provided NIST with a draft for feedback. The final draft of the survey reflects the feedback that WR&D received from NIST. Notably, all of the suggestions for improvement were incorporated into the document.

While WR&D did not conduct formal pre-testing of the survey questions and methodology, several of the subcontractors are clinicians who interact directly with EHR systems and could assess the appropriateness of the survey as qualified participants. Moreover, the contractors and subcontractors developed the survey based on materials from previous usability studies that proved to be an effective means to obtain information about users’ interactions with health information technology (HIT).

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

Participants who have used EHR systems will be recruited to participate in the collection of information via email and recruitment posters. The research team (Wiklund Research & Design and members of the subcontracting organizations) will distribute self-administered surveys to collect demographic information about participants they observe. The research team will conduct observations of consenting clinical and non-clinical workers interacting with EHRs and performing related tasks and will take real-time notes on these observations. These observations will take place in varying clinical environments within the research sites. The researchers will capture some relevant EHR interactions via fixed and handheld cameras. The researchers will record verbal and non-verbal communication related to EHR systems and participants’ interactions with the EHRs. These observations will help the prime- and sub-contractors to understand how end-users interact with EHR systems to accomplish clinical and non-clinical tasks. The survey will consist of questions intended to understand EHR system users’ background information (see filename: Survey instrument_Observations). The survey will have a maximum completion time of 10 minutes. NO identifying information about EHR users or patients, or patients’ protected health information (PHI) will be captured.

Data Management: Survey responses will be collected and kept in a secure electronic database. All survey data will undergo a cleaning process to ensure both respondent and affiliated institution anonymity and confidentiality. For example, the web survey will have questions that permit participant narrative responses. These answers will be reviewed and edited by authorized research project staff to remove any participant or patient names or other identifying text (e.g., patient’s PHI).

NIST will not have access to any of the raw data. Instead, the prime- and sub-contractors’ will analyze the raw numeric and narrative data to create Summary Reports that are completely devoid of identifying information or characteristics. Only the Summary Reports will be provided to anyone outside the research team. Therefore, NIST will never possess any information that is traceable to individual participants, specific commercial vendors, or specific healthcare entities.

The research team (Wiklund Research & Design and members of the subcontracting organizations) will only survey those participants who are observed. Accordingly, we expect a survey response rate of 100%. We will also attempt to improve the response rate by sending reminder emails and asking the department managers within the research sites to verbally remind their staff about the research opportunity.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The research team (Wiklund Research & Design and members of the subcontracting organizations) will collate and integrate the demographic data collected from the survey instrument with the data from observing participants interactions with the EHRs. We will analyze the results by looking for trends in the data overall and across different clinical roles (e.g., physician, nurse, resident, administrative secretary) and departments/facility types (e.g., intensive care unit, clinic, emergency room). Based on the observed data trends, we will develop a narrative description of user characteristics, use environments, EHR-related tasks, functions, goals, and best practices. We will measure associations between variables (e.g., between amount of EHR use and satisfaction with EHR). The findings yielded from this survey will help inform continuing research, guidelines, and policy. The results will generalize to the population to the extent a representative sample can be obtained. In cases where the data cannot be generalized to the entire EHR user population, they may be generalized to specific EHR user groups (e.g., most common tasks performed by EHR users in intensive care units).

The research team will analyze the raw numeric and narrative data to create summary reports that are completely devoid of identifying information or characteristics. The research team will provide only the summary reports to NIST. The research team and NIST will use the summary reports to develop a final report on the usability of EHRs that will be publicly available.

3