Supporting Statement Part B _Informed Consent

SUPPORTING STATEMENT

Part B

Improving Hospital Informed Consent with Training on Effective Tools and Strategies Training modules

October 6, 2014

Agency for Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 6

4. Tests of Procedures 6

5. Statistical Consultants 7

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

Four hospitals will be recruited to test two training modules to improve the process of informed consent for treatment in U.S. hospitals. The total universe of U.S. hospitals comprises 5,723 hospitals, according to the American Hospital Association. Sites will be selected to maximize diversity of hospital characteristics using a targeted, purposive recruitment approach. The purposive recruitment strategy will also allow us to attract hospitals whose leadership is already open to improving their informed consent policies and processes. These hospitals represent the training modules’ intended audience, as hospitals resistant to changing their informed consent processes are unlikely to use the training modules. The recruitment strategy will be conducted as follows:

Step 1: Inform hospitals of the opportunity to participate. Our first strategy to achieve this goal will be to post a link about the project on the Joint Commission’s website. The Joint Commission accredits approximately 77% of U.S. hospitals, and 90% of all accredited hospitals. The Joint Commission’s website is accessed by hundreds of hospital representatives on a weekly basis. The Joint Commission maintains a listserv of hospitals that also will be used to post a request for participation. The website link and listserv post will include a brief description of the project, the benefits of hospital participation (e.g., continuing education, opportunity to improved informed consent processes), and the types of hospitals we seek. Contact information will be provided for interested hospitals to notify The Joint Commission of their potential interest. If this strategy does not yield a sufficiently large and diverse sample of hospitals after two months, we will conduct active outreach through our professional contacts and through members of the Expert and Stakeholder Panel listed in Supporting Statement A.

Step 2: Obtain expressions of interest from hospital representatives that include willingness to meet the participation requirements described in Supporting Statement A (e.g., cooperation with data collection efforts).

Step 3: Exclude hospitals that do not demonstrate adequate readiness that would affect the likelihood they could train leaders and health care professionals in four units and implement changes to the informed consent process (e.g., major change happening concurrently like a change in electronic health record vendor, inability to implement changes within project time period, limited or no leadership buy-in at the hospital level or from leaders of four units within the hospital).

Step 4: Select from the remaining hospitals to ensure diversity on key characteristics. Selection will be made to ensure diversity on key characteristics, including: hospital type (e.g., academic health center, safety-net, community, integrated delivery system, military hospital), size, populations served (e.g., racial/ethnic minorities, Medicaid beneficiaries) and geographic location. Each hospital will be asked to designate a staff member as a liaison to this project. We suspect an individual involved in the hospital’s patient safety or quality improvement efforts might be best suited for hospital liaison; however, we will ultimately allow hospitals to decide who would best champion and facilitate changes in the hospital’s informed consent processes.

We make no claim that the results from this study will be generalizable. Rather, our small sample of information-rich cases will be illustrative of the kinds of barriers, facilitators and results that hospitals may experience in implementing the training modules, and will generate insights about needed training modules improvements.

2. Information Collection Procedures

Sample Size

Number of hospitals

Budget constraints will limit the number of pilot test sites to four hospitals. While it is always preferable to have a larger number of test sites, we anticipate that four sites will provide us with sufficient information to illustrate the types of barriers, facilitators and results that may be expected with training modules implementation, and to identify needed revisions to the informed consent training modules’ content and format.

Sample sizes within each hospital

Hospital units

Within selected hospitals, hospital liaisons will work with hospital leadership and unit leadership to select four units for implementation, including at least one surgical unit.

Hospital staff

Hospital leaders and health care professionals within the four selected units will participate in the pilot test by completing the training modules training for leaders or health care professionals. All health care professionals working in the selected units who are involved in the informed consent process (e.g., doctors, nurse practitioners, nurses, and interpreters) will be eligible to take the training. The training will be delivered through a Learning Management System (LMS). The number of persons trained is likely to vary considerably across hospitals and hospital units. Sample sizes provided below are rough estimates.

• Pre-/Post-Training Quiz: Both health care professionals and hospital leaders will be asked to complete a pre/post-training quiz right before taking the training and immediately after completing the training. We expect to administer this quiz to a total of approximately 640 health care professionals and 32 leaders across all sites and units, with a response rate of approximately 80%, for a final sample size of approximately 512 health care professionals and 26 leaders.

• Health Care Professionals Survey: Health care professionals will also be asked to complete a survey about their knowledge and behaviors before and after the training. We expect to administer this survey to a total of approximately 640 health care professionals, with a response rate of approximately 80%, for a final sample size of approximately 512 health care professionals.

• Rapid Feedback Patient Survey: Each participating hospital will also be asked to conduct a patient survey with a sample of 100 patients (50 pre-implementation of informed consent process improvements, 50 post-implementation) in at least one unit where the training modules were tested. We will recommend a census sample or other systematic sampling approach (e.g., every other patient until they attain the desired sample size). We expect this survey to be administered to a total of 400 patients across all four hospitals (100/hospital) with a response rate of approximately 80%, for a final sample size of approximately 320 patients.

Proposed analyses and statistical power calculations

As described in Summary Statement Part A, some of the data collected will be analyzed using qualitative methods. Quantitative analyses will complement the qualitative analyses. They will include univariate statistics (e.g., average trainee knowledge score) and, where appropriate, statistical tests to assess the differences between pre- and post-training or implementation of informed consent processes.

Data will be pooled across hospitals to perform statistical tests. Since outcome variables will be either continuous or on a Likert scale, we will use t-tests to assess differences in scores before and after the intervention. Paired t-tests will be used to test differences in knowledge scores before and after training and in the survey results pre-training and the survey 2-3 months after training. Unpaired t-tests will also be used to assess differences in patient rapid feedback scores, since different patients will be surveyed pre-implementation and then post-implementation (at least 2 months).

Exhibit 1, below, summarizes for each data source the type of variable used, statistical tests to be conducted, sample sizes, and minimum size of differences that can be detected at an 80% statistical power level given the sample size and explicit assumptions about the standard deviation (SD) and baseline data. All statistical tests will be computed in SAS or Stata.

Exhibit 1. Informed Consent Sample Size, Statistical Tests and Power Calculations

Instrument	Variables used	Statistical tests	Anticipated Sample size (pooled across hospitals)	Minimum detectable effect size at 80% statistical power level and 5% alpha
Health care professionals Pre-/Post-Training Quiz (Attachment D)	Pre- and post-test summary scores-percent correct (range: 0-100%)	Paired t-test to compare pre- and post-test scores.	512 pre 512 post	1.3 point change assuming SD=10 percentage points
Leaders Pre-/Post-Training Quiz (Attachment E)	Pre- and post-test summary scores-percent correct (range: 0-100%)	Paired t-test to compare pre- and post-test scores	26 pre 26 post	5.5 point change assuming SD=10 percentage points
Health Care Professionals Survey (Attachment G)	Average scores (0-5) for several domains: a) clinicians’ assessment of respect for patients’ rights in their unit; b) clinician self-report of attitudes about patient rights; c) clinician report on colleagues’ strategies to facilitate patient understanding; d) clinician self-report on strategies to facilitate patient understanding Average scores (range: 0-10): a) clinician rating of overall informed consent effectiveness in their unit; b) self-rating of clinician’s informed consent effectiveness c) Self-assessment of learning in various domains	Paired t-test	512 pre 512 post	0.13 point change on all variables assuming SD=1
Rapid Feedback Patient Survey (Attachment I)	Binary variables (yes/no) and ordinal variables recoded to binary form (meets/does not meet minimum standard) Average scores (0-10): a) overall satisfaction with consent process; b) Ease of understanding the consent form; c) overall rating of surgeon	Binomial tests (for binary variables) Unpaired t-tests (for average scores)	160 pre 160 post	Binomial tests: Pre-post change of 16% assuming baseline value=50% t-tests: 0.65 point change on average score variables assuming SD=2

Inclusion and selection criteria

Hospitals will be chosen based on their willingness to participate and readiness, as described above. Since we expect that hospital characteristics may impact implementation, selection will be made to ensure diversity on key characteristics, including: hospital type, population served and geographic location.

Within each hospital, hospital liaisons in collaboration with hospital leadership and unit leaders will select at least one surgical unit and three other units to implement the training modules.

Unit leaders and all health care professionals within the participating units will be eligible to take the training and complete the Pre-/Post-Training Quiz and Health Care Professionals Survey.

Additionally, within the units where the Rapid Feedback Patient Survey will be fielded, with patients or patient proxies (i.e., individuals consenting for individuals not competent to consent or legal guardians of minors) until they attain the desired sample size. Patients will be excluded if they have limited English or limited Spanish proficiency (since the survey is only available in English and Spanish) or if they choose not to participate.

3. Methods to Maximize Response Rates

Completion of the Pre-/Post Training Quiz will be a requirement to confer continuing education (CE) credits and hospital leaders will be asked to encourage completion of the training and pre-/post-training quiz, which should help maximize the response rate.

Although clinician surveys (conducted online or using other methods) often suffer from low response rates, AHRQ expects a better than average return rate for the Health Care Professionals Survey because the distribution will be targeted, using the email addresses of participants who have completed the training on the LMS. Hospital leadership and hospital liaisons will play important roles in encouraging clinicians to complete the survey.

The Rapid Feedback Patient Survey is a resource provided in the training modules that provides hospitals both the survey and guidance on how to administer it. It will also include ideas on how to maximize patient response rates with tactics such as explaining how the patients’ response will help other patients have a better hospital experience and integrating provision of the survey into the flow of the informed consent discussion.

4. Tests of Procedures

All of the data collection protocols have been reviewed by the project expert panel members (see: Supporting Statement A, Section 8.b. Outside Consultants) with experience obtaining, improving and studying informed consent practices. The expert review helps establish the face and content validity of the protocols. Additionally, to the extent possible, the data collection protocols use or adapt existing items from previously tested and validated instruments.

The items for both Pre-/Post-Training Quizzes were developed based on the content in the training modules with a few questions for the post-quiz to capture reactions to and evaluation of the training modules training immediately after completing the training. These questions are similar to questions used in other training evaluations. Both pre-/post-training quizzes will be pre-tested with a few staff from AHRQ’s contractors (Abt Associates and The Joint Commission) to ensure the items reflect the training modules content and to identify confusing or wordy items to ultimately modify the quizzes, as needed.

The Health Care Professional Survey is a newly developed survey, though items regarding professional background are taken from the AHRQ Hospital Survey on Patient Safety. We will pre-test the Health Care Professional Survey with expert panel members who are health care professionals and staff from the organizations conducting this research who are clinicians to identify wording issues, verify the length of time needed to complete the survey, ensure the questions appropriately reflect the informed consent discussion as it happens in practice, and identify any major gaps.

The Rapid Feedback Patient Survey consists of patient satisfaction items taken from CAHPS surveys, reported comprehension of informed consent items adapted from related studies of informed consent (e.g., Enama et al., 2012)¹, and newly developed items to obtain a patient’s perspective on an informed consent discussion immediately after it happened. The Rapid Feedback Patient Survey has been critically-reviewed by the expert panel members who are clinicians, patient advocates and those who studied informed consent practices with patients. It has also been reviewed by staff from AHRQ’s contractors with expertise in cognitive testing, surveys, health literacy and limited-English proficiency. Ultimately, this data collection effort will help us to assess whether hospitals might be willing to implement such a survey as part of quality improvement effort to improve informed consent and what changes may be helpful to the survey that should be instituted for the final training modules. Further testing of this instrument may be considered for a later data collection effort.

5. Statistical Consultants

Abt Associates is the contractor who will facilitate hospitals’ data collection and analysis on behalf of AHRQ. The professionals from Abt Associates have over 40 years of experience providing high quality, timely and cost effective data collection for federal agencies. Abt Associates employs many statisticians, economists and experienced research methodologists. Statistician and health economist Lauren Olsho, Ph.D., from Abt Associates, was consulted and reviewed the proposed statistical analyses. Dr. Olsho has designed several rigorous, practice-based research studies for AHRQ and other federal agencies. She is available should any questions regarding the statistical analyses for this project arise. The key project contact at Abt Associates is Sarah Shoemaker.

Key contact information for Sarah Shoemaker is provided below:

Sarah J. Shoemaker, PhD, PharmD, Senior Associate/Scientist

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138

Office: 617.349.2472

Fax: 617.386.7638

Email: sarah_shoemaker@abtassoc.com

1 Enama et al. Randomization to standard and concise informed consent forms: Development of evidence-based consent practices. Cont Clinical Trials 33 (2012) 895–902.

7