Supporting Statement Part B -- Evaluation of GLIDES -- Response to OMB 6-4-2010

SUPPORTING STATEMENT

Part B

Evaluation of the GuideLines Into Decision Support (GLIDES)

Version: February 3rd, 2010

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 4

3. Methods to Maximize Response Rates 5

4. Tests of Procedures 5

5. Statistical Consultants 6

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

This study is a component of the evaluation plan for the AHRQ-funded contract, GLIDES, or GuideLines Into DEcision Support. In the GLIDES project, AHRQ has designed and implemented decision support tools based on guidelines for the prevention of pediatric overweight and obesity and the management of chronic asthma in the pediatric population. The evaluation plan includes a physician survey, in which participating healthcare providers (primary care pediatricians and pediatric pulmonologists, as well as advanced practice nurses) will be surveyed about their experiences with the decision support tools developed for this project. In this way AHRQ can evaluate the fulfillment of knowledge transformation goals and the effectiveness of the decision support tools in improving the quality of health care at the chosen sites. This information will be used to determine the effectiveness of the tools. In addition, subgroup analyses, such as those by site, level of experience, and rate of intervention uptake, will assist in determining whether these decision support tools should be designed, targeted and implemented differently for different audiences. This evaluation is not designed to be nationally representative of all pediatricians or pediatric pulmonologists, but is rather designed to assess the effects of a site-specific intervention. The lessons learned from this research will be helpful in both refining these tools and implementing them more broadly.

Questionnaire Sample:

There are a total of 262 clinicians who are eligible for the questionnaire by virtue of taking care of patients for whom the CDS tool is designed, 49 of which are also eligible for in-person interviews. . Since this number is not very large, particularly considering prespecified subgroup analyses by site and demographics, all will be contacted for participation in the survey. Based on prior survey work in this population at these sites, a 75% response rate is anticipated, resulting in about 254 completed questionnaires from 197 clinicians.

Interview Sample:

For the interview, we will purposefully select high and low utilizers to ensure a diversity of opinion in the interviews, based on a query looking at use of the decision support tools. There were 3 sites in Florida where the asthma tool was rolled out, making at least 6 potential interviews; there were 7 sites in which the obesity tool was rolled out at Nemours, making at least 14 potential interviews: approximately 20 interviews altogether at Nemours. If we have not reached thematic saturation at this point (i.e. we are still generating new ideas and themes), we will continue interviews, again purposefully selecting high and low users. At Yale there are only 9 physicians in the pulmonology practice so we will interview all of them to obtain a reasonable sample. In the primary care setting at Yale, we will select 10 primary care attending physicians and 10 primary care house staff to conduct interviews about obesity and asthma tool utilization, again using the purposeful sampling method of high and low users for the interviews. Respondents will be selected using data from the clinical information system to identify high and low users; this is to ensure a representative sampling for the interviews.  Among these groups selection will be convenience-based - who is available and willing to talk.  Qualitative interviews are not random samplings.  Every effort will be made as per standard scientific protocol to ensure a wide diversity of opinion in the interviews.

Questionnaire and Interview Potential Respondents:

The distribution of potential respondents among type and site is as follows (note that since some clinicians are eligible for both questionnaires and the in-person interview, the total number of responses (387) exceeds the total number of unique respondents (262):

Pulmonology attending or fellow Allergist Primary care pediatrician Pediatric house staff Advanced Practice Registered Nurse Total

Yale

Asthma questionnaire 6 1 12 50 14 83

Obesity questionnaire 0 0 12 50 14 76

In-person interview 8 0 10 10 1 29

Nemours

Asthma questionnaire 14 5 50 68 9 146

Obesity questionnaire 0 0 27 0 6 33

In-person interview 6 0 14 0 0 20

Total 387

2. Information Collection Procedures

Since the universe of potential respondents is small there will be no stratifying or sampling of respondents. The analysis will have power to detect a 0.5 point difference in Likert-type scale responses between major subtypes of respondents (i.e. Yale sites vs. Nemours sites) at a 0.05% confidence level. Previous work has found that differences between categories of respondents (for example, between primary care pediatricians and pulmonologists) on these topics are more on the order of 1.0 points on a Likert-type scale. The analysis will have sufficient power to detect differences between both the physical sites and clinician types.

The questionnaires will be self-administered without aid from investigators. There will be no in-person interviews, advance appointments/letters or visits from investigators. The majority of data will be self-entered by respondents on a web-based survey tool. Some paper-based questionnaires may be collected, which will be entered into the study database by a trained research assistant with random checks initiated by the physician evaluation director.

The analysis of the qualitative data will follow standard qualitative analytic techniques using the inductive, grounded theory approach as described in Glaser's seminal work: Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, IL: Aldine; 1967.

Participants will not be re-interviewed or re-contacted. We will not impute missing data. Analyses will be conducted using SAS 9.1.2 (SAS Institute, Cary, NC).

3. Methods to Maximize Response Rates

We will maximize response rates by sending an e-mail-based survey request to each group of participants from a local champion who is known and respected by that group of respondents (i.e. a pediatric pulmonologist for the pediatric pulmonology group; a chief resident for pediatric house staff). A maximum of three reminders may be sent to non-respondents. If necessary, questionnaires will be distributed in paper-based format at clinic sessions in which physicians are present in person. This study includes all practicing clinicians in each office; thus paper-based copies of the questionnaire will not be inadvertently distributed to non-study clinicians.

The universe of respondents is divided into 4 groups: pediatric pulmonologists at Yale University School of Medicine, primary care pediatric faculty and house staff at Yale-New Haven Hospital, pediatric pulmonologists at Nemours in Florida and primary care pediatricians at Nemours in Delaware. All are physicians or nurse practitioners; all will have had direct clinical experience with decision support tools intended to assist them with prevention of childhood overweight/obesity and management of chronic asthma. The questionnaire for each condition is 5 pages long, comprised primarily of Likert-type scale questions, and addresses physicians’ perceptions of guidelines, of decision support systems in general, of their workflow, and of the decision support tools in this study in particular. There will be one questionnaire for overweight/obesity and one questionnaire for asthma. Pulmonologists will receive the asthma questionnaire. Primary care physicians will receive the overweight/obesity questionnaire. Primary care physicians at Yale will receive both questionnaires (each at a different time, or combined as one 8 page questionnaire) because they will be exposed to both decision support tools. Data collection will be web-based as much as possible with paper as a back-up. There will be no remuneration for participation.

4. Tests of Procedures

The questionnaires have been pilot tested with < 10 respondents to refine and revise the instrument. We consider this adequate.

5. Statistical Consultants

Leora Horwitz, MD, MHS and Gabriela Ramírez-Garnica, PhD, MPH, are the evaluation leads at Yale and Nemours, respectively. Each has advanced training in research design and statistics and will be collecting and analyzing the information for AHRQ. Dr. Horwitz can be reached at (203) 688-5678. Dr. Ramirez can be reached at (407) 650-7062.

6