Supporting Statement B -- Development of a Health Information Rating System 9-19-2012

SUPPORTING STATEMENT

Part B

Development of a Health Information Rating System (HIRS)

September 18, 2012

Agency for Healthcare Research and Quality (AHRQ)

Table of Contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 5

4. Tests of Procedures 5

5. Statistical Consultants 5

B. Collections of Information Employing Statistical Methods

B.1. Respondent universe and sampling methods

The study will involve primary data collection from consumer testing interviews with 48 adult consumers. The respondent universe is the U.S. adult population. We will use a non-probability, convenience sampling method to identify 48 English-speaking consumers with a high school diploma or less education. We will exclude consumers if they, or someone they care for, have received the procedure (i.e., colonoscopy), or have the condition or used the treatment (i.e., asthma or used an inhaler) presented in the selected health education materials. To identify and recruit consumers, we will engage a Boston area agency to identify individuals who meet the aforementioned criteria. The agency can recruit consumers from their database of individuals who have not participated in research before or those who have not participated in a research study in the past 6 months. The agency will assess the candidate consumers on the screening criteria: English-speaking, high school diploma or less education, and they, or someone they care for, have not received a colonoscopy, or have asthma or used an inhaler. The first 48 consumers who meet the selection criteria and are able to participate during the scheduled times will be the individuals who participate in consumer testing. No claim is made that the results from this study will be generalizable.

B.2. Information Collection Procedures

A total of 12 different health education materials will be used in consumer testing . The materials include different modalities (i.e., printable and audiovisual) and 2 health topics (colonoscopy and inhaler use). There are 6 materials for each topic. Materials for each modality and topic have received different HIRS rating levels for understandability and actionability. The selected set of materials includes both understandable and poorly understandable materials, actionable and poorly actionable materials, and just poorly understandable materials. This variation on quality will allow us to assess each domain both independently and collectively.

Each consumer will be randomly assigned 1 of the 6 materials on each of the 2 topics. This means that each consumer will be randomly assigned one of the “colonoscopy” materials and one of the “inhaler use/asthma” materials, which vary in quality on understandability and actionability, and represent different modalities of materials. This will allow us to compare the consumer testing results between materials that vary in quality for each domain to assess the construct validity of the HIRS, and help us determine how to revise and improve the HIRS.

Exhibit 1 lists the materials (hypertext includes links to the web-based educational materials), the type of materials, the HIRS understandability ratings, and the HIRS actionability ratings.

Exhibit 1. Health Education Materials for Consumer Testing

Material	Modality	Understandability	Actionability
Topic 1: Asthma/Inhaler Use
How to Use an Inhaler, by the Utah Dept Health Asthma Program	Audiovisual	Understandable	Actionable
Asthma Triggers, by Children’s Healthcare of Atlanta	Audiovisual	Understandable	Poorly Actionable
Asthma Inhaler Medication Technique - How to Take An Asthma Inhaler, by America's Allergist	Audiovisual	Poorly Understandable	N/A
Asthma: How to Use A Metered Dose Inhaler, by FamilyDoctor.org	Printable	Understandable	Actionable
How to use an inhaler - no spacer, by MedlinePlus	Printable	Understandable	Poorly Actionable
Inhaled Asthma Medications: Tips to Remember, by the American Academy of Allergy Asthma & Immunology	Printable	Poorly Understandable	N/A
Topic 2: Colonoscopy
Colonoscopy Patient Education Video, by Krames	Audiovisual	Understandable	Actionable
Colorectal Cancer Awareness by St. Vincent’s Healthcare	Audiovisual	Understandable	Poorly Actionable
Prepare for a colonoscopy, by U. of Texas MD Anderson Cancer Center	Audiovisual	Poorly Understandable	N/A
Getting Ready for your Colonoscopy, by West Chester Endoscopy Suite	Printable	Understandable	Actionable
Colonoscopy, by National Digestive Diseases Information Clearinghouse (NDDIC)	Printable	Understandable	Poorly Actionable
Colonoscopy, by the American College of Surgeons Division of Education	Printable	Poorly Understandable	N/A

Prior to the testing we will obtain informed consent (see Attachment H), collect demographic information (see Attachment C), and administer the English version of the Short Test of Functional Health Literacy in Adults (S-TOFHLA) to determine each consumer’s health literacy skill level (see Attachment D).

To begin the testing, the consumer will first read or watch each randomly assigned material. Then an interviewer will ask the consumer a set of questions for each topic to assess his/her understanding of each material’s content (understandability), and the extent to which he/she knows what actions to take or how to act (actionability), if applicable. The pre-selected materials are publicly available on-line. The materials (with hyperlinks) are provided with the type of material they are, their understandability rating, actionability rating (if applicable), and with the associated health education material questionnaire for that topic. Attachments E and F have the list of pre-selected materials and the associated questionnaires for each.

B.3. Methods to Maximize Response Rates

Since this is a non-probability sample, we do not have a target response rate. However, we will aim to maximize participation of consumers who meet the selection criteria. We will facilitate recruitment by providing an honorarium commensurate with the time requested of participants (i.e., $75 for 1.2 hours). To ensure effective identification and recruitment of the desired sample, we will hire a Boston-based firm to recruit the participants, as describe in section B.1 above.

B.4. Tests of Procedures

To test the consumer testing procedures, the interviewers will do a dry run of the protocol. To further test the procedures, the protocols were reviewed by cognitive testing experts and an expert in health literacy and patient education with experience conducting similar assessments.

B.5. Statistical Consultants

Abt Associates is the contractor that will conduct the data collection and analysis for AHRQ. The professionals from Abt Associates have over 40 years of experience providing high quality, timely and cost effective surveys for federal, state, local, and private clients. Dr. K.P. Srinath at Abt Associates was the statistician consulted for this data collection effort; his contact email is KP_Srinath@abtassoc.com and telephone (301) 634-1836. Dr. Michael S. Wolf of Northwestern University, and Allyson Ross Davies of ARD consulting were also consulted. The key contact at Abt is Sarah Shoemaker; her information is below.

Sarah J. Shoemaker, Pharm.D., Ph.D.

sarah_shoemaker@abtassoc.com

Abt Associates, Inc.

55 Wheeler Street

Cambridge, MA 02138

Phone: (617) 349-2472

5