2487-02_ssPartB_2017-02-16

February 16, 2017

EPA’s Safer Choice Label Consultations

EPA ICR No. 2487.02 OMB Control No. 2070-0189

PART B OF THE SUPPORTING STATEMENT

B.1 RESPONDENT UNIVERSE AND SAMPLING METHODS

EPA proposes the use of an online (i.e., web-based) sampling approach as a complement to the consumer focus groups described in Part A of the supporting statement. The online survey to test consumer awareness and perceptions of EPA’s Safer Choice Label is included as Attachment E of the supporting statement. To support implementation of the consumer online survey, EPA would retain Ipsos, a vendor that maintains a preexisting representative national panel.

The online survey will seek a panel-based (non-probability) sample of 2,000 U.S. adults ages 18 and older that has a reasonable degree of diversity in key demographic characteristics such as age, gender, education, and race/ethnicity. The sample of individuals invited to participate in the survey will be balanced to the U.S. population, based on demographic information (age, gender, educational attainment, geographic region, etc.) collected by the U.S. Census Bureau through the Current Population Survey. Once the data are collected, the sample is weighted on the same demographic variables to adjust the sample to U.S. adult population and correct any potential differential responses by demographic groups.

While the results will be weighted to the U.S. adult population, EPA does not intend to generalize the results to the overall population or produce precise estimates of population parameters using the surveys. The study will measure the effect of stimuli (i.e., logo designs) within the experiment and use a convenience sample of self-selected members of an established online consumer panel, rather than a probability-based sample.  The study design will yield valid and quantitative estimates of differences inside the experiment in consumer responses caused by variations between the different logo designs.

The overall sampling frame for the online consumer surveys is the panel of individuals previously recruited by Ipsos (i.e., the Ipsos Online Panel) to participate in online surveys. The Ipsos Online Panel is an actively managed research access panel that uses multi-source recruitment to maintain a representative base of respondents. It includes individuals who have volunteered to take part in market research and is extensively profiled to efficiently target respondents.

Ipsos employs a survey router (brand name is Cortex) to manage a sample so that it will be representative of a population. Ipsos follows best-in-class principles to measure and report any selection bias that arises from the use of a router:

• Designated team to manage and monitor the router, made up of Ipsos sampling and methodology experts

• Router management team has final authority over which studies are in the router and any prioritization decisions

• Restricted impact of highly-targeted studies

• Random and priority reallocation is balanced

• Respondents allowed to opt-out of the screening process with no penalty

• Diverse, large number of studies maintained in the router

• Limited number of custom screening questions allowed

• All router studies are reviewed by the router team and removed from the router if necessary

• Key metrics monitored multiple times daily to identify potential performance issues

• Supplier traffic is monitored daily to ensure consistency

• Representativeness and consistency of the router population to the general population is monitored

Panel management protocols include a series of legitimacy checks, including name/address matching using external databases, digital fingerprinting, and TrueSample. Ipsos utilizes inbuilt criteria such as the exclusion of individuals who repeatedly fail to respond to survey invitations, as well as the use of algorithms to screen out participants who exhibit undesirable survey behavior (e.g., inconsistent response, straight-lining, speeding through, etc.).

In addition, steps are taken to ensure that panel members are not overburdened with survey requests. The primary sampling rule is to assign no more than one survey per week to members. This level of survey frequency helps to keep panelists engaged as part of the panel. To minimize panel attrition, surveys are usually kept short (under 20 minutes in length). For surveys requiring 16 or more survey minutes, survey participation is rewarded with points that can be redeemed for cash or prizes. The survey we are proposing will not exceed 20 minutes, and participants will be offered a small incentive of points with a cash value of less than $1. We anticipate a 75% response rate for this survey from among invited participants in the panel.

B.2 PROCEDURES FOR THE COLLECTION OF INFORMATION

The online survey includes questions to test awareness and perceptions of EPA’s Safer Choice Label.

Ipsos conducts all of its online data collection via websites. Through the sampling process, panelists are pre-selected to answer a certain survey; the surveys are not “open access” (i.e., respondents are not self-selected), and respondents do not know the survey content. Panelists receive an invitation email with the following information:

• Survey information (end date, survey number, survey duration, number of incentive points)

• A unique URL that provides access to the questionnaire

• Physical address for Ipsos

• Member support email address

• Link to privacy policy

• Opt-out information

To ensure high quality data, data submitted through the consumer online surveys will be subjected to statistical validation techniques, such as disallowing out-of-range values.

All data collection and analysis will be performed in compliance with OMB, Privacy Act, and Protection of Human Subjects requirements.

Confidentiality, Privacy and Opt-In Details:

Participation in research is voluntary at the time respondents are asked to join the panel, at the time they are asked to participate in any particular survey, and at the time they answer any given question in a survey. Panelists are not coerced to participate in any research and are not removed from the panel as a result of failure to participate in any particular survey project or program of studies. Panelists have the option to ‘opt-out’ of the panel at any time by notifying Ipsos. Ipsos maintains a toll-free phone number and its own call-center panel management facility to receive requests for information and action from panelists.

Survey responses are confidential, with identifying information never revealed without respondent approval. When surveys are assigned to panel members, they receive notice in their password protected e-mail account that the survey is available for completion. Surveys are self-administered and accessible any time of day for a designated period. Participants can complete a survey only once. All panelists, when joining the panel, are given a copy of the Privacy and Terms of Use Policy. The privacy terms are also available electronically at all times to panelists via the panel member website.

B.3 METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH NON-RESPONSE

Ipsos’ methodology for maximizing response rates includes the following:

• Targeted recruitment through various “wide net” methodologies (e.g., email campaigns, affiliate networks, banner ads, text ads, search engine, co-registration, offline-to-online, specialized websites)

• Use of a point system to incentivize panelists, along with sweepstakes draws. Points systems are recognized as being the best in class in online market research, as they are seen as a neutral system which does not skew the participation of specific groups of people

• Customized incentives and materials for recruiting special targets such as mothers of babies, age group 55+, etc., to be used only as needed

• Continuously testing new recruitment sources and methods

• Use of an internal data quality process that incorporates data quality checks at the survey level to reduce or eliminate random responding, illogical or inconsistent responding, overuse of item non-response, and too rapid survey completion.

We do not anticipate challenges with non-response, given the broadly defined eligibility criteria for the survey. However, in the event of non-response challenges, we will send reminders to non-respondents, and monitor and correct any potential non-response biases.

Survey results will not be used for evaluation purposes, or to make policy or regulatory decisions. We believe a non-probability sample is adequate for EPA’s purpose of supporting communications testing and consumer outreach and education efforts.

B.4 TEST OF PROCEDURES OR METHODS

Before the online surveys are implemented, a contractor will pilot test the instrument(s) and method of data collection. Lessons from the pilot test will be identified, and changes as necessary will be incorporated into the instrument and method. All pilot tests will involve no more than nine individuals.

B.5 INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS AND INDIVIDUALS COLLECTING AND/OR ANALYZING DATA

Alan Roshwalb, PhD, will serve as the Sampling Statistician for this project. Dr. Roshwalb is a Senior Vice President at Ipsos and joined the company in 1995. He will work with the Deputy Statistician in executing the survey using the methodology detailed above and weighting the results to the U.S. adult population as described in Section B.1. While with Ipsos, he has served as a survey program design consultant, lead data analyst, sampling consultant, and project manager on many projects for clients such as the FDA, the Army Surgeon General, the United States Postal Service (USPS), the Federal Trade Commission, the Social Security Administration, the U.S. Patent and Trademark Office (USPTO), the Consumer Product Safety Commission, the National Institutes of Health, and Kaiser Permanente. Dr. Roshwalb is a seasoned expert in the design of surveys and sampling theory and provides consultation on quality management issues including the development of performance standards to a variety of service industry clients. He has taught statistics and applied quantitative methods, and he has authored professional papers on statistical and research methodology for social science, marketing, accounting, and management periodicals such as The Accounting Review and The Journal of Business and Economic Statistics. Dr. Roshwalb can be reached at 202-420-2029.

B.6 APPROACH TO CONDUCTING CONSUMER FOCUS GROUPS

The proposed consumer focus groups included in this ICR will not employ statistical methods.  Our approach for conducting the focus groups is outlined below.

 

Recruitment and Screening

EPA will conduct 10 focus group sessions among adult participants in three to four locations. Each focus group session will have nine participants. The focus group sessions will be conducted in person, in locations throughout the country. In selecting locations for the sessions, EPA will ensure diversity in region and market size. For example, EPA will select locations that represent diverse regions of the country, including East coast, West coast, Midwest and Southwest regions. EPA will also include a mix of larger and smaller markets. Each participant will be asked to attend a single session.

 

Participants will be recruited in each location by local recruiting resources. Participant positions will be filled through the use of opt-in databases of local residents that are maintained by local focus group facilities. To develop these databases, recruiters call local residents at random from telephone listings to ask them to participate in the database. The criteria for inclusion in the databases are generally very broad. Participants are then screened more narrowly for specific studies, based on the objectives and screening criteria of each study.

 

As is typical with focus group research, the sampling for the sessions is purposive in that recruiters are asked to find participants that meet specific criteria (i.e., consumers who consider the environment when making purchasing decisions in addition to general adult consumers). We plan to conduct five focus groups among general adult consumers and five among adult consumers that lean towards the purchase and use of green products. To identify participants, we will use two sets of screening questionnaires, one for general adult consumers (Attachment B) and one for adult consumers that lean towards the purchase and use of green products (Attachment C). For each of the five general adult focus groups, 90 people will be screened (450 in total). Similarly, 90 people will be screened (450 in total) for the five “leaning green” focus groups.

Recruiters will ensure they meet the specific target audience requirements needed for the research. In addition, recruiters will ensure diversity among participants regarding socioeconomic and other demographic factors. A standard moderator’s guide will be used in all focus group sessions to guide conversations. Each consumer focus group is designed to be a one-time information collection activity with consumers specifically recruited to provide insight into how to improve educational and awareness activities.

 

Incentives for participants: In-person focus group participants are generally provide with an incentive – usually a cash or check payment at the close of their research session – to reflect the level of burden they assume in appearing in person for the session. For the purposes of this data collection, we anticipate the total incentive/travel reimbursement offered to each consumer participant to be $75. Providing monetary incentives is a standard market research practice for ensuring an acceptable level of participation. An incentive may significantly improve validity and reliability to an extent beyond what is possible through other means. For example, an incentive will help in the recruitment of participants with diverse views and levels of interest in the topic area.  EPA is particularly interested in a balanced group of consumers and believes that providing an incentive will improve the quality of the effort.    

Confidentiality: Participants are assured by the research facility, and always by the moderator at the beginning of each group, that their names and responses are kept confidential, and will not be disclosed to anyone but the individuals conducting research in this investigation, except as otherwise required by law. Reports of the focus group sessions will not mention individuals by name. Participants are also assured that no one will try to sell them anything following this research.

 

Outcome: The focus group data will be reviewed for recurring themes and the report will focus on themes identified during the review. Preliminary and final summary reports will be provided for all focus group sessions. Findings will be used to modify relevant messages about the Safer Choice Program as necessary to respond to the needs of EPA constituencies. Differences across group location and audience will be noted when appropriate, but given the qualitative nature of this research, findings will be considered descriptive and directional but not definitive.

4