3170-0022 Supporting Statement B-30day OMB

30-Day Federal Register Notice

CONSUMER FINANCIAL PROTECTION BUREAU

PAPERWORK REDUCATION ACT SUBMISSION

SUPPORTING STATEMENT PART B

Generic Information Collection Plan for the Development and Testing of Disclosures and Related Materials

(OMB CONTROL NUMBER: 3170-0022)

Data collection methods and procedures under this generic clearance will vary depending on the individual collection. However, the primary purpose of collections under this clearance will be to design, develop, and evaluate disclosures and related materials relating to consumer financial products and services. Some projects will focus on disclosure development and cognitive testing using qualitative research methods; others will use quantitative methods to rigorously evaluate and compare disclosures. Each method is described in the sections below.

1. UNIVERSE AND RESPONDENT SELECTION

The activities under this clearance may involve samples of respondents selected either to cover a broad range of consumers or to include specific characteristics related to consumer financial products or services. Further, to the extent the testing under this clearance relate to compliance, samples of respondents may be selected to reflect characteristics of covered entities that may utilize such materials to facilitate compliance with Bureau regulations.

The specific sample planned for each individual collection and the method for soliciting participation will be described fully in each collection request. Generally, the Bureau anticipates that testing will involve convenience samples, with participants selected to include specific characteristics related to certain products. For example, when the Bureau recruited samples to test mortgage servicing disclosures, potential participants were screened by criteria including their home purchase and mortgage loan experiences. Initial screening of volunteers will be done to identify eligible respondents.

In conducting its testing, the Bureau or its contractor may contract with research facilities in various locations to recruit the appropriate populations. Demographics to be considered in recruitment and screening include geographic location, metro/micropolitan city size, age, race and ethnicity, education, income, gender, and marital status.

In some cases, the Bureau may contract with firms that have existing access to panels of consumers who have expressed an interest in participating in research studies. These may include firms that provide recruitment and facilities for conducting in-depth qualitative interviews with a population local to the facility, or online panels that field structured data collections with consumers across the country. The Bureau and its contractor will screen potential participants from the panel to ensure they represent a diversity of characteristics and meet eligibility criteria for the study.

The Bureau typically seeks a mix of participants and characteristics and does not expect to meet all of the criteria in any one site or single round of data collection. Patterns will be tracked across sites and collections to ensure a range of consumers participate in the study. To the extent practicable, sites will be selected that provide diversity among small, medium, and large sized cities, as well as geographic diversity.

Because the projects conducted under this clearance will use a convenience sampling approach, results will not be used to produce estimates of a population.

2. METHODS AND PROCEDURES FOR COLLECTING INFORMATION

Data collection methods and procedures will vary and the specifics of these will be provided with each collection request. The Bureau expects to use a variety of methodologies for these collections. For example, the Bureau or its contractors may use one-on-one cognitive usability interviews or focus group interviews with professional guidance and moderation. In addition, the Bureau may collect information through structured questionnaires administered online. The following describes the methods that the Bureau has used to test consumer financial disclosures and anticipates using for qualitative and quantitative testing under this clearance.

Focus Groups

Focus groups are a useful method for gathering qualitative information from consumers about their preferences and interactions with consumer financial products and services to inform the design of a disclosure. This method typically occurs near the beginning of the disclosure development process.

Focus group interviews usually consist of 6-10 people per group and last typically for one hour. In these small group sessions, a moderator uses an interview guide to pose questions asking the respondents to discuss their individual experiences with and perspectives on consumer financial products, services, and disclosures. The moderator tries to involve everyone in each question to better understand the range of attitudes, opinions, and concerns within the group. Participants may review existing or revised disclosures and be asked to provide feedback, or they may discuss their preferences for prioritizing disclosure content.

One-on-One Interview Method

This qualitative approach is the method the Bureau most frequently uses in the disclosure development and qualitative testing process. One-on-one interviews use usability testing techniques to provide in-depth insights about a consumer’s experience with a disclosure and to identify promising or problematic approaches to disclosure communications and design.

One-on-one interviews typically last 60-90 minutes. At the beginning of the interview, respondents are informed of the interview timing and that they can step out or end the interview at any time. Respondents will be monitored for fatigue by the contractor’s testing personnel and the Bureau. The contractor’s testing personnel and the Bureau will keep track of the length of interviews.

The specific activities are determined by the hypotheses to be tested. The most commonly used method is a think-aloud interview in which a respondent is asked to review a disclosure and to describe how they would use the disclosure and what questions they would have. The interviewer then follows up with questions pertinent to the disclosure and usually probes extensively to ascertain the participant’s degree of comprehension. Respondents may view multiple versions of a disclosure or multiple approaches to presenting a subset of information on the disclosure, providing similar feedback and discussing their preferences and reactions across options.

All respondents are debriefed at the end of the interview. The interviewer reiterates the purpose of the project and answers respondents’ questions about the study.

In many cases, the Bureau will conduct several such interviews using prototype disclosures that it has designed. After completing a round of interviews, the Bureau will revise the disclosures in response to the information gathered through the interviews, then conduct another round of interviews with the revised disclosures. This iterative method enables the Bureau to incorporate input from a variety of consumers to improve disclosures.

Quantitative Testing

Quantitative testing uses structured data collection methods to quantitatively measure the performance of a disclosure. The Bureau has used this approach to evaluate disclosures after using qualitative methods to develop them. Under this clearance, quantitative testing may also be used during the disclosure development process to help the Bureau prioritize among available options or evaluate a feature of a disclosure.

Quantitative testing will be administered by providing consumers with disclosures to view and by asking for responses to a structured questionnaire. This will often be conducted online, although this method may vary with the specific research questions that will be addressed by the testing. After participants are recruited and screened for eligibility for the study, they will be randomly assigned to view different disclosures or disclosure elements. Other elements of the testing, such as the questionnaire and procedures for administering the testing, will be held constant as applicable. Bureau researchers will conduct power analyses before each such test to ensure the sample size recruited is sufficient for statistical analysis.

This approach will enable the Bureau to measure how a revised disclosure compares with an existing disclosure (such as the testing that the Bureau conducted when comparing its revised mortgage origination disclosure forms with existing forms), or it can provide comparisons of alternative versions of a disclosure (such as the Bureau is seeking to conduct for disclosures related to debt collection activities). However, as the Bureau generally expects to use convenience sampling methods, the results will not provide population estimates. Instead, such studies will provide statistically valid comparisons between conditions.

The Bureau typically measures consumer comprehension of disclosures when conducting quantitative testing. Other outcome measures examined may include more subjective ratings of the disclosure’s usability and design, or consumers’ attention to the disclosure. The Bureau will identify the outcome measures of interest for each quantitative test conducted under this clearance.

3. METHODS TO MAXIMIZE RESPONSE

To maximize response to quantitative tests, the Bureau will use best practices in survey design and methodology. This will include ensuring the questionnaires used in quantitative testing are short enough to minimize attrition and enable consumers to complete the questionnaire without fatigue. The Bureau will commonly adhere to a goal of 15 minutes for respondents to participate in quantitative tests, on average. Other practices that will be used to maximize response will include follow-up reminders with consumers who have been invited to participate in the testing

Each collection request will provide information regarding the methods used to maximize response in light of the testing procedures applicable to that collection request.

4. TESTING OF PROCEDURES

Pretesting of data collection materials and methods will be conducted by the Bureau or its contractors, where applicable, using internal staff or a contractor’s employees as part of their routine job responsibilities. For quantitative data collection, the Bureau will generally conduct qualitative testing of the materials with a small number of consumers to refine the instruments, in addition to technical pretesting of the survey recruitment and programming.

5. CONTACTS FOR STATISTICAL ASPECTS AND DATA COLLECTION

The name and telephone numbers of these individuals will be provided in the clearance request for each specific data collection. The Office of Research is currently staffed with numerous employees with PhDs in social science, who are trained in relevant methods. For any research studies that employ statistical methods submitted for approval under this information collection plan, such methods will be reviewed by a qualified statistician for scientific rigor, including compliance with the Office of Management and Budget’s (OMB) statistical programs and standards, prior to submission to OMB.

Page 3 of 3