0990-TPP evaluation supporting_statement_A_1_27_12

Outcome Evaluation of “Teenage Pregnancy Prevention: Integrating Services, Programs, and Strategies through Community-Wide Initiatives”

Part A: Justification

Heather Tevendale

Health Scientist

Phone: 770-488-6147

Email: Hrx9@cdc.gov

Fax: 770-488-6291

January 31, 2021

1. Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC) Agency in collaboration with the Office of Adolescent Health (OAH) are working to reduce the number of teenage pregnancies by addressing disparities in teen pregnancy and birth rates. The Teenage Pregnancy Prevention (TPP) program aims to measure the effectiveness of innovative, multicomponent, community-wide initiatives targeting the reduction of teen pregnancy and births with an emphasis on communities with the highest rates, African-American, and Latino/Hispanic youth between the ages of 15-19. CDC is seeking OMB review and approval for a new information collection as a part of the baseline stage of the Outcome Evaluation of the Teenage Pregnancy Prevention: Integrating Services Programs and Strategies through Community-wide Initiatives.

Background

Each year in the United States, almost 750,000 young women aged 15–19 become pregnant. The U.S. rate of 71.5 pregnancies per 1,000 women aged 15–19 is the highest of all Western industrialized countries. Great disparities exist among U.S. teenagers by race and ethnicity and in different communities. Black and Hispanic women have the highest teenage pregnancy rates (126 and 127 per 1,000 women aged 15–19, respectively); non-Hispanic Whites have the lowest rate (44 per 1,000). Further, trend analyses conducted by the Guttmacher Institute and the CDC noted great variations in overall teenage pregnancy and birth rates by State and within racial and ethnic groups by State. These variations may largely reflect the distribution of different racial or ethnic groups as well as sociodemographic risk and protective factors, as well as State differences in public policies related to access to contraception and sex education.

A large body of research has focused on factors contributing to the problem of teenage pregnancy. These include youth-level factors (e.g. early sexual intercourse, inconsistent use of contraception, drug and alcohol use, delinquency, poor educational performance, low expectations of the future); family-level factors (e.g. family marital disruption, parents' lack of education, mother and/or sister having been a teen mother, lack of parental support and/or supervision); and environmental factors (e.g. violent crime, poverty, unemployment). Research has shown that these factors are associated with teenage sexual behavior and pregnancy; however, the strength and direction of the relationship varies greatly between studies. Further factors related to community attitudes and beliefs about sexual behavior, pregnancy, and childbearing include beliefs, values, and social norms that are acknowledged contributors but not often studied rigorously. Although the evidence strongly suggests that teenage pregnancy is a multifaceted problem stemming from interrelated internal and external factors, pregnancy prevention programs have typically focused on one factor only (e.g., abstinence education focused on preventing early sexual activity). Several recent reviews have emphasized that multicomponent programs implemented at the local level may offer the greatest potential in teenage pregnancy prevention. Rather than focusing on one element, multicomponent programs may include some combination of job readiness training, academic tutoring, recreation, mentoring, sexuality education, lifeskills training, and health and mental health care. The proposed baseline survey is critical to understanding the effectiveness of these efforts and how they can be improved and used in additional communities.

Legal or Administrative Requirements that Necessitate the Collection

In his budget for Fiscal Year (FY) 2010, President Obama proposed a new Teenage Pregnancy Prevention (TPP) Initiative to address the high teen pregnancy rates by replicating evidence-based models and testing innovative strategies. On December 16, 2009, the President signed the Consolidated Appropriations Act, 2010 (Public Law 111-117). Division D Title II of the Act provides $110,000,000 for making competitive contracts and grants to public and private entities to fund medically accurate and age appropriate programs that reduce teen pregnancy and for the Federal costs associated with administering and evaluating such contracts and grants. Relevant portions of the legislation are included as Attachment A.

As part of this initiative, CDC released two funding opportunity announcements (FOAs) related to innovative evidence-based teenage pregnancy prevention programs: (1) DP10-1009, Teenage Pregnancy Prevention: Integrating Services, Programs, and Strategies Through Community-Wide Initiatives and (2) DP10-1025, Reducing Teen Pregnancy Through Family Planning: Integrating Services, Programs, and Strategies Through Community-Wide Initiatives. The program goals for the FOAs are to (1) reduce the rates of pregnancies and births to youth in the target area, (2) increase youth access to evidence-based and/or evidence-informed programs to prevent teenage pregnancy, and (3) increase linkages between teenage pregnancy prevention programs and community-based family-planning services. This data collection request relates to the evaluation of the first FOA, the Teenage Pregnancy Prevention: Integrating Services Programs and Strategies through Community-wide Initiatives. The CDC is proposing this data collection in compliance with Section 301 of the Public Health Service Act (42 U.S.C 241) Authority of Secretary: “The Secretary shall conduct in the Service, and encourage, cooperate with, and render assistance to other appropriate public authorities, scientific institutions, and scientists in the conduct of, and promote the coordination of, research, investigations, experiments, demonstrations, and studies relating to the causes, diagnosis, treatment, control, and prevention of physical and mental diseases and impairments of man.”

The CDC is seeking approval of the proposed outcome evaluation, in which 2 of the nine funded communities will participate in an in-depth evaluation. The in-depth evaluation communities are being selected through an Evaluability Assessment process, where evaluation team members visit each community to discuss data availability, project activities, and potential for evaluation. In each of the selected communities, youth between the ages of 15-19 will be recruited and interviewed over a 12 week period. CDC proposes to utilize a modified version of the Pregnancy Prevention Approaches (PPA) household survey (OMB No. 0970-0360) for baseline data collection. The PPA survey first received OMB approval on July 26, 2010 for use in the Administration for Children and Families’ evaluation of the Adolescent Pregnancy Prevention Approaches (PPA). It is also being proposed for use by the Office of Adolescent Health and the Office of the Assistant Secretary for Planning and Evaluation in the Impact Evaluation of the OAH Teen Pregnancy Prevention Program Grantees. The survey, however, has been modified for the Outcome Evaluation of Teenage Pregnancy Prevention: Integrating Services, Programs, and Strategies through Community-Wide Initiatives. See Attachment C for a complete instrument with revisions (the yellow highlights indicate new items added for use in the proposed study).

The PPA survey will be administered by the contractor, ICF International (ICF). Upon OMB approval ICF will begin collecting baseline data.

Overview of Data Collection Procedures

Data collectors prepare for the field each week by meeting with the field supervisor to receive household assignments. Field supervisors will prepare individual packets for each data collector that will contain the following documents:

• Assigned addresses with household ID numbers

• Blank expense report form

• Clipboard

• Crisis brochures

• Copies of the lead letter sent in advance

• Copies of the paper version of the survey

• Manila envelopes for completed consent forms, receipt copies, expense reports

• Household contact logs

• Household record forms

• Household phone screener forms

• Parent brochures

• Parent consent forms

• Participation incentives

• Sorry I Missed You door hangers

• Survey Definitions flipbook

• Young adult consent forms

• Youth assent forms

• Youth Payment receipts

Once the data collector arrives at the assigned address and find that the home is occupied, the data collector will proceed to screen the home to find out if anyone in the home is eligible to participate in the study. The data collector will introduce themselves and answer any possible questions. The next step is to read the screening form to the person who answered the door. The screener will determine if the screening respondent is 18 years of age or older. If the screening respondent is 18 or older, the data collector’s next set of questions will determine how many people live in the home over the age of 20, under the age of 15, between the ages of 15 and 19 years, and their demographic information. If there is an eligible teen in the home, the data collector will ask the parent’s /guardian’s permission for the teen to participate in the study. Any household that has multiple youth in the 15-19 years age range will have the youth whose birthday is closest to the survey visit participate.

After a youth has been selected to participate in the study and both the parent and teen are available, the data collector will obtain consent to participate from both the parent and teen (unless the youth is aged 18 or 19 years). If the parent or guardian is not available the data collector will reschedule and attempt to contact the parent/guardian at a later date. A copy of the consent will be given to the parent or guardian so they can follow along as the data collector reads the form aloud. The data collector will answer any questions that the parent or guardian may have. A signed copy of the consent form will be provided to the parent or guardian for their records. The parent or guardian will be asked if the teen is available. The teen will be asked if he or she would like to participate in the study. The data collector will answer any questions the teen may have about the survey. The data collector will also receive assent from the teen. If the respondent says no to the survey at that point, the data collector will thank the respondent politely for his or her time and make notation on the call log. Paper copies of the assent will also be available should a teen want to read it before he/she begins the survey.

Once the teen has agreed to participate, the data collector will ask for a private place to set up the laptop. Each respondent will be provided with earbuds so that teens are not embarrassed by the questions or their responses. The need for privacy will be stressed to both the parent/guardian and teen. Data collectors will explain that privacy is essential for the validity of the interview. Parents will be reminded that the interviews are confidential. If there is no location in the house that the respondent can be completely alone due to space or furniture set-up the data collector will attempt to provide as much privacy as possible for the respondent. If the computer screen on which the youth is shown the survey questions cannot be kept from view of others in the home, the interview will be rescheduled. Additionally, data collectors will never continue an interview with a participant that they know. The field supervisor will reassign the data collector if the assigned address includes a resident known to the data collector.

The survey will utilize audio-computer-assisted self-interviewing (A-CASI), which can increase the comfort level of respondents when disclosing sensitive behaviors, and adding an

audio component to a CASI tool is a successful way to increase the accessibility of the system. Throughout the survey, there are instructions that emphasize the importance of honesty and reinforce that responses will be kept private to reduce potential social desirability bias. During the administration of the survey, if the respondent has questions about unfamiliar terms, the data collector will direct the teen to a flipbook that will include definitions of terms, alternate terms (e.g. slang), and photos where appropriate. There will also be prompts within the system that reference the flipbook of definitions.

In closing, the data collector will thank the respondent, answer any final questions and provide the respondent with the study incentive. The data collector will explain that their field supervisor may call later for quality control purposes only.

The interview protocol, as well as all other aspects of this evaluation, has been approved by both the CDC IRB and the ICF IRB.

Overview of the Data Collection System

The in-depth outcome evaluation measures values, beliefs, and social norms about risky behavior among youth in up to three target grantee communities and up to three comparison communities. The new data collection activity utilizes a repeat cross-sectional design, in which data will be collected from representative samples of adolescent girls and boys ages 15 to 19 years old. This request for approval covers the baseline administration of the household administration of the modified PPA survey.

For the household administration of this baseline survey, CDC proposes to rely on a two-stage stratified clustered sample. At the community level, this strategy would parallel in several aspects the one implemented by the CDC’s National Survey of Family Growth (NSFG). We intend to administer the survey with between 1,200 and 1,500 youth per community and administration wave (2 target and 2 comparison communities will be selected).

Data collectors will visit homes in-person based on the sampling protocol and inquire about whether the household includes youth between 15 and 19 years of age. If so, data collectors will proceed to secure informed consent to participate from both the parent and the youth (unless the youth is aged 18 or 19). If the parent is unavailable, the data collector will follow a protocol to revisit the home at a more convenient time. Due to the sensitive nature of the questions on the youth self-report survey, the survey will utilize A-CASI technology to maintain the confidentiality of responses while enhancing response and completion rates. Respondents with Internet accessibility will be offered flexibility in response modes; they will be provided with a URL through which they can securely access the survey while the data collector waits. This option affords the respondents a means of response that may be more familiar to them and would also include a non-audio option. Data will be kept secured for five years at the conclusion of the study.

The Outcome Evaluation of Teenage Pregnancy Prevention: Integrating Services, Programs, and Strategies through Community-wide Initiatives will focus on the combined change in two proportions: (1) the proportion of teens who did not have sexual intercourse during the 12 months prior to the interview and (2) the proportion of teens who used contraception consistently during the 12 months prior to the interview.

Items of Information to be Collected

At no time during the proposed data collection activity will the CDC be in possession of the identifiable information. The contractor, ICF, will de-identify all information prior to transmission. Address, name, phone number, and birth dates will be collected from households selected for the sample. Individual responses will be kept in a secure server, separate from the survey responses, and will be deleted once participating households have been identified and contacted and the data are secure, validated, and cleaned. Please see Section A.10 for further description of the process for de-identifying data.

Table 1 displays a summary of the domains and sample questions included in the modified PPA survey. The complete instrument is included as Attachment C.

Table 1. Example of Data to be Collected

Domain	Sample question topics
Youth background	Employment status How they spend their time Near-term plans for education/employment/military, etc. School grade
Sexual Activity	Sexual behaviors Condom use Birth control use Sexual transmitted diseases Views and perceptions
Reproductive health service use	Types of services used, locations of services
Youth development	Employment activity Extracurricular activities
Dating relationships	Dating activity Dating violence
Substance abuse problems	Detailed questions on substance use
Exposure to Prevention Messages	Types of exposure (TV, billboards, fliers, social media)
Parental relationship	Parental knowledge of activities, frequency of discussions with parents about school, relationships, sex, drugs and alcohol

Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age

Not applicable, there are no plans to develop a Website.

2. Purpose and Use of Information Collection

If this request is approved, we will collect baseline data on sample members’ characteristics, family relationships, knowledge, attitudes, expectations, and behavior surrounding sexual activity and contraception use, prior receipt of services related to reproductive health, and exposure to information on reproductive health. These data will be obtained from a baseline survey administered to sample youth participating in the study.

Baseline variables are important in several ways for the analysis. They will be used to establish the baseline equivalence of the treatment and control groups on measurable variables and thus to confirm the integrity of the community selection process. Baseline variables will be used to define subgroups for which impacts will be estimated, and to adjust impact estimates for the baseline characteristics of nonrespondents to the follow-up survey. Many baseline variables will be measures of community outcomes to be measured again at follow-up.

The original PPA household survey (OMB No. 0970-0360) was developed for use across multiple teen pregnancy prevention/reduction programs (see discussion above). The use of a core set of questions across multiple programs will allow for cross-program comparisons of data and will enhance the utility of the data collected.

3. Use of Improved Information Technology and Burden Reduction

The in-depth evaluation will utilize A-CASI software to support the data collection needs of both the target and comparison communities. A-CASI technology can increase the comfort level of respondents when disclosing sensitive behaviors, and adding an audio component to a CASI tool is a successful way to increase the accessibility of the system. In addition, a Web-based alternative will allow access to the survey through an Internet-enabled browser. For both the A-CASI and Web-based alternatives, English and Spanish versions of the survey will be provided.

4. Efforts to Identify Duplication and Use of Similar Information

The PPA survey (OMB No. 0970-0360 Exp. Date 7/31/2013) was previously validated for the use in evaluation of teenage sexual risk behavior. However, CDC believes that the field of teenage pregnancy prevention will benefit from the knowledge gained from the proposed evaluation of current program implementation in funded grantee sites implementing community-wide, multi-component programs.

5. Impact on Small Businesses or Other Small Entities

The information collected will not have a significant impact on small entities; however, some programs are embedded within community-based organizations. Administration of the PPA survey will be completed by staff hired by ICF, and thus will not place a burden on small entities. In selecting communities for the in-depth evaluation described in this request, ICF staff members are visiting all grantees to conduct Evaluability Assessments. During these visits, every effort is being taken to reduce burden associated with completion of discussions with program staff.

6. Consequences of Collecting the Information Less Frequently

The current application represents a one-time data collection effort.

There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. A 60-day Federal Register Notice was published in the Federal Register on March 16, 2011, vol. 76, No. 51, pp. 14397-14398 (Attachment B). There were no public comments.

9. Explanation of Any Payment or Gift to Respondents

According to Yancey¹, incentives are likely to produce modest increases in survey responses (2006). Respondents will be offered a remuneration of $20 for their participation in the PPA survey (OMB No. 0970-0360 Exp. Date 7/31/2013). The survey is expected to take less than 45 minutes to complete.

10. Assurance of Confidentiality Provided to Respondents

Written consent (Attachment D) will be secured from a parent or guardian in the youth’s home before assent (Attachment E) is sought from the youth. In cases where the youth is aged 18 or 19, consent (Attachment F) will be sought from the youth only. No data will be collected unless required consent and assents have been obtained. Parental consent will be a core component of data collector training and will aim to increase parental knowledge about teenage sexual behaviors and pregnancy prevention.

Research has shown that the more parents understand about the issue and prevention efforts, the greater the likelihood they will consent to their child’s participation. The proposed protocol, therefore, will integrate an education component introducing parents to the issue of teenage sexual behaviors and provide information about prevention efforts either locally (in the target community) or otherwise (comparison community).

The youth’s assent process will be triggered by the adolescent age field in A-CASI, where the first screen will explain and then ask for the youth’s agreement with the assent statement. The repeated cross-sectional design permits the de-identification of all respondents subsequent to data validation and cleaning. Address, name, and other identifiable information from individual responses will be kept in a separate, secure server and will be deleted once participating households have been identified and contacted and the data are secure, validated, and cleaned.

Data collectors will synchronize their software with the central server, resulting in the automatic uploading of collected data and downloading of any questionnaire updates. Following confirmation of a successful transmission of data to the central server, the corresponding data are deleted from the local workstation. In its standard implementation, the software contains a set of common validations that are available to ensure data quality such as (a) checking that the response is of the appropriate data type (for example, that only numeric data are entered when a numeric response is expected); (b) checking that the response is within a specific length; and (c) checking that the response is within a specified range (for numeric data). In addition, the software can be tailored to study-specific validation requirements down to the individual-question level.

The QuickSAT software will use symmetric key encryption modules that have been validated to

Federal Information Processing Standard (FIPS) 140-2 to secure data files prior to transmission.

All cryptography operations will be performed by Microsoft cryptographic service providers

operated in FIPS mode. QuickSAT will also use Transport Layer Security (TLS, also known as

Secure Socket Layers or HTTPS) to further protect data and protocol information during

transmission. The application and database servers used to support this system will be hosted at our contracted Tier 4 data center, which provides managed data backup systems, disaster recovery with quick implementation of business continuity plans, uninterrupted 24x7x365 support, security controls in line with NIST recommendations, and guaranteed 99.995% system availability. In addition to A-CASI, an alternative data collection tool will be a Web-based program that can be accessed via an Internet-enabled browser. The design and functionality will closely resemble that of the CASI tool and will provide data collectors with the flexibility to tailor the data collection tool to the particular interviewee. ICF will also design and develop a Web-based data management system that will provide authenticated project staff the ability to monitor data submission activity and extract datasets for analysis. Both Web-based tools will be developed as custom applications using ASP.net or ColdFusion and Microsoft SQL Server. All servers are physically located in the same limited access room with a key or slide card access to minimize the possibility of unauthorized access, use, or dissemination of the information being collected.

Prior to data collection informed consent (Attachment D) will be obtained from a parent or legal guardian if the respondent is younger than 18 years old; consent will be sought from 18-19 year olds (Attachment F). The informed consent explains that the information will be secured to protect the respondent’s identity and responses. In addition, an explanation will be provided explaining identifiers linking them to the data will be removed and that their names will not appear in any published reports. The child assent (Attachment E) form will be the first screen in A-CASI. The screen will explain their responses will not be linked back to them and at anytime they can refuse to participate or skip a question they are not comfortable answering.

Respondents are informed that their participation is strictly voluntary and may cease participation at any time during the process.

11. Justification for Sensitive Questions

The PPA survey (OMB No. 0970-0360) includes measures that are sensitive in nature. These measures (Table 2) are important in understanding the knowledge and attitudes of 15-19 year olds about pregnancy prevention and intentions in regards to risky behaviors. Respondents are asked about race/ethnicity to evaluate whether exposure to broad based messages about teen pregnancy differ in their impact across racial and ethnic groups. Such data is useful to CDC and target communities with program planning and implementation. The knowledge gained from the measures will help to inform policy and programs around pregnancy prevention.

Because of the sensitivity of the questions the study design will be explained allowing respondents an opportunity to ask questions during the consent and assent process and/or refuse participation in the evaluation.

Table 2. Summary of Sensitive Questions and their Justification*

Topic	Justification
Intentions regarding sexual activity (questions 4.14-4.16)	Intentions regarding engaging in sex and other risk-taking behaviors are extremely strong predictors of subsequent behavior (Buhi and Goodson, 2007). Intentions are strongly related to behavior and will be an important mediator predicting behavior change.
Sexual activity questions 7.1–7.29)	Sexual activity is an important outcome for the evaluation and sexual activity at baseline is a powerful predictor of later outcomes. Having data at baseline increases the precision of our estimates of impacts on sexual activity at follow-up.
Drug and alcohol use (questions 5.1–5.12)	There is a substantial body of literature linking various high-risk behaviors of youth, particularly drug and alcohol use, sexual intercourse, and risky sexual behavior. The effectiveness of various program strategies is expected to differ for youth who are and are not experimenting with or using drugs and alcohol (Tapert et al., 2001; Li et al., 2001; Boyer et al., 1999; Fergusson and Lynskey, 1996; Sen, 2002; Dermen et al., 1998; Santelli et al., 2001.)

*(OMB No. 0970-0360 Expiration date 7/31/2013)

12. Estimates of Annualized Burden Hours and Costs

The estimated annualized burden (Table 3) is based on the total number of target respondents multiplied by the number of times they will be administered the forty-five minute, (this time is based on the previous 30 minute PPA approved survey with 15 additional minutes to include the new measures), PPA survey (Attachment A). Estimated annualized burden costs to youth are based on the Federal minimum wage, and are displayed in Table 4.

Table 3: Estimated Annualized Burden

Type of Respondent	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
15-19 year olds	Evaluation of Adolescent Pregnancy Prevention Approaches Household (Modified PPA) Survey (OMB No.0970-036 )	6,000	1	.75	4,500
Total		6,000	–	–	4,500

Table 4: Estimated Annualized Burden Costs

Type of Respondent	No. of respondents	No. of Responses per Respondent	Avg. Burden per Response (in hours)	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
15-19 Year olds	6,000	1	.75	4,500	$7.25	$32,625
Total	6,000					$32,,625

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

The CDC does not perceive that any additional cost will be incurred by the respondents.

14. Annualized Cost to the Government

The CDC TPP evaluation estimated cost to the Government is $1,751,135. The annual cost $875,567.5 includes the contractor that will be conducting and analyzing the evaluation. The projected cost of the Government senior staff (GS-14) at 30 percent time is $30,310.50. The total cost to the Government is estimated to be $1,781,445.5.

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

After the completion of wave 1 data collection assuming OMB clearance, ICF will complete descriptive analyses of the data in both the target and comparison communities summarizing the characteristics of the respondent youth. Subgroup analyses will be performed to assess potential differences among identified groups on youth descriptive variables (e.g., across racial and ethnic groups). These analyses will be conducted using cross-tabulation procedures (e.g., chi-square) with categorical variables and between group procedures (e.g., ANOVA and t-tests) with variables that are continuous. Analyses will also be reported with and interpreted in the context of information gained from the in-depth evaluation components including data about program design, program implementation, and other community prevention resources.

In addition, a comparison of youth self-reported pregnancies and births to those reported in existing vital records datasets for the same time period will be examined. This information will aid the interpretation of findings based on the data collected. Analysis will also be conducted across the target and comparison communities to determine whether baseline differences exist between the communities. If baseline differences go undetected, differences may be attributable to the intervention rather than to differences that existed prior to the onset of the intervention. If differences are detected, they can be used to inform the follow-up data collection strategy and can be statistically controlled in any future outcomes analyses.

Two proposals will be submitted to professional meetings based on the findings. Below are potential venues for conference presentation:

• American Public Health Associations Annual Meetings

• Maternal and Child Health Epidemiology (MCH EPI) Conference

• Association of Maternal and Child Health Programs Family Voices

Conferences

• Society for Adolescent Health and Medicine Annual Meetings

• American School Health Associations Annual School Health

Conference

• Healthy Teen Network Annual Conferences

• American Evaluation Association Annual Conferences

Table 5: Data Collection Time Schedule

Activity	Time Schedule
Train Data Collectors	1 month after OMB approval
Beginning Data Collection	1-3 months after OMB approval
Complete Data Collection	4-6 months after OMB approval
Validation	7-8 months after OMB approval
Analysis	9-10 months after OMB approval
Final report	12 months after OMB approval

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certifications.

1 Yancey, AK; Ortega, A; Kumanyika, S. (2006). Effective Recruitment and Retention of Minority Research Participants. Annual Review Public Health, 27:1-28.