(OMB-final report-september7rev

Contract No.:
233-02-0086
MPR Reference No.: 6247-400

Supporting Justification
for OMB Clearance
of Data Collection
Instruments for the
Head Start Oral Health
Initiative Evaluation
September 7, 2006

Diane Paulsell
Laura Hawkinson
Beth Zimmerman
Anne Hopewell
Sandra Silva

Submitted to:
Administration for Children and Families
Office of Research Planning and Evaluation
Seventh Floor West 370
L’Enfant Promenade
Washington, DC 20447
Project Officer:
Laura Hoard

Submitted by:
Mathematica Policy Research, Inc.
P.O. Box 2393
Princeton, NJ 08543-2393
Telephone: (609) 799-3535
Facsimile: (609) 799-0005
Project Director:
Diane Paulsell

CONTENTS

Chapter
I

Page
JUSTIFICATION............................................................................................................1
A. CIRCUMSTANCES NECESSITATING DATA COLLECTION..........................1
1.
2.

Telephone Interviews and Site Visits ..............................................................3
Program Recordkeeping System......................................................................4

B. HOW, BY WHOM, AND FOR WHAT PURPOSE INFORMATION IS TO
BE USED .................................................................................................................5
C. USE OF AUTOMATED, ELECTRONIC, MECHANICAL, AND OTHER
TECHNOLOGICAL COLLECTION TECHNIQUES............................................7
D. EFFORTS TO AVOID DUPLICATION OF EFFORT ..........................................8
E. SENSITIVITY TO BURDEN OF SMALL ENTITIES ..........................................9
F. CONSEQUENCES TO FEDERAL PROGRAM OR POLICY ACTIVITIES
IF THE COLLECTION IS NOT CONDUCTED OR IS CONDUCTED
LESS FREQUENTLY THAN PROPOSED ...........................................................9
G. SPECIAL CIRCUMSTANCES.............................................................................10
H. FEDERAL REGISTER ANNOUNCEMENT AND CONSULTATION .............10
1.
2.

Federal Register Announcement....................................................................10
Consultation ...................................................................................................10

I.

PAYMENTS OF GIFTS TO RESPONDENTS ....................................................10

J.

CONFIDENTIALITY OF THE DATA.................................................................11

K. ADDITIONAL JUSTIFICATION FOR SENSITIVE QUESTIONS ...................11
L. ESTIMATES OF HOUR BURDEN OF THE COLLECTION OF
INFORMATION....................................................................................................12
M. ESTIMATE OF TOTAL ANNUAL COSTS AND BURDEN TO
RESPONDENTS OR RECORDKEEPERS ..........................................................12
N. ESTIMATES OF ANNUALIZED COSTS TO THE FEDERAL
GOVERNMENT....................................................................................................14

ii

CONTENTS (continued)

Chapter

Page
O. REASONS FOR PROGRAM CHANGES OR ADJUSTMENTS ........................14
P. PLANS FOR TABULATION AND PUBLICATION AND SCHEDULE
FOR THE PROJECT .............................................................................................14
1.
2.

Publication Plans............................................................................................14
Tabulation Plans.............................................................................................15

Q. APPROVAL NOT TO DISPLAY THE EXPIRATION DATA FOR OMB
APPROVAL ..........................................................................................................16
R. EXCEPTION TO THE CERTIFICATION STATEMENT ..................................16
II

COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS…17
A. RESPONDENT UNIVERSE AND SAMPLING METHODS….. ...................... 17
B. STATISTICAL METHODS FOR SAMPLE SELECTION AND DEGREE
OFACCURACY NEEDED ...................................................................................19
C. METHODS TO MAXIMIZE RESPONSE RATE AND TO DEAL WITH
NONRESPONSE...................................................................................................19
1.
2.

Telephone Interviews And Site Visits ...........................................................19
Program Recordkeeping System....................................................................20

D. TEST OF PROCEDURES AND METHODS TO BE UNDERTAKEN ..............21
E. INDIVIDUALS CONSULTED ON THE STATISTICAL ASPECTS OF
THE DESIGN ........................................................................................................21
REFERENCES..............................................................................................................22
APPENDIX A: INTERVIEW GUIDES ....................................................................A.1
APPENDIX B: PROGRAM RECORDKEEPING SYSTEM:
PROPOSED DATA ELEMENTS ....................................................B.1

iii

TABLES

Table

Page

I.1

PROGRAM RECORDKEEPING SYSTEM FOR THE HEAD START ORAL
HEALTH INITIATIVE EVALUATION: PROPOSED SCREENS AND
DATA CATEGORIES...................................................................................................6

I.2

ESTIMATED RESPONSE BURDEN FOR RESPONDENTS FOR THE
HEAD START ORAL HEALTH INITIATIVE EVALUATION...............................13

iv

I. JUSTIFICATION

A. CIRCUMSTANCES NECESSITATING DATA COLLECTION
Since the publication of Oral Health in America: A Report of the Surgeon General (2000)
and its companion document, A National Call to Action to Promote Oral Health (2003), national
attention to the unmet oral health needs of many of the nation’s children and families has
increased significantly.

These reports identify dental caries as the most prevalent chronic

childhood disease. They also document the disproportionate burden of this disease on lowincome populations—children living in poverty suffer twice as many dental caries as their more
wealthy peers. The surgeon general also documented that chronic oral disease in poor children is
disproportionately more likely to go untreated, because their families commonly lack insurance
or access to dental providers. While more than 51 million school hours were lost to dental illness
overall in 2000, poor children lost 12 times more school days than their middle-class
counterparts (U.S. Department of Health and Human Services 2003). Untreated dental disease
can impede children’s ability to eat, speak, and learn and often has a lifelong negative impact on
overall health (U.S. Department of Health and Human Services 2003).
Head Start program data reflect the magnitude of the problem for Head Start children. In the
2003–2004 program year, 78 percent of all Head Start children had a dental examination within
90 days of enrollment to assess oral health status, but less than 80 percent of those needing care
were able to access oral health treatment (Hamm and Ewen 2005).1 Among children enrolled in

1

The Head Start Program Performance Standards require that a health care professional determine within 90
days of enrollment whether children are up-to-date on a schedule of age-appropriate preventive dental care,
following the recommendations of the Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) program.
Most states, however, do not have a specific schedule for dental services as part of the EPSDT. The American
Association of Pediatric Dentists, the American Association of Pediatrics, and the American Dental Association
recommend that children have a dental exam by age 1. Thus, despite the pressing need for oral health screening,

1

Early Head Start, only 69 percent had access to continuous dental care in 2004. Only 30 percent
had received a professional dental exam in the previous year; 61 percent had received a dental
exam as part of a well-baby checkup (Hamm and Ewen 2006).
In 2006, the Office of Head Start invested $2 million in grants to 52 Head Start, Early Head
Start, and Migrant/Seasonal Head Start programs to implement the Head Start Oral Health
Initiative.2

Grantees will operate the initiative for four years.

The initiative provides an

important opportunity for grantees to draw on their community partnerships and lessons learned
from previous efforts, in order to develop and test the implementation of innovative service
delivery models to improve the oral health of Head Start children and families. By funding a
diverse group of 52 Head Start programs across the country, the Office of Head Start has the
potential to make a significant contribution toward improving the oral health care delivery
systems that serve Head Start families and other low-income populations.
To ensure consistent, systematic collection and analysis of data on the initiative’s
implementation, the Administration for Children and Families contracted with Mathematica
Policy Research, Inc. (MPR), and its subcontractor, Health Systems Research, Inc. (HSR), to
conduct a two-year evaluation of the Oral Health Initiative. The purposes of the evaluation are
to document grantees’ implementation experiences and challenges, to identify promising models
and service delivery strategies, to assess the feasibility of replication or expansion of the models
in other programs and communities, and to disseminate information about lessons learned to the
broader Head Start community. Data collection activities will focus on learning about program

(continued)
diagnosis, and treatment, programs need more specific guidance about the schedule on which these services should
be provided to children ages birth to 5.
2

Throughout this report, references to Head Start programs and families include Head Start, Early Head Start,
and Migrant/Seasonal Head Start programs and families unless otherwise noted.

2

operations and service delivery strategies, rather than on assessing the initiative’s effects on the
oral health outcomes of Head Start children and families. The evaluation will collect and
analyze information from three main sources: (1) telephone interviews with program directors
from all 52 Oral Health Initiative grantees, (2) site visits to a subset of 16 grantees and (3) a
recordkeeping system designed for use by all grantees. This submission requests approval to
conduct these three components of the study.
1.

Telephone Interviews and Site Visits
Through telephone interviews with all 52 grantee directors, we will collect a core set of

consistent data about implementation of the Head Start Oral Health Initiative from all grantees
participating in the initiative. We plan to conduct these interviews in late 2006, after we receive
clearance from the Office of Management and Budget (OMB). We expect each interview to last
approximately 1.5 hours.

In preparation for each interview, interviewers will review each

grantee’s application and abstract information about the grantee’s proposed program design,
staffing, and plans for service delivery under the Oral Health Initiative. We will develop a
standard format for writing up notes from the telephone interviews that will serve as a basis for
brief site profiles to be included in an interim report.
In Year Two, we will conduct site visits to 16 of the 52 grantees (site selection is described
later, in Section II.A) Each visit will last approximately 1.5 days. We expect that staffing
configurations, as well as the number of community partners and their roles in supporting the
initiative, will vary across grantees. Therefore, it is likely that the number and type of staff and
partners we interview will also vary. Nevertheless, we have identified three types of respondents
we expect to interview: (1) grantee directors; (2) key staff who work on the Oral Health
Initiative, such as coordinators, oral health advocates, and family service workers; and (3) key
staff from each grantee’s primary community partners for the initiative. In addition to interviews
3

with these individuals, site visitors will conduct a focus group with parents at each site. We
expect that each focus group will have 10 to 12 participants. To conduct the site visit activities,
we will use semistructured discussion guides for each type of informant we talk to on site
(see Appendix A).
We will take several steps to ensure consistent, high-quality data collection across grantees.
Before conducting the telephone interviews and the site visits, we will provide comprehensive
training for the interview and site visit teams to review the study’s objectives, the research
design, and the data collection procedures. After we conduct an initial set of interviews and site
visits, we will reconvene our team to debrief, discuss any issues that have come up, and ensure
that both MPR and HSR staff are following consistent procedures. In addition, senior team
members will review and provide feedback on notes from initial interviews and site visits; we
will also conduct cross-organizational reviews of interview and site visit notes to ensure
consistency across MPR and HSR staff.
2.

Program Recordkeeping System
A program recordkeeping system designed specifically for the Head Start Oral Health

Initiative will be an important source of information for the evaluation.

Adding to the

information collected from telephone interviews with all grantees and site visits to a subset of
grantees, the recordkeeping system will facilitate collection of consistent quantitative data from
all 52 grantees on the characteristics of children and families participating in the initiative, core
services provided to families and children, and participation of community partners in service
provision. The system will provide researchers with a consistent set of information about
enrollment in the Oral Health Initiative and services provided, across all grantees, regardless of
how these services vary across programs.

4

Our proposed design for the recordkeeping system has four data input screens. Table I.1
presents the types of information we propose to collect on each screen; Appendix B presents a
detailed proposal for the data elements to be included on each screen. The first screen is
designed to collect background information on families and children receiving services through
the initiative, primarily demographic and other information that Head Start programs typically
collect during the application and enrollment process. It will be entered for the child and family
when the family begins receiving services through the Oral Health Initiative. Background
information on families and children will need to be entered only once, although there may be an
occasional need to update this information (for example, if a disability is identified for a child
after the family enrolls in the initiative).

The second screen will capture information on

grantees’ community partners that provide services through the initiative, including the type of
partner and whether the partner has a formal partnership agreement in place with the grantee.
Like information on children and families, information on partners will need to be entered only
once per partner.
The third screen will capture data on services provided to children and pregnant women
through the Oral Health Initiative. To maintain a complete record of services provided, staff
members must make new entries to this screen throughout the data collection period. The fourth
screen will be used to record program-level data on oral health education services provided to
children and families (such as parent education workshops, one-on-one counseling on oral health
topics during home visits, and provision of oral hygiene supplies).
B. HOW, BY WHOM, AND FOR WHAT PURPOSE INFORMATION IS TO BE USED
This study will inform the Office of Research, Planning, and Evaluation, Administration on
Children and Families in the U.S. Department of Health and Human Services, the Office of Head

5

TABLE I.1
PROGRAM RECORDKEEPING SYSTEM FOR THE HEAD START ORAL HEALTH
INITIATIVE EVALUATION: PROPOSED SCREENS AND DATA CATEGORIES
Recordkeeping System Screen

Data Categories

Child and Family Characteristics

Demographic information for child/pregnant woman
Characteristics of primary caregiver (parent)
Dental insurance coverage
Enrollment date
Whether child/pregnant woman has ever received a dental
exam

Community Partner Information

Name of community partner
Type of community partner
Whether formal partnership agreement is in place
Whether new or existing partner

Oral Health Services

Date service provided
Type of service (prevention, exam, or treatment)
Location where service provided
Service provider
Support services provided (transportation, translation)
Whether family referred to service by grantee
Whether followup is required
Followup completed
Dental home established, date and provider

Oral Health Educational Services

Oral health education services received by parents
Oral health education services received by children
Oral health education services received by staff
Provision of oral hygiene supplies

6

Start, and Head Start programs across the nation. The information collected for this evaluation
will be used by policymakers and program administrators to help shape future initiatives that aim
to improve the oral health care delivery systems and promote oral health care prevention for
Head Start children and families, as well as other low-income groups.

In addition, the

information will be useful for Head Start program operators and technical assistance providers as
they seek to improve their approaches to providing or arranging oral health services for children
and families enrolled in their programs.
C. USE OF AUTOMATED, ELECTRONIC, MECHANICAL,
TECHNOLOGICAL COLLECTION TECHNIQUES

AND

OTHER

MPR will design a web-based program recordkeeping system to collect information in a
uniform manner across all grantees. We will design the system to be as user-friendly as possible.
To use the system, each grantee will need access to a computer with an internet connection.
Each grantee will be responsible for entering the data for its own Oral Health Initiative. Users
will enter the recordkeeping system through a logon screen by entering a password stored in the
system for each user. Programs will designate their own user names and passwords, which MPR
staff will then program into the system. The system will be able to accommodate multiple user
names and passwords for each site to facilitate more than one user per program.
After logging into the system, users will be shown the main menu of system options. The
menu will contain sections that correspond to the four types of data input screens: (1) child and
family characteristics, (2) community partners, (3) services provided to children and pregnant
women, and (4) oral health education services. It will also contain buttons for three types of data
entry functions—(1) adding new entries, (2) viewing existing entries, and (3) editing existing
entries—as well as a button for generating reports.

Once a user is taken to the screen and a

function selected, a series of fields will guide the user to enter the requested information. To

7

reduce the time required to perform data entry, the entry screens will be designed to use check
boxes and drop-down boxes whenever possible. In addition, the record-keeping system will be a
dynamic one in which certain boxes appear and disappear based on the information selected.
This design will guide users through the system more easily by indicating the relevant data fields
for a specific child/pregnant woman, community partner, or service.
Data stored and transmitted as part of the program recordkeeping system will be kept tightly
secure. Password security will be utilized to make sure that only authorized users of the system
can gain access. The use of Secure Socket Layer (SSL) certificates will ensure that data will be
encrypted as it is sent across the internet. The website collecting the data will be hosted on one
of MPR’s secure web servers accessible through the internet, but the data will not be stored
directly on this web server. Data will be stored in a database residing on MPR’s local area
network that is protected from unauthorized outside access using industry-standard hardware and
software firewall protection.
D. EFFORTS TO AVOID DUPLICATION OF EFFORT

There are few existing sources of information on the program activities and types of services
that will be provided under the Head Start Oral Health Initiative. Because this is a new initiative
that has not been evaluated, there is no existing source that contains the kinds of information that
the evaluation of the Head Start Oral Health Initiative will provide. In particular, there is no
other source of individual-level data on Head Start children’s receipt of oral health services or
information on receipt of oral health education services. The proposed program recordkeeping
system will not include information that Head Start programs are already asked to collect for
other federal reporting requirements, other than minimal demographic information about families
and children enrolled in the initiative and whether children have received oral health
assessments. To avoid duplication of individual-level data collection about child and family
8

demographics, programs will be requested to extract information on children and families from
their current management information systems whenever possible.
E. SENSITIVITY TO BURDEN OF SMALL ENTITIES

The information requested is the minimum required to meet the study objectives. The
burden on the grantees has been minimized as much as possible by designing a recordkeeping
system that uses check boxes and drop-down lists whenever possible to reduce the time required
to complete data entry. In addition, before each site visit, the site visitor will contact the Head
Start program director to explain the purpose of the visit and review possible dates for the visit.
We will provide Head Start directors with alternative dates and allow them to select dates that
are most convenient for program staff. Following this initial contact, we will send the program
director a letter that details what we hope to accomplish during the visit, whom we need to
interview, the approximate amount of time needed for each interview, whom to include in focus
groups, and the amount of time needed for the focus groups.

Moreover, throughout the

evaluation we will consult with a subset of grantees about ways to minimize the burden of our
data collection activities.
F. CONSEQUENCES TO FEDERAL PROGRAM OR POLICY ACTIVITIES IF THE
COLLECTION IS NOT CONDUCTED OR IS CONDUCTED LESS FREQUENTLY
THAN PROPOSED

The data collected in this evaluation are critical to our understanding of how Head Start Oral
Health Initiative grantees implement strategies and services to improve oral health care delivery
systems for Head Start children and families and to promote oral health prevention practices. If
these data were not collected or collected less frequently, we would not be able to describe the
implementation successes and challenges that the grantees experience, how the program models
they develop evolve over time, and the implementation lessons they learn through the initiative

9

during the three-year grant period. Moreover, without these data the Office of Head Start would
not be able to provide guidance to other Head Start programs about how to implement strategies
that show promise for being replicable and sustainable.
G. SPECIAL CIRCUMSTANCES
There are no special circumstances.
H. FEDERAL REGISTER ANNOUNCEMENT AND CONSULTATION
1. Federal Register Announcement
The initial Federal Register announcement was printed on July 5, 2006 in Volume 71, No.
128, pp. 38,168-38,169. The second notice was printed on August 31, 2006 in Volume 71, No.
169, pp. 51830. For additional information, see the OS certification statement.
2. Consultation
Individuals outside ACF who have been consulted on the feasibility of this study and the
availability of data sources are:
• Kimberly Boller, Mathematica Policy Research, Inc.
• Jane E. Steffensen, University of Texas
• Maria Rosa Watson, Epidemiology and Dental Public Health Consultant
• John Rosetti, consultant to the Maternal and Child Health Bureau, DHHS

I. PAYMENTS OF GIFTS TO RESPONDENTS
MPR will pay each parent $20 for participation in the site visit focus groups. This $20
should cover the cost of travel and other expenses (such as child care) incurred to attend the
focus group.

10

J.

CONFIDENTIALITY OF THE DATA

This study is being conducted in accordance with all relevant regulations and requirements,
including the Privacy Act of 1974 (5USC 552a), the Privacy Act Regulations (34 CFR Part 5b),
and the Freedom of Information Act (5 CFR 552) and related regulations (41 CFR Part 1-1, 45
CFR Part 5b, and 40 CFR 44502). As part of the introduction to each interview, site visit
respondents will be told that none of the information they provide will be used for monitoring or
accountability purposes and that the results of the study will be presented in aggregate form only.
The program recordkeeping system manual will also contain an introductory statement to this
effect.
MPR routinely uses the following safeguards to carry out data security, and HSR will
implement these safeguards as well:
• All employees at MPR sign a confidentiality pledge that emphasizes the importance
of confidentiality and describes their obligations.
• Identifying information will be maintained in separate tables in the database, which
are linked to the data entry screens only by sample identification number.
• Access to the file linking sample identification numbers with identifying information
will be limited to a small number of individuals who have a need to know this
information.
• Access to hard-copy documents will be strictly limited. Documents are stored in
locked files and cabinets. Discarded material is shredded.
• Computer files will be protected with passwords, and access will be limited to
specific users. Especially sensitive data are maintained on removable storage devices
that are kept physically secure when not in use.

K. ADDITIONAL JUSTIFICATION FOR SENSITIVE QUESTIONS

We are not collecting any sensitive data. We will ask site visit respondents about the
characteristics and needs of the children and families they serve, the services being providing
through the Head Start Oral Health Initiative, and their experiences implementing the initiative.
Through the program recordkeeping system, we will collect information about the characteristics
11

of families and caregivers, characteristics of community partners, and the services provided
through the initiative. None of these questions is considered sensitive.
L. ESTIMATES OF HOUR BURDEN OF THE COLLECTION OF INFORMATION

Table I.2 provides a summary of the sample size per group of respondents, estimated
response time per respondent, and total response time. We estimate the total respondent burden
for the entire study to be 10,827 hours—843 hours in 2006 and 9,984 hours in 2007. We
estimate the burden of responding to the telephone and site visit interviews and participating in
the focus groups protocols to be 78 hours in 2006 and 416 in 2007. Burden in 2006 is lower
because it only includes program director telephone interviews, while the 2007 estimate includes
the site visits to 16 grantees. We base our time estimates for the site visit activities on our
experience using similar protocols for site visits in the national evaluation of Early Head Start
and the evaluation of the Early Head Start Enhanced Home Visiting Pilot Project. Total burden
for the program recordkeeping system is estimated to be 10,333 hours (765 hours in 2006 and
9,568 hours in 2007). This is based on an estimated data entry burden of five minutes per
enrolled child per month for 1 month in 2006 and 12 months in 2007. We have based our
estimates on experience using a similar program recordkeeping system for the Early Head Start
Enhanced Home Visiting Pilot evaluation.
M. ESTIMATE OF TOTAL ANNUAL COSTS AND BURDEN TO RESPONDENTS OR
RECORDKEEPERS

Pilot sites and caregivers will not incur any costs for participating in evaluation activities.
The burden and cost of collecting data on Head Start Oral Health Initiative enrollment and
services are part of the ongoing programmatic activities that each grantee is expected to carry out
as part of its participation in the initiative. Programs will use existing computers and internet
links to record information in the program recordkeeping system designed and provided by
12

TABLE I.2
ESTIMATED RESPONSE BURDEN FOR RESPONDENTS FOR THE HEAD START ORAL HEALTH
INITIATIVE EVALUATION

Instrument
Director Telephone Interview
Program Recordkeeping System

Number of
Respondents
52

Number of
Responses per
Respondent
1

52

184

Average
Burden per
Response
(Hours)
1.5

a

0.08

TOTAL FOR 2006
Site Visit Protocols
Director Interview
Key Staff Interview
Community Partner Interview
Parent Focus Group
Recordkeeping System

Annual
Burden
(Hours)
78.0

b

765.4
843.4

16
48
80
160
52

1
1
1
1
184a

1.5
1.5
1.0
1.5
1.0b

24.0
72.0
80.0
240.0
9,568.0

TOTAL FOR 2007

9,984.0

TOTAL FOR 2006 AND 2007

10,827.4

a

Average expected number of children and pregnant women to be tracked per grantee, ranging from 40 to 200. We
expect to track approximately 200 children and pregnant women per grantee, except for a few grantees serving
fewer than 200 children.

b

Based on an estimated burden of 5 minutes per child/pregnant woman per month.

13

MPR. In their original grant applications, Head Start programs were asked to budget for data
collection activities to support the national evaluation.
N. ESTIMATES OF ANNUALIZED COSTS TO THE FEDERAL GOVERNMENT

The estimated cost to the federal government through April 2008 of the Head Start Oral
Health Initiative evaluation—including designing and administering the data collection
instruments; collecting, processing, and analyzing the data; and preparing reports summarizing
the results—is $494,564, or $247,282 per year. This estimate is based on MPR’s previous
experience managing data collections of this type.
O. REASONS FOR PROGRAM CHANGES OR ADJUSTMENTS

This is a new data collection.
P. PLANS FOR TABULATION AND PUBLICATION AND SCHEDULE FOR THE
PROJECT
1.

Publication Plans

As part of this data collection, we will produce an interim and a final report. The interim
report, due in July 2007, will focus on findings from the telephone interviews with grantee
directors and the first six months of data recorded in the program recordkeeping system. The
final report, to be completed in March 2008, will summarize findings from all the data collection
activities—including the grantee director telephone interviews, site visits to 16 grantees, and all
data recorded in the program recordkeeping system. To supplement dissemination of these
reports, MPR and HSR staff members will also seek to present their research at several
professional conferences.

With approval from ACF, we will submit our research for

consideration at the annual Early Head Start Birth-To-Three Institute, the Biannual Head Start
Research Conference, and other relevant professional gatherings.

14

2.

Tabulation Plans

a.

Telephone Interviews and Site Visits

Because of the large number of grantee sites in the evaluation, we will use a qualitative
analysis software package, Atlas.ti (Scientific Software Development 1997), to organize and
code the data collected during the site visits.

This software will allow the research team

members to use a structured coding system for organizing and categorizing data, entering the
data into a database according to the coding scheme, and retrieving data that are linked to
primary research questions. After the telephone interview and site visit information is coded,
data can be retrieved from this system on particular research questions across all sites or
individual respondents within sites, as well as by type of respondent (for example, oral health
advocate, program director, or community partner). This approach will facilitate examination of
how grantees vary in their program models, service delivery strategies, community partnerships,
implementation successes and challenges, and other program features.
b. Program Recordkeeping System

To analyze the information collected in the program recordkeeping system, we will conduct
descriptive analyses, such as computing frequencies, means, and distributions of measures for
each of the pilot sites and for groups of sites. For example, we will compute the average number
of clinical treatment services received by children, as well as the range across and within sites.
We will also compute the percentage of children and pregnant women who received different
kinds of services (such as clinical exams or fluoride treatments). Tables will present frequency
distributions and means for particular services across all grantees and for particular groups of
grantees. Data from the recordkeeping system will also be used to form a general description of
families and children at the time of enrollment in the Oral Health Initiative.

15

Q. APPROVAL NOT TO DISPLAY THE EXPIRATION DATA FOR OMB APPROVAL

All study materials will display the OMB expiration date.
R. EXCEPTION TO THE CERTIFICATION STATEMENT

No exceptions to the certification statement are requested.

16

II. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

A. RESPONDENT UNIVERSE AND SAMPLING METHODS

This submission is for data collected from Head Start Oral Health Initiative grantees during
telephone interviews with grantee directors, site visits to a subset of grantees, and a program
recordkeeping system. The telephone interviews (late 2006) will be conducted with directors of
all 52 Head Start programs participating in the initiative. The site visits (fall 2007) will be
conducted at 16 of the 52 grantee sites.
We will work with ACF to determine appropriate criteria for selecting a subsample of
16 pilot sites for the site visits. We expect to select sites that are diverse in terms of ACF region,
urban/rural location, target population, and program characteristics (such as program model and
types and intensity of services provided) deemed important by ACF. In addition, we will use
telephone interview and program recordkeeping system data to assess the effectiveness of
grantees in reaching their target populations and providing oral health preventive, treatment, and
educational services at desired levels of intensity. We will use these data to identify 12 grantees
with strong performance in these areas and 4 low-performing grantees. Site visits to highperforming grantees will help us identify promising outreach and service delivery strategies for
various populations of Head Start children and families. Visits to low-performing grantees will
help us understand the challenges that can hinder programs’ ability to conduct outreach and
deliver services to the target population.
To provide useful information to the Head Start community, the 16 selected grantees will
need to represent the variety of contexts in which Head Start, Early Head Start, and
Migrant/Seasonal Head Start programs operate. Thus, our site selection process will need to
balance the goals of visiting a variety of programs and collecting information on promising
practices relevant to specific hard-to-serve populations and community contexts. To accomplish
17

these goals, we will work with federal staff and consultants to identify subgroups of grantees of
interest, such as those located in rural and urban communities; those serving special populations,
such as Native Americans, migrant farmworkers, and English-language learners; or other subsets
of programs with particular characteristics. The number and types of subgroups, and the number
of grantees within subgroups selected, will need to be carefully considered to ensure we develop
a thorough understanding of implementation. We will need to carefully consider the subgroups
that include high- and low-performing grantees compared to subgroups that include only highperforming grantees. In addition, some grantees may fit into more than one subgroup of interest.
For example, grantees that service migrant farmworkers are also likely to be located in rural
areas.
To select participants for the parent focus group, we will select a random sample of parents
of children entered into the program recordkeeping system. We will select 20 parents, twice as
many as we expect to attend the focus group. Accounting for refusals and no-shows, we expect
that selecting and contacting 20 parents for each group will yield 10 actual participants. We will
ask program directors at the selected grantees to designate appropriate staff to help us recruit
these parents for participation in the focus group.
All 52 pilot sites will implement the program recordkeeping system. Records will be kept
on the children and pregnant women enrolled in the initiative. To reduce the burden on grantees
when Head Start programs plan to target more than 200 children and pregnant women for
services under the initiative, we will work with ACF and these grantees to select a subset of
Head Start services locations to include in program recordkeeping system data collection. We
will collect program recordkeeping system data on all children served in these locations. We
will work with grantees to identify service locations that are likely to be most heavily involved in
the initiative and that represent a diversity of community characteristics and Head Start families.

18

We do not plan to select a random sample of children to track through the recordkeeping
system, for several reasons.

Because the primary goal of the study is to learn about

implementation and the potential for the Oral Health Initiative to improve oral health care service
delivery systems and increase families’ access to the services, we plan to include service
locations that are likely to be most actively involved in the initiative. These locations are more
likely to yield useful information about the levels of service delivery that can be achieved using
different service delivery strategies, compared to service locations that are not as actively
involved. In addition, we expect that tracking a random sample of children across all service
locations will be more burdensome for grantees than concentrating data collection in a subset of
locations, because fewer staff will need to be involved in conducting data entry.
B. STATISTICAL METHODS FOR SAMPLE SELECTION AND DEGREE OF
ACCURACY NEEDED

The Head Start programs that will be part of this evaluation are all the grantees of the Head
Start Oral Health Initiative. The programs will be different in their geographic context, target
populations, program models, service delivery strategies, and implementation experiences.
Consequently, the collected data will be analyzed separately for, and, if possible, across,
different program subgroups. While we will use the information selected to identify potentially
promising practices for providing oral health services, we will not generalize the evaluation’s
findings beyond the grantee sites.
C. METHODS TO MAXIMIZE RESPONSE RATE AND TO DEAL WITH
NONRESPONSE
1.

Telephone Interviews and Site Visits

We expect that all the selected grantees will agree to participate in the telephone interviews
and site visits. Our past experience with other evaluations of Head Start initiatives indicates that

19

participation rates are typically close to 100 percent of selected programs. Members of the
research team have already presented information on the evaluation to grantees during an ACFhosted grantee kickoff meeting. To help ensure high participation, we will coordinate with the
programs to determine convenient dates for telephone interviews and visits. All grantees will be
mailed or faxed materials in advance explaining the purpose of the study and the main topics to
be discussed during the interviews. In addition, during the site visits, site visitors will refine the
questions so that they are applicable to the program and the role of the respondents being
interviewed, making it easier for staff to respond.
Recruitment of participants for the parent focus groups will require close cooperation
between the research team and grantee staff. To select participants for the parent focus group,
we will choose a random sample of parents of children entered into the program recordkeeping
system. We will select 20 parents, twice as many as we expect to attend the focus group (to
account for refusals and no-shows). We will ask program directors at the selected grantees to
designate appropriate staff to help us recruit these parents for participation in the focus group.
We will discuss the scheduling of the focus groups with site staff. In many sites, we expect to
conduct the parent focus groups in the evening to accommodate parents’ work schedules. In
addition, as stated in Section I.I, we will pay each parent who participates in a focus group $20.
2.

Program Recordkeeping System

We expect that all the grantees will enter information into the program recordkeeping
system. To help ensure this, MPR will provide technical assistance to help grantee staff enter the
required information with minimal difficulties. We will provide three main types of support:
(1) a user’s manual, (2) a system orientation through conference calls with the grantees, and
(3) ongoing technical assistance available from MPR staff. To maintain the quality of the data,
we will ask pilot sites to enter information at least monthly. To ensure that procedures are
20

implemented correctly, we will monitor data entry closely during the initial months of the data
collection period, and then monthly after that. Because the data will be collected in a central
database automatically, in real time, we will be able to monitor the status of data entry for
particular grantees at any time. Problems with quality and completeness of the data can be
identified quickly. When we identify problems, we will contact the grantee and work with them
to resolve the problem.
D. TEST OF PROCEDURES AND METHODS TO BE UNDERTAKEN

The telephone and site visit interview guides draw heavily on protocols that were developed
for site visits to Head Start and Early Head Start programs for other studies, including the
National Evaluation of Early Head Start (Administration for Children and Families 2002), the
Early Head Start Enhanced Home Visiting Pilot Evaluation (Paulsell et al. 2006), the Early Head
Start Fatherhood Demonstration (Bellotti et al. 2003), and the Head Start National Reporting
System Quality Assurance Study (Paulsell et al. 2004). We made modifications to the protocols
to address the specific objectives of this evaluation and based on experience using them in
previous studies conducted by MPR. Our design of the program recordkeeping system builds on
a similar system developed for the Early Head Start Enhanced Home Visiting Pilot Evaluation
(Paulsell et al. 2006).
E. INDIVIDUALS CONSULTED ON THE STATISTICAL ASPECTS OF THE DESIGN

No individuals beyond the study team were consulted on the statistical aspects of the design

21

REFERENCES

Administration for Children and Families. “Pathways to Quality and Full Implementation in
Early Head Start Programs.” Washington, DC: U.S. Department of Health and Human
Services, 2002.
Bellotti, Jeanne, Cheri Vogel, Andrew Burwick, Charles Nagatoshi, Melissa Ford, Barbara
Schiff, and Welmoet Van Kammen. “Dedicated to Dads: Lessons Learned from the Early
Head Start Fatherhood Demonstration.” Princeton, NJ: Mathematica Policy Research, Inc.,
2003.
Hamm, Katie, and Danielle Ewen. “From the Beginning: Early Head Start Children, Families,
Staff and Programs in 2004.” CLASP Head Start Policy Brief no. 7. Washington, DC:
Center for Law and Social Policy, March 2006.
Hamm, Katie, and Danielle Ewen. “Still Going Strong: Head Start Children, Families, Staff, and
Programs in 2004.” CLASP Policy Brief no. 05-50. Washington, DC: Center for Law and
Social Policy, November 2005.
Paulsell, Diane, Debra Mekos, Patricia Del Grosso, Patti Banghart, and Renee Nogales. “The
Enhanced Home Visiting Pilot Project: How Early Head Start Programs Are Reaching Out
to Kith and Kin Caregivers.” Final interim report submitted to the U.S. Department of
Health and Human Services, Head Start Bureau. Princeton, NJ: Mathematica Policy
Research, Inc., January 12, 2006.
Paulsell, Diane, Linda Rosenberg, Renee Nogales, Charles Nagatoshi, Susan Sprachman, Louisa
Tarullo, and John Love. “Meeting the Challenge: How Head Start Programs Implemented
the National Reporting System.” Report submitted to the U.S. Department of Health and
Human Services, Head Start Bureau. Princeton, NJ: Mathematica Policy Research, Inc.,
December 2004.
Scientific Software Development. Atlas.ti: Visual Qualitative Data Analysis Management Model
Building in Education Research and Business. Berlin, Germany: Scientific Software
Development, 1997.
U.S. Department of Health and Human Services. A National Call to Action to Promote Oral
Health. NIH Publication no. 03-5303. Rockville, MD: DHHS, Public Health Service,
National Institutes of Health, National Institute of Dental and Craniofacial Research, spring
2003.
U.S. Department of Health and Human Services. Oral Health in America: A Report of the
Surgeon General. Rockville, MD: DHHS, National Institute of Dental and Craniofacial
Research, National Institutes of Health, 2000.

22

APPENDIX A
INTERVIEW GUIDES

Head Start Oral Health Initiative
Director Telephone Interview Protocol

INTRODUCTION (2 minutes)
My name is ___________________ and I work for [MATHEMATICA POLICY
RESEARCH/HEALTH SYSTEMS RESEARCH], an independent research firm. As you know,
we are conducting a study for the Administration on Children and Families about Head Start
agencies’ experiences implementing the Head Start Oral Health Initiative. Findings from the
study will be helpful to other Head Start agencies implementing similar initiatives. Thank you
for agreeing to participate in this telephone interview.
I would like you to feel comfortable giving your opinions and impressions. The information
we gather will be used to write a report for the Administration for Children and Families about
programs’ experiences implementing the Head Start Oral Health Initiative, including successes,
challenges, and lessons learned by grantees. Our report will describe the experiences and
viewpoints expressed by staff across grantees, but specific comments will not be attributed to
specific individuals or programs. No one will be quoted by name. We will also use the
information to create a profile for your site, and we will give you an opportunity to review and
comment on a draft version of the profile before we finalize it.

Do you have any questions before we get started?
About You
To begin, I’d like to learn about your role in the Oral Health Initiative.
1. What is your official job title? What are your primary responsibilities?
2. How long have you worked for [GRANTEE]?
3. How long have you held your current position? What other positions have you held
within the agency?
GRANTEE CHARACTERISTICS (10 minutes)
Now I’d like to confirm some information about your agency. To begin, your agency’s
address and phone number is: CONFIRM FROM PROPOSAL

A.2

4. Who is your main contact for the Oral Health Initiative? What is that person’s job
title? Email address?
5. What are the main programs (other than Head Start) that your agency
operates/services you provide?
6. What is the size of your organization? How many families does your agency serve
annually? Approximately how many staff do you have?
7. I’d also like to confirm that your agency operates Head Start, Early Head Start,
and/or Migrant/Seasonal Head Start programs. CONFIRM FROM PROPOSAL.
8. How many Head Start, Early Head Start, and/or Migrant/Seasonal Head Start
families is your agency funded to serve annually? CONFIRM FROM PROPOSAL.
9. What Head Start service options do you offer to families – center-based services,
home-based services, combination, multiple options? How many centers does your
program operate? CONFIRM FROM PROPOSAL.
10. How long has your agency provided services in [COMMUNITY]? How long has
your agency operated the Head Start, Early Head Start, and/or Migrant/Seasonal
Head Start program(s)?
11. What is the operating schedule (program year) for your Head Start, Early Head Start
and/or Migrant/Seasonal Head Start program(s)?
COMMUNITY AND FAMILY CHARACTERISTICS (10 minutes)
Let’s talk about your community and the characteristics of families and children targeted for the
Oral Health Initiative.
12. What is your program’s geographic service area? Is it primarily urban, rural,
suburban, or a mix? Are you operating the Oral Health Initiative in the entire service
area, or only a portion of it? If so, what part and why? If implementing in multiple
locations, does implementation differ across sites, and if so, how?
13. Can you please describe the Head Start families you are serving through the Oral
Health Initiative? What languages do they speak? What are their ethnic and cultural
backgrounds?
14. What is the availability of other services for children and families, such as medical
care, transportation, and social services?
15. What are families’ main barriers to accessing oral health care? What is the
availability of oral health care providers in the community? General dentists?
Pediatric dentists? Other providers? Do oral health care providers in your
community accept Medicaid? Are they willing to serve young children? Are
providers available who speak the languages spoken by Head Start families?

A.3

16. In general, what are families’ cultural norms and practices related to oral health care?
Oral health care beliefs and practices for young children? PROBE ONLY IF
NEEDED: What is the prevalence of practices that threaten oral health, such as
putting babies to bed with bottles, using pacifiers past age 3, giving children
sweetened drinks, other?
GRANTEE GOALS, OBJECTIVES, AND KEY COMPONENTS (5 minutes)
At this point, I’d like to begin talking specifically about the Health Start Oral Health Initiative.
To start, let’s talk about how your agency designed the initiative and decided which services to
offer.
17. Why did you decide to apply for an Oral Health Initiative grant?
18. What are your program’s goals and objectives for the Oral Health Initiative? Have
these goals and objectives changed since you began implementation? If so, how
have they changed and why?
19. What are the key components of your Oral Health Initiative?
20. How many children are you planning to serve, and what ages? Will your program
provide services to pregnant women? Other family members? CONFIRM FROM
PROPOSAL. How did you decide which children and families to target for Oral
Health Initiative services?
21. What is your annual budget for the Oral Health Initiative? CONFIRM FROM
PROPOSAL. Approximately what proportion of funds do you spend on staff
salaries, direct purchase of dental services, oral hygiene supplies, and other types of
expenses?
DESIGN PROCESS (10 minutes)
22. How did your program identify goals and objectives for the Oral Health Initiative
and decide which services to provide?
23. Who was involved in designing the Oral Health Initiative? Was your health advisory
committee, policy council, or another advisory body involved in the planning
process? If so, who are the members of this committee and what is its role? Are
there any dental representatives on your health advisory committee? Did you work
with community oral health coalitions or other community groups in planning your
grant? Regional office and/or TA staff, regional oral health consultants?
24. What other resources did you use for designing the initiative? For example, did you
draw on any state plans related to oral health (state oral health plans, plans resulting
from a Head Start oral health forum)?
25. Did you do a community needs assessment or use data from one that was already
done? If so, how did you use the needs assessment data?

A.4

26. In designing the Oral Health Initiative, did you build on previously existing oral
health activities in your program, or did you design a new approach? If you built on
previous activities, please tell me about these activities. If you designed a new
approach, how did you design it?
27. What community and family characteristics were most important in your decisions
about the design of your Oral Health Initiative? How did you tailor your approach to
fit the particular circumstances of your families and community?
STAFFING STRUCTURE AND TRAINING (10 minutes)
Now I’d like to learn about how your Oral Health Initiative is staffed.
28. Approximately how much time do you spend on the initiative on a weekly or
monthly basis?
29. How many staff work on the oral health initiative? What are their job titles and main
duties related to the initiative? What are their qualifications?
30. Are all planned positions for the Oral Health Initiative filled? If not, why not, and
what plans do you have to fill the positions? If positions are filled, how soon after
receiving Oral Health Initiative funding were you able to fill them?
31. How did you decide how to staff the initiative? Did you hire new staff, reassign
existing staff, or both? Why did you take this approach?
32. How well is the staffing structure for the Oral Health Initiative working out so far?
Do you have sufficient staff resources to operate the initiative as planned?
33. Did staff receive any special training in preparation for their work on the Oral Health
Initiative? If so, which staff received training? Please describe the training they
received. Did they receive training on how to conduct visual inspection of teeth and
mouth to identify children who need follow up care? How to provide oral health
education to parents and children? Cultural issues related to oral health? Other
topics?
34. Have staff received any training for the initiative since the initiative started? Do you
have future training plans for Oral Health Initiative staff? IF NOT ALREADY
MENTIONED IN #33 ABOVE: Any plans to provide training on conducting visual
inspection of teeth and mouth to identify children who need follow up care? How to
provide oral health education to parents and children? Cultural issues related to oral
health? Other topics?
35. Do you have plans in place to train new staff hired in the future due to turnover in
Oral Health Initiative staff? Please describe.
36. Has your program received any training or technical assistance from the Head Start
T/TA system, the regional oral health consultants, or other sources to support your
work on the Oral Health Initiative? Have your staff attended any regional cluster
trainings on oral health? If so, was the training helpful?

A.5

COMMUNITY PARTNERS (10 minutes)
37. How many and what types of organizations have you partnered with to provide
services through the Oral Health Initiative? What was your rationale for recruiting
them? Are there other partners that you still need to pursue? If so, please describe
them and their potential role in the Oral Health Initiative?
38. What strategies did you use to identify and reach out to these partners?
39. Do you have formal partnership agreements with these partners? If so, what is
included in the agreements?
40. What are the partners’ roles in the Oral Health Initiative? What services do they
provide to Head Start children and families?
41. Do you make referrals to community partners for services? If so, do you receive
information from them about treatment and needed follow up? How do these referral
systems work?
42. Have you provided any training to community partners or other oral health care
providers to improve their ability to address oral health issues for young children?
To improve their cultural competence for working with Head Start families? Was
this helpful to community partners? Did it increase their receptivity to serving Head
Start children and families?
43. How are the partnerships going so far?
partnerships, and what has been challenging?

What has worked well about the

SERVICE DELIVERY (25 minutes)
Now I’d like to learn about the services you provide to children and families through the Oral
Health Initiative. I’ll start with some questions about oral health risk assessments and exams,
and then ask about clinical preventive and treatment services.
Risk Assessment and Clinical Services
44. Does your program conduct or arrange for routine oral health assessments using
clinical or other means (such as clinical assessments, parent questionnaires,
assessment of medical history, assessment of demographic risk factors)? Who
conducts these assessments (for example, dentists, dental hygienists, nurses, health
coordinators, others)?
45. Does your program use a formal oral health risk assessment tool? If so, which tools
do you use and why did you select them?
46. How does your program use the results of the risk assessments? For example, are
oral health care providers able to use the assessment results to make a diagnosis or
development a treatment plan based on this assessment?
47. What types of other clinical preventive services do you provide through the Oral
Health Initiative? For example, do you provide cleanings, sealants, fluoride
A.6

treatments, or other preventive services? Which services are provided by your
program and which are provided by partners? Where and by whom are the services
provided? How are the costs of these services covered (for example, program grant
funds, insurance reimbursement, donated by provider)?
48. What types of clinical treatment services do you provide through the Oral Health
Initiative? Which services are provided by your program and which are provided by
partners? Where and by whom are the services provided? How are the costs of
these services covered (for example, program grant funds, insurance reimbursement,
donated by provider)?
49. Which clinical services do you provide to children, pregnant women, and other
family members? Does this differ for different populations of children and families?
Services to Support Access to Dental Services
50. Do you have referral systems in place for helping families access needed clinical
services? If so, how do these work?
51. Do you keep track of treatment outcomes and needed follow up services? If so, how
do you do this?
52. Do you provide services to help families access needed clinical services, such as help
them make appointments, provide transportation, or provide translation services? If
so, who provides these services?
53. What is your definition of a dental home? Does your program help families establish
dental homes for their children? If so, how do you do this?
Oral Health Education
54. Do you provide education and skills-building activities to parents about oral health
promotion? If yes, please tell me about these services and the main educational
messages you aim to deliver. Who provides this education? How and where are the
educational messages delivered (for example, during parent meetings, home visits, or
by distributing written materials)? Are parents instructed on how to do visual
inspections of children’s teeth using such techniques as “Lift the Lip”?
55. Do you provide education and skills-building activities on oral health promotion
specifically to pregnant women? If so, who provides this education, and where? Are
the educational messages different from those provided to other Head Start parents?
If so, how? What happens after the baby is born? How do the educational messages
change?
56. Do you provide oral health education and skill building activities to children? Who
provides this education, and where is it provided? How are the educational messages
delivered (for example, classroom activities, home visit)?
57. Do you use a curriculum to provide oral health education to children and families? If
so, what curriculum do you use and why did you choose it? Have you made any

A.7

adjustments to the curriculum? If so, why? What feedback have you received on the
curriculum from teachers, other staff, and families?
58. Do you provide oral hygiene supplies to children and families? If so, what types of
supplies do you provide, and to whom? How do you provide them and how often?
Do parents receive training on how to use the supplies?
59. To what extent have you tailored education and other non-clinical services to the
needs and cultural norms of your target population for the Oral Health Initiative?
Can you please provide some examples?
60. Have you taken steps to expand your Oral Health Initiative to the broader
community? For example, have you participated in community health fairs or other
community education events?
EARLY IMPLEMENTATION EXPERIENCES (10 minutes)
I’d like to wrap up the call by hearing your views on the successes and challenges you’ve
experienced implementing the initiative so far.
61. Is your funding for the initiative sufficient to implement it as planned? Do you have
access to additional funding sources for operating the Oral Health Initiative? If so,
what are these sources and which costs do they cover?
62. Have you applied to any other sources for additional funding to operate the Oral
Health Initiative? If so, where is you application in the review process? How will
you use the funds if you receive an award?
63. At this early stage, how much progress have you made toward meeting your goals
and objectives for the Oral Health Initiative?
64. Since you began implementing the Oral Health Initiative, have you made changes to
your original design? If so, what are the changes and why did you make them?
65. What have been your most important successes so far? What are you most proud of?
66. What are the most significant challenges your program has faced so far?
67. What strategies have you used to address these challenges? How well do you think
these strategies are working?
68. Have you consulted with other Oral Health Initiative grantees about implementation
challenges or other issues? If so, how did this happen—email, phone, facilitated by
Head Start Oral Health Consultant? What issues did you discuss?
69. Is there anything more the Office of Head Start, the regional office, or the Head Start
T/TA network could do to support your work on the Oral Health Initiative?
70. Is there anything else you would like to add before we end the discussion?
Thank you again for participating in the interview.

A.8

Head Start Oral Health Initiative
Director Site Visit Interview Protocol

INTRODUCTION (2 minutes)
Thank you for agreeing to participate in this interview. My name is ___________________
and I work for [MATHEMATICA POLICY RESEARCH/HEALTH SYSTEMS RESEARCH],
an independent research firm. As you know, we are conducting a study for the Administration
on Children and Families about Head Start agencies’ experiences implementing the Head Start
Oral Health Initiative. Findings from the study will be helpful to other Head Start agencies
implementing similar initiatives.
Everything you tell me is confidential. I would like you to feel comfortable giving your
opinions and impressions. The information we gather will be used to write a report for the
Administration for Children and Families about programs’ experiences implementing the Head
Start Oral Health Initiative, including successes, challenges, and lessons learned by grantees.
Our report will describe the experiences and viewpoints expressed by staff across grantees, but
specific comments will not be attributed to specific individuals or programs. No individual staff
member will be quoted by name.
Do you have any questions before we get started?
About You
To begin, I’d like to review your role in the Oral Health Initiative.
1. IF STAFF ARE THE SAME AS THOSE INTERVIEWED IN THE TELEPHONE
INTERVIEW ASK: Have your responsibilities on the Oral Health Initiative changed
since the telephone interview?
2. ASK EACH RESPONDENT ONLY IF STAFF ARE DIFFERENT THAN THOSE
INTERVIEWED PREVIOUSLY BY TELEPHONE: I’d like to learn about your role
in the Oral Health Initiative.
-

What is your official job title, and what are your primary responsibilities?

-

How long have you worked for [GRANTEE]?

-

How long have you held your current position? What other positions have
you held within the agency?

GRANTEE CHARACTERISTICS (5 minutes)
I’d like to update our information about your agency.

A.9

3. Since the telephone interview in [MONTH AND YEAR], have there been changes
in:
-

Your program’s organizational structure

-

The Head Start program options you operate (home-based, center-based
mixed)

-

The size of your program, such as the number of children, pregnant women,
and families served in Head Start, Early Head Start, and/or Migrant/Seasonal
Head Start

-

Your operating schedule

COMMUNITY AND FAMILY CHARACTERISTICS (10 minutes)
Let’s talk about your community and the characteristics of families and children targeted for the
Oral Health Initiative.
4. Have there been any changes to your program’s geographic service area since the
telephone interview? Any changes in the portion of your service area targeted by the
Oral Health Initiative? If so, why have these changes happened?
5. Have there been any changes in the characteristics of the children and families you
are targeting for participation in the Oral Health Initiative? If so, is this because of
changes in your population of Head Start families or because of changes in how you
are targeting Oral Health Initiative services?
6. Have there been any changes since the telephone interview in the availability of oral
health care providers in the community? If so, has availability of providers increased
or decreased, and why has this happened?
7. Have there been changes in the availability of other services, such as health care,
transportation, and other social services? If so, has availability increased or
decreased, and why has this happened?
8. During the telephone interview, you listed the families’ main barriers to oral health
care as [FILL IN FROM TELEPHONE INTERVIEW NOTES]. Have any of these
barriers been eliminated? If so, how? Are there other barriers families face now?
9. Please tell me about families’ cultural norms and practices related to oral health
care? What are their oral health care beliefs and practices for young children?
During the telephone interview, you mentioned the prevalence of the following
practices that threaten oral health [FILL IN FROM TELEPHONE INTERVIEW
NOTES]. Have you seen any change in the prevalence of these practices? Have you
identified other common practices that negatively impact oral health? If so, what are
they?
10. Do children and families targeted for the Oral Health Initiative has access to health
and dental health insurance coverage? If yes, do they have private insurance, or are

A.10

they covered primarily by public insurance programs such as Medicaid? Do dental
care providers in the community accept public insurance coverage?
11. Approximately what proportion of children targeted for the Oral Health Initiative
have a disability or developmental delay? What percentage are English Language
Learners? Does this create additional barriers to accessing oral health care services?
If so, how and why?
GRANTEE GOALS, OBJECTIVES, AND KEY COMPONENTS (5 minutes)
Now I’d like to begin talking specifically about the Health Start Oral Health Initiative. To start,
let’s talk about how your goals and objectives for the initiative.
12. During the telephone interview, you listed [FILL IN FROM TELEPHONE
INTERVIEW NOTES] as your primary goals and objectives for the Oral Health
Initiative. Have these changed since the telephone interview? If so, what changes
have you made and why?
13. During the telephone interview, you listed [FILL IN FROM TELEPHONE
INTERVIEW NOTES] as the key components of your Oral Health Initiative. Have
these changed? If so, how have they changed and why?
14. At the time of the telephone interview, your program was planning to serve [FILL IN
NUMBERS OF INFANTS AND TODDLERS, PRESCHOOLERS, PREGNANT
WOMEN, AND OTHER FAMILY MEMBERS] through the Oral Health Initiative.
Have these targets changed since the telephone interview? If so, how and why?
15. During the telephone interview, you said you planned to make the following changes
to your original design for the Oral Health Initiative [FILL IN FROM TELEPHONE
INTERVIEW NOTES]/did not plan to make changes to your original changes for the
Oral Health Initiative. Since that time, have you made other changes to your design?
If so, what changes and why?
STAFFING STRUCTURE AND TRAINING (15 minutes)
Let’s now turn to how the Oral Health Initiative is staffed.
16. During the telephone interview, you reported spending approximately [FILL IN
TIME FROM TELEPHONE INTERVIEW NOTES] on the Oral Health Initiative.
Has your time on the initiative increased or decreased since then? If so, how much
time do you spend on the initiative now, and why has it changed?
17. During the telephone interview, you reported that the following staff were assigned
to the Oral Health Initiative: [FILL IN FROM TELEPHONE INTERVIEW
NOTES]. Have there been any changes to the staffing structure since that time? If
so, what changes did you make and why? How well have these changes worked out?

A.11

18. Have you had any turnover in staff assigned to the Oral Health Initiative? If yes,
which positions? Have vacant positions been filled? How has staff turnover affected
the design or implementation of your Oral Health Initiative? Has turnover affected
what you have been able to accomplish on the initiative so far?
19. How well is the staffing structure for the Oral Health Initiative working out so far?
Do you have sufficient staff resources to operate the initiative as planned?
20. Based on your experience with the initiative so far, if you could, would you make
changes to the staffing structure? If so, why?
21. IF THE AGENCY HAS HIRED OR ASSIGNED NEW STAFF FOR THE ORAL
HEALTH INITIATIVE SINCE THE TELEPHONE INTERVIEW, ASK: What are
the qualifications of new staff hired/assigned to the Oral Health Initiative? Did they
receive any special training in preparation for their work on the initiative? Did they
receive training on how to conduct visual inspection of teeth and mouth to identify
children who need follow-up care?
22. Since the telephone interview, what additional training have staff received for their
work on the Oral Health Initiative? Who provided it? In your opinion, how helpful
was this training?
23. Do you have plans to provide additional staff training to support the initiative? If so,
what kind of training and why are your planning to provide it?
24. Since the telephone interview, what training or technical assistance has your program
received from the Head Start T/TA system, the regional oral health consultants, or
others to support your work on the Oral Health Initiative? In your opinion, how
helpful was this T/TA? If it was helpful, what made it helpful? If it was not helpful,
do you have recommendations for how to improve it?
25. Do you have additional T/TA needs for the Oral Health Initiative? If so, what kind
of T/TA do you need? Do you have a plan in place to obtain it? If no, why not?
COMMUNITY PARTNERS (15 minutes)
During the telephone interview, you identified the following community partners that work with
you on the Oral Health Initiative: [FILL IN FROM TELEPHONE INTERVIEW NOTES].
26. Have you ended your partnerships with any of these partners since the telephone
interview? If so, which ones and why?
27. Have you formed any new partnerships since the telephone interview? If so, who are
the partners and why did you decide to recruit them?
28. Do you have formal partnership agreements with these partners (both new and
existing)? If so, what is included in the agreements?

A.12

29. What are the partners’ roles in the Oral Health Initiative? What services do they
provide to Head Start children and families? Have these roles changed since the
telephone interview? If so, how have they changed and why?
30. Do you make referrals to community partners for services? If so, do you receive
information from them about treatment and needed followup? How do these referral
systems work?
31. Have you provided training to community partners or other oral health service
providers about providing oral health services to your target population? If yes,
please describe the training you provided. Why did you decide to provide it, and
how helpful was the training?
32. Have your community partnerships been implemented as originally planned? If no,
what has changed and why has it changed?
33. How often do you communicate with the community partners and what form does
the communication take (meetings, phone calls, emails, referrals)? What do you
typically communicate about? How well does communication with partners work?
34. In your opinion, what aspects of your community partnerships have worked well,
and what has been challenging? What strategies have you used to work through the
challenges? How well have these strategies worked?
35. Based on your experience with the Oral Health Initiative, are there other kinds of
partners that would have been helpful? If so, what types of partners and why?
36. If you could, is there anything you would change about your partnerships or
partnership agreements? If so, what would you change and why?
37. What is the potential for sustaining these partnerships after grant funding ends?
38. What advice would you give to other programs about selecting and working with
community partners on a similar oral health initiative? In developing and sustaining
relationships with partners over time?
SERVICE DELIVERY (25 minutes)
Now let’s talk about your experiences with providing or arranging services through the Oral
Health Initiative. We’ll start with a discussion of oral health risk assessments and exams, and
then talk about clinical preventive and treatment services.
Risk Assessment and Clinical Services
39. During the telephone interview, you said that your program did the following to
assess children’s oral health needs: [FILL IN TYPE OF ASSESSMENT, WHO
CONDUCTS IT, AND TOOL USED]. Have you made any changes to the way risk
assessment is conducted, such as the type of assessment, who conducts the

A.13

assessment, or the assessment tool used? If so, what are the changes and why did
you make them?
40. During the telephone interview, you reported the following uses of the risk
assessment results: [FILL IN FROM TELEPHONE INTERVIEWS]. Have you
changed the way that risk assessment results are used? If so, what are they changes
and why did you make them?
41. During the telephone interview, you reported providing [LIST CLINICAL
PREVENTIVE SERVICES FROM TELEPHONE INTERVIEW NOTES.] Has this
changed, and if so how and why? Which services are provided by your program and
which are provided by partners? Where are the services provided?
42. During the telephone interview, you reported providing [LIST CLINICAL
TREATMENT SERVICES FROM TELEPHONE INTERVIEW NOTES.] Has this
changed, and if so how and why? Which services are provided by your program and
which are provided by partners? Where are the services provided?
43. Which clinical services do you provide to children, pregnant women, and other
family members? Has this changed from your original plan for the initiative? If so,
how and why did it change?
44. How receptive have families been to the clinical preventive and treatment services
you provide through the Oral Health Initiative? How has their receptivity changed
over time? If receptivity has improved, what have you done to improve it?
Services to Support Access to Dental Services
45. During the telephone interview, you described the following referral system/no
referral system: [FILL IN FROM TELEPHONE INTERVIEW]. How well has the
referral system worked? Has it changed over time? If so, how has it changed and
why?
46. Do you keep track of treatment outcomes and needed follow-up services? If so, how
do you do this?
47. Do you provide services to help families access needed clinical services, such as help
them make appointments, provide transportation, or provide translation services? If
so, who provides these services?
48. Does your program help families establish dental homes for their children? If so,
how do you do this? Has this changed over time, and if so how? What is your
definition of a dental home?

A.14

Oral Health Education
49. During the telephone interview, you reported providing the following education and
skills-building services to parents about oral health promotion: [FILL IN FROM
TELEPHONE INTERVIEW NOTES]. Have you made any changes to these
activities, such as the content of the educational messages, who delivers these
services, or where they are delivered (for example, during parent meetings, home
visits, or by distributing written materials)? If so, what changes have you made and
why?
50. IF PROGRAM SERVES PREGNANT WOMEN: During the telephone interview,
you reported providing the following education and skills-building services
specifically to pregnant women: [FILL IN FROM TELEPHONE INTERVIEW
NOTES]. Have you made any changes to these activities? If so, what changes have
you made and why?
51. During the telephone interview, you reported providing the following education and
skills-building activities to children: [FILL IN FROM TELEPHONE INTERVIEW
NOTES]. Have you made any changes to these activities, such as the content of the
educational messages, who delivers these services, or how they are delivered (for
example, classroom activities or home visits)? If so, what changes have you made
and why?
52. During the telephone interview, you reporting using [NAME OF
CURRICULUM/NO CURRICULUM] to provide oral health education to children
and families? Has this changed, and if so, what curriculum are you using now and
why did you choose it? Have you changed or adapted portions of the curriculum to
better meet your needs? If so, how have you changed it and why? If you stopped
using a prior curriculum, why did you stop?
53. How well do you think the curriculum is working? Do you have suggestions for
improving it?
54. During the telephone interview, you reported providing [LIST ORAL HYGIENE
SUPPLIES/NO ORAL HYGIENE SUPPLIES FROM TELEPHONE INTERVIEW
NOTES]. Has this changed, and if so how and why? IF PROGRAM PROVIDES
SUPPLIES: What types of supplies are you providing now, and to whom? How do
you provide them and how often? Do parents receive training on how to use the
supplies?
55. To what extent have you tailored education and other non-clinical services to the
needs and cultural norms of your target population for the Oral Health Initiative?
Can you please provide some examples? Have you seen evidence that these
strategies have been effective (for example, increased attendance at educational
events or trainings)?
56. How receptive have families been to the educational services you provide through
the Oral Health Initiative? How has their receptivity changed over time? Do you

A.15

think the education and training has resulted in any changes in families’ oral health
practices? If yes, can you give me some examples?
SUSTAINABILITY (5 minutes)
Now let’s talk about the future of the Oral Health Initiative.
57. What do you think is the future of your Oral Health Initiative? Will you be able to
sustain the services when grant funding ends?
58. What services could your program continue to provide without grant funding, and
which services would you have to discontinue? To what extent are the services
provided to children and families through the Oral Health Initiative reimbursable
through insurance?
59. Will you be able to sustain the referral systems developed for the initiative after
grant funding ends?
60. Will you be able to continue helping children and families find dental homes?
61. What potential funding sources are available to sustain the services after grant
funding ends?
IMPLEMENTATION LESSONS (10 minutes)
I’d like to wrap up the discussion by hearing your views on the successes and challenges of the
Oral Health Initiative and any lessons you’ve learned.
62. Do you have systems in place for monitoring your progress in achieving your goals
and objectives for the Oral Health Initiative? If so, can you please describe these
systems? How helpful have they been? Have you used them to make program
improvements? If yes, can you give me some examples?
63. At this point, how much progress have you made toward meeting your goals and
objectives for the Oral Health Initiative?
64. What has the Oral Health Initiative grant enabled your program to do that you were
not able to do before? For example, add new services? Provide oral health services
to more children? To other family members? To add new community partners? To
establish referral systems? To provide more training to staff, families, or partners?
Other?
65. Have you been able to use the Oral Health Initiative grant to leverage other resources
to support oral health activities in your program and/or community? If yes, how did
you do this and what resources did you leverage?
66. What have been your most important successes so far? What are you most proud of?

A.16

67. What are the most significant challenges your program has faced so far?
68. What strategies have you used to address these challenges? How well do you think
these strategies are working?
69. What are the most important lessons your program has learned about providing oral
health services?
70. What changes, if any, would you like to make to your Oral Health Initiative and
why?
71. What advice would you give to other programs that want to implement a similar
initiative?
72. Is there anything else you would like to add before we end the discussion?
Thank you again for participating in the interview.

A.17

Head Start Oral Health Initiative
Key Staff Site Visit Interview Protocol

INTRODUCTION (5 minutes)
Thank you for agreeing to participate in this interview. My name is ___________________
and I work for [MATHEMATICA POLICY RESEARCH/HEALTH SYSTEMS RESEARCH],
an independent research firm. As you know, we are conducting a study for the Administration
on Children and Families about Head Start agencies’ experiences implementing the Head Start
Oral Health Initiative. Findings from the study will be helpful to other Head Start agencies
implementing similar initiatives.
Everything you tell me is confidential. I would like you to feel comfortable giving your
opinions and impressions. The information we gather will be used to write a report for the
Administration for Children and Families about programs’ experiences implementing the Head
Start Oral Health Initiative, including successes, challenges, and lessons learned by grantees.
Our report will describe the experiences and viewpoints expressed by staff across grantees, but
specific comments will not be attributed to specific individuals or programs. No individual staff
member will be quoted by name.
Do you have any questions before we get started?
About You
To begin, I’d like to learn about your role in the Oral Health Initiative.
1. What is your official job title? What are your primary responsibilities?
2. How long have you worked for [GRANTEE]?
3. How long have you held your current position? What other positions have you held
within the agency?
4. Prior to your current position, have you had experience providing oral health
services?
COMMUNITY AND FAMILY CHARACTERISTICS (10 minutes)
Let’s talk about your community and the characteristics of families and children targeted for the
Oral Health Initiative.

A.18

5. In your opinion, what are the main barriers families in your community face in
accessing oral health care services, and particularly oral health care for young
children?
6. What is the availability of oral health care providers in the community? General
dentists? Pediatric dentists? Other providers? Do oral health care providers in your
community accept Medicaid? Are they willing to serve young children?
7. What is the availability of health care, transportation, and other services for children
and families?
8. Tell me about the families and children you serve through the Oral Health Initiative.
Are you providing services to infants and toddlers, preschoolers, pregnant women,
other family members?
9. In general, what are families’ cultural norms and practices related to oral health care?
Oral health care beliefs and practices for young children? What is the prevalence of
practices that threaten oral health, such as putting babies to bed with bottles, using
pacifiers past age 3, giving children sweetened drinks, other?
STAFF TRAINING (10 minutes)
Tell me about the training you have received for the Oral Health Initiative.
10. Did you receive any orientation or training for the Oral Health Initiative before you
began providing services to children and families? If yes, please tell me about the
training. What topics were covered, and who provided the training? How long did
the training last?
11. Have you received any training for the Oral Health Initiative since you began
working on it? If yes, please tell me about the training. What topics were covered,
and who provided the training? How long did the training last?
12. Have you received any training, either before or after you started working on the
Oral Health Initiative, on how to conduct visual inspection of teeth and mouth to
identify children who need follow-up care?
13. How helpful has this training been for the work you do on the Oral Health Initiative?
Which training was the most helpful, and why? What were the most important
things you learned?
14. Are there other topics related to the Oral Health Initiative on which you would like
more training?
15. If you were to give advice to another program that was trying to start up a similar
oral health initiative, based on your experience, is there any training you think is
essential for staff who will work on the initiative?

A.19

SERVICE DELIVERY (25 minutes)
Now let’s talk about your experiences providing services through the Oral Health Initiative.
16. To start, what are your main goals for the work you do on the Oral Health Initiative?
17. Were you involved in designing the Oral Health Initiative? If so, tell me about the
process.
Risk Assessment and Clinical Services
18. Are you involved in conducting or arranging for routine oral health assessments
using clinical or other means (such as clinical assessments, parent questionnaires,
assessment of medical history, assessment of demographic risk factors)? Who
conducts these assessments (for example, you or other Head Start staff, dentists,
dental hygienists, nurses, health coordinators, others)?
19. Do you use a formal oral health risk assessment tool? If so, which tool do you use
and how was it selected? How well do you think the tool works?
20. How does your program use the results of the risk assessments? For example, are
oral health care providers able to use the assessment results to make a diagnosis or
develop a treatment plan based on the assessment?
21. Do you have suggestions for improving your program’s oral health risk assessment
tools or process?
22. Are you involved in providing or arranging for provision of other clinical preventive
services through the Oral Health Initiative? For example do you provide or arrange
for cleanings, sealants, fluoride treatments, or other preventive services? What is
your role in providing these services? Which services are provided by your program
and which are provided by partners? Where are the services provided? How are the
costs of these services covered (for example, program grant funds, insurance
reimbursement, donated by provider)?
23. Are you involved in providing or arranging for provision of clinical treatment
services through the Oral Health Initiative? What is your role in providing these
services? Which services are provided by your program and which are provided by
partners? Where are the services provided? How are the costs of these services
covered (for example, program grant funds, insurance reimbursement, donated by
provider)?
24. Which clinical services do you provide to children, pregnant women, and other
family members?
25. How receptive have families been to the clinical preventive and treatment services
you provide through the Oral Health Initiative? How has their receptivity changed
over time?

A.20

Services to Support Access to Dental Services
26. Does your initiative have referral systems in place for helping families access needed
clinical services? If so, please tell me about these systems. What is your role in the
referral process? How well does the referral system work? Would you make
changes to it if you could?
27. Does your program keep track of treatment outcomes and needed follow up services?
If so, how this done? What is your role in this process? Is the tracking system
helpful to you in your work with children and families? If so, how is it helpful?
28. Do you provide or arrange for services to help families access needed clinical
services, such as help them make appointments, provide transportation, or provide
translation services? If so, what is your role in this process? Approximately what
proportion of children and families receive these services? Without the services,
would families still be able to access needed dental care?
29. Does your program help families establish dental homes for their children? If so,
what is your role in this process? How easy or difficult is it to help families establish
dental homes? What are the main barriers to establishing dental homes? Which oral
health providers serve as dental homes for the children and families in your
program? What is your definition of a dental home?
Oral Health Education
30. Are you involved in providing education and skills-building activities to parents
about oral health promotion? If so, please tell me about the main educational
messages you deliver and how often you provide these services. How are the
educational messages delivered and where (for example, during parent meetings,
home visits, or by distributing written materials)? Do you instruct parents on how to
do visual inspections of children’s teeth using such techniques as “Lift the Lip”?
31. Are you involved in providing education and skills-building on oral health promotion
specifically to pregnant women? If so, tell me about the education you provide, how
these services are delivered, and how often. Are the educational messages different
from those provided to other Head Start parents? If so, how?
32. Are you involved in providing oral health education and skills-building activities to
children? If so, tell me about these services and how often you provide them. How
are the educational messages delivered (for example, classroom activities, home
visit)?
33. IF STAFF ARE INVOLVED IN EDUCATIONAL ACTIVITIES: Do you use a
curriculum to provide oral health education to children and families? If so, what
curriculum do you use and why did you choose it? How well do you think the
curriculum is working? Is it a good match for the needs of the children and families
you work with? Are there changes you would make to it if you could?

A.21

34. Do you provide oral hygiene supplies to children and families? If so, what types of
supplies do you provide, and to whom? How do you provide them and how often?
Do you provide parents with training on how to use the supplies? Do you think
families use these supplies? What evidence do you have that the supplies are being
used?
35. To what extent have you tailored education and other non-clinical services to the
needs and cultural norms of your target population for the Oral Health Initiative?
Can you please provide some examples?
36. How receptive have families been to the screening and educational services you
provide through the Oral Health Initiative? How receptive are parents, pregnant
women, and children? How effective do you think your approach to education and
training is to changing families’ oral health care practices? What are families doing
differently after participating in these education and skills-building activities? How
has their receptivity changed over time?
WORKING WITH COMMUNITY PARTNERS (10 minutes)
I’d like to shift gears now and talk about your interaction with community partners on the
Oral Health Initiative.
37. Do you work with community partners on the Oral Health Initiative? If yes, what
types of partners do you work with?
38. What is your role in working with community partners? For example, do you make
referrals to them, follow up on treatment outcomes and plans, coordinate services, or
plan joint parent education events on oral health?
39. IF STAFF MAKE REFERRALS TO PARTNERS: How do you make referrals to
community partners for services? Do you receive information from them about
treatment and needed follow up? How do these referral systems work?
40. Have you provided training to community partners about providing oral health
services to your target population? If yes, please describe the training you provided.
Why did you decide to provide it, and how helpful was the training?
41. How often do you communicate with community partners and what form does the
communication take (meetings, phone calls, emails, referrals)? What do you
typically communicate about? How well does communication with partners work?
42. In your opinion, how are the partnerships going so far? What has worked well about
the partnerships, and what has been challenging? What strategies have your used to
work through the challenges? How well have these strategies worked?
43. Based on your experience with the Oral Health Initiative, are there other kinds of
partners that would have been helpful? If so, what types of partners and why?
44. If you could, is there anything you would change about the partnerships or
partnership agreements? If so, what would you change and why?

A.22

IMPLEMENTATION LESSONS (15 minutes)
45. At this point, how much progress have you made toward meeting your goals for the
Oral Health Initiative?
46. What have been your most important successes so far? What are you most proud of?
47. What are the most significant challenges your program has faced so far?
48. What strategies have you used to address these challenges? How well do you think
these strategies are working?
49. What are the most important lessons your program has learned about providing oral
health services?
50. What changes, if any, would you like to make to your Oral Health Initiative and
why?
51. What advice would you give to other programs that want to implement a similar
initiative?
52. Is there anything else you would like to add before we end the discussion?
Thank you again for participating in the interview.

A.23

Head Start Oral Health Initiative
Community Partner Site Visit Interview Protocol

INTRODUCTION (10 minutes)
Thank you for agreeing to participate in this interview. My name is ___________________
and I work for [MATHEMATICA POLICY RESEARCH/HEALTH SYSTEMS RESEARCH],
an independent research firm. As you know, we are conducting a study for the Administration
on Children and Families about Head Start agencies’ experiences implementing the Head Start
Oral Health Initiative. Findings from the study will be helpful to other Head Start agencies
implementing similar initiatives.
Everything you tell me is confidential. I would like you to feel comfortable giving your
opinions and impressions. The information we gather will be used to write a report for the
Administration for Children and Families about programs’ experiences implementing the Head
Start Oral Health Initiative, including successes, challenges, and lessons learned by grantees.
Our report will describe the experiences and viewpoints expressed by staff across grantees, but
specific comments will not be attributed to specific individuals or programs. No individual staff
member will be quoted by name.
Do you have any questions before we get started?
About You
To begin, I’d like to ask some questions about you and your agency.
1. What is your official job title, and what are your primary responsibilities?
2. How long have you worked for [AGENCY]?
3. How long have your held your current position? What other positions have you held
within the agency?
Your Agency
4. What is your organization’s primary mission?
5. What are the main programs your agency operates and services you provide? What
oral health services do you provide?
6. What are the main characteristics of your agency’s client population?

A.24

COMMUNITY AND FAMILY CHARACTERISTICS (5 minutes)
Now I have a few questions about the community and the children and families you serve
through your partnership with the Head Start Oral Health Initiative.
7. What is your impression of the availability of oral health care providers in the
community? General dentists? Pediatric dentists? Other providers? Do oral health
care providers in your community accept Medicaid? Are they willing to serve young
children?
8. What is your impression of the availability of other services for children and
families, such as health care, transportation, and other social services?
9. In your opinion, what are families’ main barriers to accessing oral health care?
10. Tell me about the children and families who serve through the Oral Health Initiative.
What are their primary oral health care needs? What are their cultural norms and
practices related to oral health care? Oral health care beliefs and practices for young
children? What is the prevalence of practices that threaten oral health, such as
putting babies to bed with bottles, using pacifiers past age 3, giving children
sweetened drinks, other?
11. Did you have experience providing services to Head Start children and families
before the Oral Health Initiative began, or was this a new experience for you? If it
new, has it been easier or more difficult that you thought it would be?
PARTNERSHIP WITH THE HEAD START ORAL HEALTH INITIATIVE (10 minutes)
Let’s talk about your partnership with the Head Start program.
12. Did your partnership with Head Start begin with the Oral Health Initiative, or were
you already partnering with the program before this initiative began? IF PREVIOUS
PARTNERSHIP: Tell me about your previous partnership? What was your role?
13. How did your organization become involved in the Oral Health Initiative?
14. Why did your agency decide to enter into the partnership? What interested your
agency in the Oral Health Initiative?
15. Was your agency involved in the process of designing the Oral Health Initiative, or
did your involvement begin after the Head Start program received the grant?
16. Tell me about your role in the Oral Health Initiative. What are the main services you
provide?
STAFFING AND COORDINATION (5 minute)
Let’s talk about staff from your agency that provide services through the partnership and how
you coordinate the work with Head Start.
A.25

17. How many staff from your agency provide services through partnership? What
proportion of their time do they spend on it? What are their job titles and
qualifications?
18. How do you coordinate the work your agency does on the Oral Health Initiative with
the Head Start program? Do Head Start staff refer children and families to you?
How does this work?
19. How often do you communicate with the Head Start program and what form does the
communication take (meetings, phone calls, emails, referrals)? What do you
typically communicate about? How well does communication for the partnership
work? Do you have suggestions for improving it?
20. Has your agency received any training from the Head Start Oral Health Initiative
related to providing services to Head Start children and families? If yes, please
describe the training you received. Who provided the training? Was it helpful?
Why or why not?
SERVICE DELIVERY (15 minutes)
Now I’d like to hear about the services you provide to Head Start children and families
through the Oral Health Initiative.
21. Do you conduct routine oral health risk assessments using clinical or other means
(such as clinical assessments, parent questionnaires, assessment of medical history,
assessment of demographic risk factors)? If so, do you use a formal oral health risk
assessment tool? Which tool do you use and why did you select it?
22. Do you use information from routine oral health risk assessments, whether you or
someone else conducts them, to make diagnoses or develop treatment plans for Head
Start children and/or other family members?
23. Do you provide other clinical preventive services to children and families through
the Oral Health Initiative? If so, what services do you provide? For example, do
you provide clinical exams, cleanings, fluoride treatments, or other preventive
services? Which services are provided by your program and which are provided by
partners? Where are the services provided?
24. Do you provide clinical treatment services to Head Start children and families? If so,
what types of services do you provide and where do you provide them?
25. How are the costs of risk assessment and other clinical services you provide to Head
Start children and families covered?
Payment by program?
Insurance
reimbursement? Services donated? Other?
26. Are you involved in keeping track of treatment outcomes and needed follow-up
services for Head Start children and families? Do you report treatment outcomes to
the Head Start program? If so, how do you do this?

A.26

27. Do you provide services to help Head Start families access needed clinical services,
such as help them make appointments, provide transportation, or provide translation
services? If so, how do you decide which families need these services? Do you
receive referrals from Head Start?
28. Do you provide or arrange for dental homes for Head Start children and families? If
so, how do you do this? What is your definition of a dental home?
29. Do you provide education and skills-building activities to families about oral health
promotion? If so, how do you do this—one on one with parents, during home visits,
during parent education workshops, other? What are the main educational messages
you provide? Do you use a curriculum to provide oral health education to families?
If so, what curriculum do you use and why did you choose it? Do you instruct
parents on how to do visual inspections of children’s teeth using such techniques as
“Lift the Lip”?
30. Do you provide oral hygiene supplies to children and families? If so, what types of
supplies do you provide, and to whom? How do you provide them and how often?
Do parents receive training on how to use the supplies?
31. To what extent have you tailored education and other non-clinical services to the
needs and cultural norms of your target population for the Oral Health Initiative?
Can you please provide some examples?
32. In your experience, how receptive have Head Start families been to the services you
provide through the Oral Health Initiative? How has their receptivity changed over
time?
33. In addition to working with Head Start children and families, does your agency
contribute supplemental funding or other in-kind resources to the Head Start Oral
Health Initiative?
LESSONS LEARNED (15 minutes)
At this point, I’d like to hear about the lessons you’ve learned so far from your involvement
in the Head Start Oral Health Initiative.
34. How well is the partnership going so far? What as worked well, and what has been
challenging?
35. Have you been able to implement the partnership as planned? If you’ve made
changes, what changes did you make and why?
36. Do you have suggestions for improving the partnership? Improving referral
systems? Communication with Head Start? Other suggestions?
37. How long do you think your partnership with Head Start will last? Will you
continue the partnership after grant funding for the Oral Health Initiative ends?
38. What advice would you give to other organizations like yours about partnering with
Head Start on a similar oral health initiative?
A.27

39. In your opinion, what have been the most important successes of the Head Start Oral
Health Initiative so far?
40. What are the most significant challenges the Oral Health Initiative has faced? What
strategies have been used to address these challenges? How well do you think these
strategies are working?
41. What lessons has your agency learned about providing oral health services to Head
Start children and families? What advice would you give to other service providers
about working with this population?
42. Do you have suggestions for improving the Head Start Oral Health Initiative in your
community? Are there changes you would make if you could? Is there additional
training from Head Start that would have been helpful?
43. Is there anything else you would like to add before we end the discussion?
Thank you again for participating in the interview.

A.28

Head Start Oral Health Initiative
Parent Focus Group Discussion Guide

INTRODUCTION (10 minutes)
Thank you very much for agreeing to participate in this discussion. Your participation is very
important to the study. I’m __________ and I work for [MATHEMATICA POLICY
RESEARCH/HEALTH SYSTEMS RESEARCH], an independent research firm/organization.
We are conducting a study for the federal Administration for Children and Families to learn
about the Head Start Oral Health Initiative. As part of the study, we want to learn the oral health
care services that children and families are receiving through Head Start and about your opinions
about the services.
• I am going to moderate the discussion. It is really important for everyone to speak up
so we can have a lively and informative discussion.
• We ask that you respect each other’s point of view. There are no right or wrong
answers. You are the experts – we want to learn from you.
• It will be helpful if you speak one at a time, so everyone has a chance to talk.
• We have many topics to cover during the discussion. At times, I may need to move
the conversation along to be sure we cover everything.
• We also ask that you not repeat any of the discussion you’ve heard after you leave
today.
• We also want you to know that being part of this discussion is up to you, and you can
choose to not answer a question if you wish. Being part of this discussion will also
not affect the services you receive from Head Start.
• I would like to tape-record our discussion. I am taping our discussion so I can listen
to it later when I write up my notes. No one besides our research team will listen to
the tape. Everything you say here is private. When we write our report, we will
include a summary of people’s opinions, but no one will be quoted by name.
• If you want to say anything that you don’t want taped, please let me know and I will
be glad to pause the tape recorder. Does anybody have any objections to being part
of this focus group or to my taping our discussion?
• The discussion will last about 1½ hours, and we will not take any formal breaks. But
please feel free to get up at any time if you need to.
Once again, thank you for coming today. Are there any questions before we get started?

A.29

1. Let’s go around the room and introduce ourselves. Please tell us:
• Your first name
• The name of age of your child who is enrolled in Head Start or Early Head Start
ACCESS TO SERVICES IN THE COMMUNITY (20 minutes)
To begin, I’d like to ask some questions about how easy or difficult it is for you get oral health
care and other services you need in the community.
2. In your experience, how easy or difficult is it to find dentists or other dental
providers who are willing to treat children under the age of 5? Under the age of 3?
3. Overall, what are the main problems you face in arranging dental services for your
children? PROBES: Finding dentists? Waiting lists for appointments?
Transportation to dentists’ offices? Few dentists located near where you live?
Paying for dental care? Finding dentists who speak your language? Other
problems?
4. Can I see a show of hands, how many of you have dental insurance coverage to pay
for dental care, such as Medicaid or SCHIP? If you don’t have dental insurance, do
you have another way to pay for dental services? For example, will Head Start pay
for dental services? Are dentists available who provide free or low-cost dental care?
5. Can I see a show of hands, how many of you have a single place you can go to get
dental treatment for your children? For those who have a single dental provider, how
did you find this provider? Did Head Start help you find this dentist? How do you
pay for the services? Are there services that your child needs that this dentist will
not provide? About how often does you child visit this dentist?
6. Do you go to the dentist? Do you have a single place you can go to get dental
treatment for yourself? If so, how did you find this provider? Did Head Start help
you?
7. How easy or difficult is it for you to find other health services your children need,
such as medical care? IF DIFFICULT: What are the main problems you have trying
to find doctors and other medical providers? Overall, would you say it is easier or
more difficult to arrange medical or dental services for your children? Why?
PARENTS’ ATTITUDES AND BELIEFS ABOUT ORAL HEALTH CARE (15 minutes)
Now I’d like to talk about your views on the dental services that you and your children need.
8. How important do you think children’s dental health is for their overall healthy
growth and development? Is it important, somewhat important, or not related to their
physical health and growth? Why do you think this?

A.30

9. Ideally, at what age do you think a child should receive his or her first dental exam?
Why? After the first exam, how often do you think children under age 5 should see a
dentist?
10. Assuming the services are available, how often to you think adults should see a
dentist? Why?
11. Tell me about how you take care of your children’s teeth at home. Between birth
and age 1? At age 1? At age 3? At age 5? At what age do you think it is important
to begin brushing children’s teeth?
12. Do you look at your children’s teeth at home? If so, tell me how you do this. About
how often do you do this?
13. Has your child’s doctor ever talked to you about how to take care of your child’s
teeth, or about when your child should begin seeing a dentist? If so, please tell me
what advice the doctor gave you.
RECEIPT OF ORAL HEALTH SERVICES (25 minutes)
At this point I’d like to hear about the dental services you and your children are receiving.
14. Can I see a show of hands, has someone examined your Head Start child’s mouth at
least once? Who did the exam? A dentist? A dental hygienist? A pediatrician or
doctor? Someone who works for Head Start?
15. Where did this dental exam happen? At a dentist’s office or clinic? At a pediatrician
or doctor’s office? At Head Start? At another location? Were you able to be
present, or did you receive information from the exam about your child’s dental
health?
16. Did any of your children need follow-up services or treatment? If so, what did they
need? How easy or difficult was it for you to arrange these services for your child?
Did Head Start help you, for example, by helping you find a dentist, make an
appointment, get to the dentist’s office, or pay for the services?
17. Can I see a show of hands, has your Head Start child seen a dentist or another dental
provider in the past six months? For what reason—an exam, a cleaning, for
treatment? If it was for treatment, what kind of treatment did your child receive?
18. At this point, are you able to arrange the dental services you think your child needs?
Regular check-ups and cleanings? Treatments?
19. Can I see of show of hands, how many of you have seen a dentist in the past six
months? What was your reason for seeing a dentist? Check-up and cleaning?
Treatment? How easy or difficult was it to find a dentist? Did Head Start help you?
20. How often do you usually go to the dentist? Do you get regular check-ups, or do you
see a dentist only when you have problems with your teeth? Where do you usually
go for dental services? How do you pay for the services?

A.31

21. What are the primary reasons you do not go to the dentist? PROBES: Is it because
you don’t have dental insurance, you can’t take time off from work, you don’t have
transportation, you’ve had bad experiences with dentists in the past, or you are afraid
to go to the dentist?
22. Has Head Start ever helped you with the following for your child? For you or other
family members? If so, please tell me what they did to help.
• Find a dentist or other dental provider
• Make an appointment with a dentist or dental provider
• Provide transportation to a dental appointment
• Provide translation services during a dental appointment
• Help you pay for dental services
23. Overall, how satisfied are you with the help you received from Head Start to arrange
dental services for your Head Start child? For yourself? For other family members?
24. Do you have suggestions for how Head Start could improve these services? Are
there other things Head Start could do to help you get the dental services you and
your family need?
ORAL HEALTH EDUCATION (20 minutes)
Now I’d like to talk about the information you and your child have received about how to
take care of your teeth.
25. In the past year, have you gone to any workshops or parent trainings about oral
health? If so, what did you talk about during that event? Who provided the
information? Did you receive any information in writing about oral health at that
event?
26. Have you received information about oral health in any other way, such as during a
home visit or during a visit to a dentist’s office? If so, what did you talk about and
who provided the information?
27. Have you received any written information such as pamphlets or handouts, or videos
about how to take care of your own or your children’s teeth? What did you learn
from these materials? IF SOME FAMILIES DO NOT SPEAK ENGLISH AS A
FIRST LANGUAGE: Did you receive these materials in English, or another
language?
28. IF NOT ALREADY MENTIONED: Did any of the educational activities and
materials provide you with instruction on how to examine your child’s teeth and
mouth at home, and what to look for? If so, tell me about what you learned.

A.32

29. In the past year, do you know if your Head Start children received any education or
training on how to take care of their teeth, such as learning how to brush, from Head
Start? Did this happen in a Head Start classroom, during a home visit, or during a
visit to a dentist? Tell me about what your child learned.
30. Can I see a show of hands, have you ever received supplies from Head Start for
taking care of your children’s teeth (such a toothbrush, toothpaste, other)? Do you
receive these supplies regularly, occasionally, or just once? Did you receive any
instruction about how to use the supplies? Who provided the instruction?
31. Overall, how helpful has the education and training on oral health you have received
from Head Start been for you? Did you learn anything new from this information?
If so, what did you learn? How helpful do you think the education and training has
been for your children?
32. Is there anything you would change about the oral health education and training that
Head Start provides? Do you have any suggestions for improving this component of
the program?
CONCLUSION
Those are all of the questions I had for you today. Is there anything else about dental
services provided by Head Start that you think I should know about? Anything else you
would like to mention before we end?
Thank you for taking the time to share your thoughts and experiences. Our discussion has
been very useful for helping me learn more about the Head Start Oral Health Initiative.

A.33

APPENDIX B
PROGRAM RECORDKEEPING SYSTEM:
PROPOSED DATA ELEMENTS

TABLE B.1
CHILD AND FAMILY CHARACTERISTICS SCREEN FOR THE
PROGRAM RECORDKEEPING SYSTEM: PROPOSED DATA
ELEMENTS AND RESPONSE CATEGORIES
Data Elements
Primary recipient
Child or woman’s name
Identification number
Child’s date of birth
Due date (if pregnant woman)
Gender of child
Race/ethnicity of the child

Whether child/pregnant woman has dental
insurance coverage
If yes, type of coverage

Whether child has ever had a dental exam

If yes, date if known
Primary caregiver’s (parent) namea
Primary caregiver’s date of birth
Primary caregiver’s gender
Primary caregiver’s race/ethnicity

Primary language spoken at home

Response Category
Child
Pregnant woman
Open field
6-digit number
Open date field
Open date field
Male
Female
American Indian or Alaska
Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other
Pacific Islander
White
Yes
No
Private insurance
Medicaid
SCHIP
Other
Yes
No
Don’t know
Open date field (month/year)
Open field
Open date field
Male
Female
American Indian or Alaska
Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other
Pacific Islander
White
English
Spanish
Other

B.2

Response Type
Drop-down list
Open field
System will generate
Date field
Date field
Drop-down list
Check boxes (all that
apply)
Yes/No

Check Box
Drop-down list

Check Box

Date field
(month/year)
Open field
Date field
Drop-down list
Check boxes (all that
apply)
Yes/No

Drop-down list

TABLE B.1 (continued)
Data Elements
If not English, how well primary caregiver
speaks English
Primary caregiver’s relationship to the child

Head Start enrollment date
Oral Health Initiative enrollment date (or date
first service received)
Exit date

Response Category
Very well
Well
Not well
Parent or stepparent
Grandparent
Other relative
Other nonrelative
Open date field
Open date field

Response Type
Drop-down list

Open date field

Date field

a

Drop-down list

Date field
Date field

All data fields on primary caregiver (parent) characteristics will be completed for pregnant women when
they are the primary targets of the intervention.

B.3

TABLE B.2
COMMUNITY PARTNER SCREEN FOR THE PROGRAM RECORDKEEPING SYSTEM:
PROPOSED DATA ELEMENTS AND RESPONSE CATEGORIES
Data Element
Community partner name
Identification number
Type of partner

Formal partnership agreement
If yes, date of agreement
Community partner prior to
Oral Health Initiative
Partnership end date

Response Category
Open field
6-digit number
General dentist
Pediatric dentist
Dental hygienist
Dentistry school
Dental hygiene school
Pediatrician
Family practitioner
OB/GYN
Nurse practitioner
WIC program or clinic
Public health department
Other clinic
Hospital
Part B or C
Other service provider
Other
Yes
No
Open date field
Yes
No
Open date field

B.4

Response Type
Open field
System will generate
Drop-down list

Check box
Date field
Check Box
Date field

TABLE B.3
SERVICES SCREEN FOR THE PROGRAM RECORDKEEPING SYSTEM: PROPOSED DATA ELEMENTS
AND RESPONSE CATEGORIES
Data Elements
Name of child/pregnant women
Identification number
Date of service
Type of service

Type of clinical preventive service

Type of clinical treatment

Location of services

Type of service provider

If community partner, name of partner
Identification number

Response Category
Select from drop-down list
6-digit number
Open date field
Non-clinical dental screening
Clinical preventive service
Clinical treatment service
Clinical exam
Cleaning
Fluoride rinse
Fluoride varnish treatment
Fluoride tablets prescribed
Xylitol wipes
Root planing and scaling (preventive)
Anticipatory guidance
Other
Fillings (1-2)
Fillings (2 or more)
Extractions (1-2)
Extractions (2 or more)
Steel crowns
Root canal
Bridge/dental implant
Root planing and scaling (therapeutic)
Treatment requiring hospitalization and/or
sedation
Other
At grantee site
Service provider office
Hospital
At home
Mobile van or mobile clinic
Other location
Grantee staff
Community partner
Other community provider
Select from list
6-digit number

B.5

Response Type
Drop-down list
System will insert
Date field
Drop-down list

Drop-down list

Drop-down list

Drop-down list

Drop-down list

Drop-down list
System will insert

TABLE B.3 (continued)
Data Elements
If other community provider, type of
provider

Support services provided
If yes, type of service

If yes, service provider

If community partner, name of partner
Followup required
If yes, type of followup

If yes, followup action

If completed, date completed
Referred to service by grantee
Dental home established
Type of dental home
If community partner, select
If other provider, type

Response Category
General dentist
Pediatric dentist
Dental hygienist
Dentistry school
Dental hygienist school
Pediatrician
Family practitioner
OB/GYN
Nurse practitioner
WIC program or clinic
Public health department
Other clinic
Hospital
Part B or C
Other service provider
Other
Yes
No
Transportation
Help making an appointment
Translation
Other
Grantee staff
Community partner
Other community provider
Select from list
Yes
No
Referral
Appointment
Treatment
Counseling
Other
Referral made
Appointment pending
Followup competed
Open date field
Yes
No
Open date field
Community partner
Other community provider
Select from list
Private dental office
Community health center
Mobile van or mobile clinic
University dental clinic
Other

B.6

Response Type
Drop-down list

Check box
Drop-down box

Drop-down box

Drop-down list
Check box
Drop down list

Drop down list

Date field
Check box
Date field
Drop-down list
Drop-down list
Drop-down list

TABLE B.4
ORAL HEALTH EDUCATION/SUPPLIES SCREEN FOR THE PROGRAM RECORDKEEPING
SYSTEM: PROPOSED DATA ELEMENTS AND RESPONSE CATEGORIES
Data Elements
Whether parent education services
provided in past month
Parent education workshop provided
Number of workshops provided
Total number of workshop attendees
Parent education provided during home visits
Number of home visits with oral health education
Oral Hygiene supplies provided to families
in the past month
Types of supplies provided

Total number of families who received any supplies
Parent education provided through written materials
sent home with children in the past month
Staff training on oral health provided in past month
Total number of training attendees

B.7

Response Category
Yes
No
Yes
No
Open numeric field
Open numeric field
Yes
No
Open numeric field
Yes
No
Fluoride toothpaste
Toothbrushes
Floss
Fuoride rinse
Xylitol gum
Xylitol wipes
Other supplies
Open numeric field
Yes
No
Yes
No
Open numeric field

Response Type
Check box
Check box
Numeric field
Numeric field
Check box
Numeric field
Check box
Check boxes (all that apply)

Numeric field
Check box
Check box
Numeric field