ss_Bright Futures_NEW_Part B

Supporting Statement
HRSA Office of Women’s Health Bright Futures for Women’s Health and Wellness Survey

B. Collection of Information Employing Statistical Methods
1. Respondent Universe and Sampling Methods
The data collection effort for this evaluation will not require any complex statistical sampling
methods. Because there are only four program sites participating in the evaluation, the results
will not be statistically significant and therefore cannot be generalizable. Nevertheless, we will
seek to collect responses from each of the sites that reflect the consumers and providers at those
sites and provides useful and valuable information to HRSA on the guides.
A list of candidate sites was identified based on recommendations from OWH and/or the
evaluation’s Expert Committee members, from which the final four sites were selected. Through
the site selection process, we have attempted to include sites that are likely users of the
BFWHW emotional wellness guides. By identifying site and respondent characteristics in the
final report, other organizations may be able to assess the potential usability and acceptability of
these guides by the young and adult women they serve.
At all four evaluation sites (two Maternal and Child Health Bureau grantees, one Ryan White
Part D grantee, and one Federally Qualified Health Center/Community Health Center), young
and adult woman who read and write English (the BFWHW emotional wellness guides being
tested are the English language versions) and who receive services during the data collection
period will be given either the Young Woman’s Guide to Emotional Wellness or the Woman’s
Guide to Emotional Wellness. Adult women who might have received the guide will be invited
to complete a consumer survey. At sites serving young women who might have received the
guide, if parents have consented to allow them to participate, these young women will be
invited to complete a consumer survey. Approximately 750 surveys will be distributed to
young and adult women across sites, for an expected total of 563 surveys collected (75%
response rate). Please read the Section 3 (Methods to Maximize Response Rates) below for an
explanation of why the 75% response rate was chosen.
Additionally, providers who deliver services to young women and adult women will be asked
to distribute the BFWHW emotional wellness guides and will be invited to complete a one-time
survey about the guides (estimated to be approximately ten providers at each site for a total of
40 respondents).
At the evaluation sites, 6-8 adult women and 6-8 young women who read and write English,
who are currently receiving services, and who have not yet received the BFWHW guides, will
be identified by program administrators and staff and invited to participate in focus groups to
be conducted by the evaluation team during the site visit. During preliminary discussions with
sites, some sites indicated that they would select women already participating in existing
support groups (e.g., prenatal support groups) to also participate in the evaluation focus
groups. Therefore, these women participating in the focus groups will not represent a randomly
selected group, a limitation of the evaluation.

1

2. Information Collection Procedures
Data collection procedures have been designed to maximize response, minimize burden on the
respondents, and promote accuracy and completeness of responses. Described below are the
specific data collection procedures that we propose to use.
Consumer Surveys
Prior to the implementation of evaluation activities, the evaluation team will work with each of
the sites to identify the site program staff who will distribute the informed consent forms and
the consumer surveys.
All eligible adult and young women consumers who agreed to participate by giving informed
consent will be provided with the survey within the context of a service appointment (see
Consumer Survey in Appendix E).The women selected to complete the survey will represent
those women who came for their appointment during the period of data collection for the
evaluation. Thus, these women will not be randomly selected, which is a limitation of the
evaluation.
The primary strategy for survey completion will involve distributing the survey to the women
with a pre-stamped pre-addressed envelope addressed to the evaluation team and asking the
women to complete the survey on-site. The respondents will be instructed to seal the survey in
the envelope and return it to the site program staff who gave them the survey. Site program
staff will then mail completed surveys to the evaluation team on a weekly basis. Women will be
encouraged to complete the survey on-site. However, another option for consumers who are
unable to complete the survey on-site will be to return the completed surveys in the mail
themselves using the provided pre-stamped envelope.
In addition to working closely with each of the sites prior to the initiation of data collection, the
evaluation staff will maintain close contact with each of the sites over the course of the data
collection. The evaluation team will also provide each of the sites with a point of contact who
will be available by telephone and email whenever needed.
Distributor Survey
The professionals who will be distributing the BFWHW emotional wellness guides (e.g.
physicians, nurses, counselors, social workers, and outreach workers) will be given copies of the
Women’s Guide and Young Women’s Guide to review as well as a copy of the Health Care
Providers Guide to Promoting Women’s Emotional Wellness. The evaluation staff will work
with the sites to understand how, to whom and under what circumstances the BFWHW
emotional wellness guides will be distributed and discussed with women consumers. The
distributors of the guides will be asked to complete a web-based survey about their experience
as a distributor (Appendix G). Distributors will receive an email with a link to a web-based
survey, which provides a secure and anonymous response option that should be convenient for
this professional population. For distributors who would rather complete a paper-based written
survey, the survey in this format will be made available to them. Distributors will be asked to
complete the web-based survey or return the paper survey to the evaluation team within two
weeks of receipt.

2

3. Methods to Maximize Response Rates
In order to maximize response rates, the program administrator(s) of the selected sites will be
asked to sign a letter of commitment to actively participate in the evaluation. Thus far, the
selected sites have demonstrated great enthusiasm about participating in this evaluation.
Although we understand enthusiasm does not translate to a higher response rate, we feel the
active buy-in of the sites may increase the probability of successful implementation and
potentially moderately higher response rates. Additionally, to facilitate a higher response rate,
the consumer survey has been designed to be completed easily and quickly; it is short in length
and tailored to an appropriate literacy level. We expect that most of the surveys will be
completed on-site and returned to program staff that same day, which may also increase the
response rate. The evaluation team has a successful track record of collecting data from
evaluation sites in past evaluations (refer to two evaluations reports included in the Appendix
as Appendix I and J).
The most challenging aspect of maximizing response rates will occur in the collection of the
surveys from respondents. We are attempting to minimize any barriers in accomplishing this
task by using several features in the evaluation approach. First, women will be encouraged to
complete the survey on-site and able to return the survey to an identified staff member. If they
are not able to complete the survey on site, the women can mail the survey. However, we
anticipate that very few women will need to complete the survey by mail. Second, the
individual to whom the surveys will be returned to will be easily identifiable. Third, the
designated program staff responsible for overseeing the implementation will send the
completed anonymous surveys to the evaluation team. Although sites will be asked keep track
of the number of surveys distributed/completed, the evaluator will perform the data entry from
the completed surveys as well as the analysis and report writing at their offices. Fourth, the
surveys are written in a clear and comprehensible manner, no higher than a sixth-grade reading
level, and (based on pilot testing) should take no longer than 15 minutes to complete. Finally,
women will be offered a nominal cash award ($10) in acknowledgement of their time and effort
in completing the survey or participating in the focus group. This effort gathers information
from a diverse audience in different types of settings and is intended to reflect varying
populations and sites that might find the emotional wellness guides to be supportive of their
delivery of health and social services. Given the use of the convenience sample approach,
however, inferences will not be attempted and limitations of the results will be stressed in the
final report.
4. Tests of Procedures
The consumer surveys have been tested for literacy level and burden estimates based on a pilot
test conducted by the evaluator. Modifications to the survey entailed clarifying terminology and
language as well as rewriting or eliminating questions that may be confusing to respondents.
For example, using the term “assent” confused respondents, so we replaced “assent” with
“consent” in the consumer survey. Another respondent found the term “Domestic
Partnership/Cohabitation” confusing and thought other respondents would also be confused
by the phrase. Thus, we used “Living with Partner” in the consumer survey. While there was no
formal report produced from the pilot testing, four women participated in the pilot testing.
Evaluation staff visited a local Healthy Start site. The four women first received an explanation

3

of the purpose of the evaluation and received copies of the Emotional Wellness Guides, the
consumer survey and the informed consent form. Each woman took time to read the documents
and then completed the informed consent form and the survey. After completing the survey,
each woman was paired with an evaluation staff member. This staff member asked questions
such as “Was it easy or difficult to complete?” and “Were any of the questions confusing?” For
each specific question, the evaluation staff member also asked:
a) Are there any aspects of this question that is confusing? Does anything need to be
defined, clarified, or re-worded?
b) Do the answer options make sense? Could you easily pick the one that applies to
you?
c) Can you think of any reason this question might be hard for other consumers to
answer?
The feedback gained from the pilot testing helped shape the final version of the consumer
survey.
5. Statistical Consultants
The data collection and analysis will be conducted by The Lewin Group, 3130 Fairview Park
Drive Suite 800 Falls Church, VA 22042. The Project Director is Sharrie McIntosh, who can be
reached at 703-269-5611. The Project Manager is Carrie Blakeway, who can be reached at 703269-5711. The Statistical Consultant is Dr. Karen Linkins, who can be reached at 480-471-7516.

4