0990-rfcc

Supporting Statement for Rural/Frontier Women’s Health Coordinating Centers (RFCCs) National Evaluation

Table of COntents

A. JUSTIFICATION

1. Need and Legal Basis

2. Information Users

3. Improved Information Technology

4. Duplication of Similar Information

5. Small Businesses

6. Less Frequent Collection

7. Special Circumstances

8. Federal Register Notice/Outside Consultation

9. Payment/Gift to Respondents

10. Privacy

11. Sensitive Questions

12. Burden Estimate (Total Hours and Wages)

13. Capital Costs

14. Cost to Federal Government

15. Program or Burden Changes

16. Publication and Tabulation Dates

17. Expiration Date

18. Certification Statement

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

2. Procedures for the Collection of Information

3. Methods to Maximize Response Rates and Deal with Nonresponse

4. Test of Procedures or Methods to be Undertaken

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

A. Justification

1. Need and Legal Basis

The Department of Health and Human Services Office on Women’s Health (OWH) is seeking clearance to conduct data collection efforts as part of the National Evaluation of the Rural/Frontier Coordinating Center (RFCC) program. The Office on Women’s Health funded the creation of three RFCCs in September 2004, and awarded eight additional RFCC contracts in fiscal year 2005. The impetus for creating the RFCCs was to, “identify, coordinate, and leverage the network of existing resources to provide a full range of culturally and linguistically appropriate health services to women and their families.”¹

The residents of America’s rural and frontier communities confront significant barriers to accessing health and social services, including geographic isolation, a scarcity of public transportation, a lack of available services, limited employment opportunities, an inability to pay for health care services, and a shortage of qualified health care professionals.² In its March 2004 report, From Rural to Remote America, Family Health Care in Alaska, Idaho, Oregon and Washington, the Association of Maternal and Child Health Programs identified a number of demographic characteristics and health level indicators of rural residents that are attributable to the significant disparity in health care status between urban and non-urban populations.³ The study found that residents of rural communities:

• Are older, poorer, less educated and often uninsured;

• Are increasingly transient migrant and seasonal workers;

• Have lower immunization rates;

• Frequently lack early and adequate prenatal care;

• Suffer higher rates of infant and childhood mortality;

• Have greater alcohol and tobacco use during pregnancy; and

• Have poorer dental care.

Women residing in rural and frontier communities face many challenges in meeting their basic health care needs. As a result of numerous barriers, this vulnerable population is underserved and generally in a poorer state of health compared to women living in urban areas of the U.S. To effectively meet the numerous health care needs of this diverse population, rural health providers must not only offer comprehensive health care services but also integrate these services to maximize awareness, access, and quality. RFCCs were created to accomplish this task.

Evaluating the effectiveness of RFCCs is essential for determining whether these centers are the best vehicles for “coordinating and leveraging new and existing resources for women’s health in rural and frontier communities.” The OWH is seeking to evaluate all eleven RFCCs. The evaluation and accompanying analysis will enable policy makers to determine whether RFCCs increase access to primary health care services in rural and frontier communities and ultimately improve the health status of rural and frontier women and their families. This evaluation will also enable the OWH to determine how well RFCCs are facilitating access to integrated and comprehensive primary care services to women and their families residing in rural and frontier regions of the U.S.

In addition, there is a legal basis for conducting this evaluation. Section 301 of the Public Health Service Act (42 U.S.C. 241) authorizes the collection of information for the purpose of evaluating the RFCC program.

The results of this evaluation will also be a crucial component for completion of the Program Assessment Rating Tool (PART) for the Office on Women’s Health. Completion of the PART is an OMB requirement for OWH. The PART looks at all factors that affect and reflect program performance including program purpose and design; performance measurement, evaluations, and strategic planning; program management; and program results. The National Evaluation of the RFCC program will provide meaningful insight into many of these factors, and will thus help enable OMB to identify OWH’s key strengths and potential weaknesses.

OWH’s evaluation methodology (see Attachment I) includes multiple phases and relies on gathering a combination of quantitative and qualitative data from a variety of sources. This approach provides a solid basis for developing a comprehensive assessment of the RFCC program. In addition to reviewing state and local health department databases, as well as periodic reports submitted by each RFCC that describe activities undertaken in order to meet OWH’s contractual requirements, the OWH is also seeking to collect data from RFCC Center Directors and Program Coordinators and from RFCC clients and patients. The OWH recognizes that there is tremendous variation across RFCCs in terms of overall structure and partnering arrangements. Some RFCCs are in a position to collect detailed clinical information about the women who access the RFCC while others are able to collect limited demographic and clinical information. Still others are only able to collect very limited information about the services women access through the RFCC. For this reason, the data collection process has been structured in order to accommodate the data collection abilities of the individual RFCCs. The data collection instruments are:

• RFCC Center Director/Program Director Questionnaire

• RFCC Women’s Services Registration Form

• RFCC Clinical Intake Forms

• RFCC Client/Patient Questionnaire

• Site Visit Interview Guide

• RFCC Activities Tracking Sheet

RFCC Center Director/Program Coordinator Questionnaire

RFCC Center Directors and Program Coordinators will be requested to jointly complete a detailed questionnaire. An identical questionnaire will be emailed to Center Directors and Program Coordinators at each site. This questionnaire will serve to validate information found in RFCC reports but will also contain additional information on the structure and operation of the RFCC, as well as key initiatives and the impact or expected impact of these initiatives on women in the community. This information is not available by any other means.

The questionnaire is designed to answer questions related to the following areas:

• The needs of women in the service area and programs and services offered to meet those needs;

• RFCC partnerships;

• Funding and financial issues;

• Current status of the RFCCs’ referral and tracking process or system;

• Workforce shortage issues and the use of allied health and lay health professionals;

• Success of innovative approaches implemented by the RFCC;

• Status of community linkages;

• Programs or initiatives designed to promote leadership development among women;

• Transportation needs and initiatives;

• Research programs;

• Programs geared towards specific geographic and cultural characteristics of the population;

• Success of educational programs and activities for RFCC clients; and

• Programs and policies in place to promote leadership development and foster relationships with community leaders.

A copy of the RFCC Center Director/Program Coordinator Questionnaire is included as Appendix A.

RFCC Women’s Services Registration Form

The RFCC Women’s Services Registration Form was designed in order to collect basic information about each woman who accesses the RFCC and to provide a mechanism for tracking the number of women served by the RFCC. The form requests general identification information such as name, address, date of birth, and emergency contact information. In addition, it requests basic demographic information such as race, health insurance status, education, income and employment status. The form also asks women to select (from a list) the types of health and social services they are interested in learning about. The form can be completed either by the RFCC client or an RFCC staff member or provider, and the information can be collected on paper, via telephone or via an RFCC website. RFCC staff members will assist Spanish speaking clients by translating registration form questions as needed. A copy of the registration form is included as Appendix B.

RFCC Clinical Intake Forms

Data on women who access clinical services through direct care providers will be collected through clinical intake forms that will be completed in addition to the Women’s Services Registration Forms. Individual providers have their own clinical intake forms and have not been asked to create new forms for this purpose. However, direct care provider RFCCs have enhanced their existing clinical intake forms to include women-focused information requested by the OWH. Non-direct care RFCCs who refer to specific providers have requested these providers to include specific elements on the form that are relevant to women’s health and goals and objectives of the RFCC program. These elements are as follows:

1. The use of complementary and alternative medicine, including use of vitamins and supplements.

2. Sleep/rest patterns.

3. Family history (both maternal and paternal sides).

4. Diet/nutrition, including use of alcohol, tobacco, other substances.

5. The types, level and amount of physical activity.

6. How stress is managed.

7. Relationships, including whether there are safety/security issues (e.g., domestic violence).

8. Sexual history, including contraception, problems, interest, etc.

9. Bladder function/issues/problems, including exam interpretation via visualization such as ultrasound.

10. Neurological function, including reflexes, gait, etc.

11. Eye exams, including dates of exams.

12. Caregiver issues and health of caregivers.

13. Date of last breast self exam and mammogram.

14. Date of last Pap smear.

15. Comprehensive questioning on issues of hot flashes and other symptoms of menopause.

16. Other services needed by client (e.g., transportation, translation, social services, etc.)

17. Other issues that the client would like to address or ask about.

There are two RFCCs that are direct clinical care providers: Women’s Wellness and Maternity Center in Tennessee and the Utah Navajo Health System in Montezuma Creek. These RFCCs will use their existing clinical intake forms to capture the required information on all women who access the RFCC specifically for clinical care. These data elements include:

• Past medical history

• Medications taken

• Allergies

• Family medical history

• Lifestyle and Social history

• Occupation

• Education

• Nutrition and diet

• Exercise

• Stress

• Race/Ethnicity

• Transportation

• Tobacco use

• Alcohol and Drug use

• HIV and AIDS

• Birth control and reproductive history

• Current physical symptoms

• Depression screening

• Health issues of the elderly

  • Incontinence

  • Falls

  • Ability to perform daily activities

• Advance Care Directives

Copies of the clinical intake forms for both of these RFCCs are included as Appendices C-1 and C-2.

The other nine RFCCs are not clinical care providers. However, all of these RFCCs either partner with or refer to clinical care providers. For a sample of 15 women who are referred for clinical care, RFCCs will obtain the completed clinical intake forms from the referral organizations. These forms will be de-identified in order to comply with HIPAA requirements. A database of clinical intake information (de-identified) from these forms will be created and used for the analysis.

RFCC Client/Patient Questionnaire

A brief client/patient questionnaire will be administered to a sample of women who access services at specific RFCCs. This questionnaire is a key element in the data collection effort, as it will provide a snapshot of individual experiences relating to the access and delivery of services at each RFCC. The questionnaires will be used to assess patient satisfaction with the RFCC in terms of helpfulness in providing information regarding specific services as well as referrals to these services. A sample of 15 women will be surveyed at each RFCC site. Questionnaires will be mailed to clients with a self-addressed stamped envelope for their completed responses. A copy of the questionnaire is included as Appendix D.

On-Site Interviews

Site visits will allow an independent assessment of the RFCCs and will provide a more detailed and thorough appraisal of the RFCCs than can be achieved through surveys. Site visits will provide valuable insight into the current day-to-day operations of each RFCC. Data collected during site visits will also be used to validate and supplement information gathered through other means. Site visit interview guides are included as Appendix E. The number of interviews at each site will range from approximately 7 to 15. Each interview is expected to last approximately one hour. Individuals interviewed will include the following:

• RFCC Center Director;

• RFCC Program Coordinator;

• RFCC Local Evaluator;

• Institutional Commitment Individual;

• Prominent partners;

• Key community stakeholders; and

• Individuals who can address each of the five required components of the RFCC : clinical care, outreach, professional education/training, research, and leadership (recognizing that one person may address two or more components).

Each site will be conducted over a one to two-day period. In addition, it is expected that individual clinical care provider sites will also be visited as appropriate and feasible.

RFCC Tracking Sheet

RFCCs will use a standard tracking sheet to record the various activities they are implementing within the five required components (see above). The standard format of this data collection instrument will enable the evaluation team to assess how the program is being carried out in different settings and with different patient populations throughout the country. The tracking sheet is included as Appendix F.

The National Evaluation is crucial to OWH’s decision-making process regarding the continued existence, design, and funding levels of the RFCC program. Evaluation findings will also help OWH understand how the RFCC program is contributing to improved care and access for underserved populations of women across the country.

1. Information Users

Information provided by each of the data collection instruments will enable the OWH to qualitatively and quantitatively assess the efforts and effectiveness of RFCCs in improving the health status of women and their families. The data will also be used to evaluate RFCCs against the following metrics:

• Performance within each center,

• Successful implementation of each of ten designated objectives (as described below), and

• Successful integration of the ten objectives.

Data collection efforts for the National Evaluation are structured based on the ten objectives listed below. For each RFCC, the ten objectives will be examined both independently and collectively in order to assess the extent to which they are integrated.

1. Creation of the Coordination Center;

2. Developing a comprehensive referral and tracking system;

3. Utilizing allied health professionals and other primary care providers;

4. Implementing innovative approaches to health care delivery;

5. Linking community resources;

6. Fostering women to lead and advocate;

7. Maximizing transportation and intake points;

8. Facilitating research in women’s health issues;

9. Promoting cultural and geographic competence in training and education; and

10. Encouraging leadership development.

In addition, the information collected will be used to answer the RFCC evaluation research questions, which are included as Attachment II. Attachment III is a matrix that indicates which research questions will be addressed by each of the data collection instruments.

Data collection efforts will provide the OWH with the information necessary to evaluate the effectiveness of the RFCC program, including the following specific measurements:

• Number of women reached,

• Women’s utilization of services and satisfaction with access to services,

• Descriptions training and educational programs developed and implemented by RFCCs, and

• The program’s ability to meet the health care needs of American Indian, Alaska Native and elderly women living in rural and frontier communities.

A.3 Improved Information Technology

Information technology will be used, where appropriate, to reduce burden among survey respondents. Information technology will be used in the administration of the RFCC Center Director/Program Coordinator Questionnaire and the RFCC Tracking Sheet. These data collection instruments will be emailed to all RFCC Center Directors and Program Coordinators. The Center Directors/Program Coordinators will be required to complete and return the RFCC Tracking Sheet electronically. However, they will have the option to either complete the RFCC Center Director/Program Coordinator questionnaires and return them electronically, or complete them manually and return them via regular mail.

The RFCC Women’s Services Registration Form and the RFCC Clinical Intake Forms will be completed by RFCC staff members or providers who assist clients in accessing services. Registration Form data will be collected primarily via telephone but also may be collected in person. RFCC clients may also provide Registration Form data via an RFCC website. Clinical Intake Form data will only be collected in person by providers.

Information technology will not play a role in the administration of the Client/Patient Survey. Technology is not an effective approach for conducting a survey of RFCC clients/patients because the target group tends to be low income and less educated, and may not have adequate skills to complete a computer-based survey, or have access to a computer. In addition, the facilities where this survey will be administered may not have computers available for this effort.

Information collected during in-person site visit interviews will be immediately transferred into an electronic data collection tool by the site visit team.

All “raw” data collected from each of the data collection instruments will be entered and stored in an electronic database. Information technology use has been carefully planned to reduce burden among survey respondents.

A.4 Duplication of Similar Information

No effort to collect similar data is being conducted at this time.

A.5 Small Businesses

This data collection effort may impact small entities as many of the RFCCs are small non-profit organizations. To minimize the impact on these groups, the RFCC Center Director and Program Coordinator Questionnaire; the RFCC Site Visit; and the RFCC Tracking Sheet will be administered only once. Each RFCC Center Director was made aware of the evaluation schedule and understands that responding to this evaluation effort is mandatory and part of activities required in order to receive funding from the OWH.

A.6 Less Frequent Collection

This is a one-time data collection effort. If this information is not collected, the OWH will be unable to accurately measure and evaluate the impact of the RFCC program against its stated objectives.

A.7 Special Circumstances

The proposed data collection effort fully complies with all guidelines of 5 CFR § 1320.5 (d) (2).

A.8 Federal Register Notice/Outside Consultation

The data collection notice for the RFCC evaluation was published in the Federal Register, page 6243, Vol. 72, No. 27 on February 9, 2007. A copy of the Federal Register notice is included as Appendix G. As requested by 5CFR1320.(d). no public comments were received in response to the notice.

The OWH hired the consulting firm Navigant Consulting Inc. (NCI) to assist in preparing for and conducting the RFCC National Evaluation. NCI is experienced in conducting program evaluations and has assisted OWH in developing the survey instruments and evaluation methodology for the RFCC National Evaluation.

A.9 Payment/Gift to Respondents

Each RFCC has been provided $25,000 in OWH funding, which is meant to compensate the RFCCs, at least partially, for participating in the required data collection activities associated with the National Evaluation of the RFCC program, as well their local evaluations, and other similar assessment initiatives. This funding is to be used to compensate RFCC staff for the time required to respond to questionnaires and tracking forms, identify RFCC clients for data collection, gather client data via telephone, mail questionnaires to clients, and all necessary follow-up. These funds can also be used for materials and other expenses associated with these efforts.

A.10 Confidentialty

The health information of RFCC clients/patients will be treated in a private manner, unless otherwise specified by law. All personal identifiers appearing on Clinical Intake forms will be removed, as defined by HIPAA, by the provider organization (either the RFCC or an outside referral organization) prior to being released to NCI and the OWH.

Respondents of the RFCC Client/Patient Questionnaire will not be asked to enter any personal identifiers. Responses will be mailed anonymously to NCI’s evaluation team for data entry and analysis. Respondent contact information will be kept separate from questionnaire data in order to ensure anonymity of respondents. Each questionnaire will be assigned a random number for quality assurance purposes. Once the data are analyzed, these numbers will be deleted.

Personal identifiers will be removed by NCI from all information collected during site visit interviews and on the RFCC Tracking Sheet.

For all data collection instruments, including the RFCC Women’s Services Registration Form, the evaluation team will house data in secure electronic databases. Individuals involved with data collection will be fully informed of policies and procedures regarding confidentiality of survey and interview data. In addition, NCI routinely employs the following safeguards to ensure confidentiality:

• All employees sign a confidentiality pledge that emphasizes the importance of confidentiality and describes their obligations.

• Access to personal identifying information of sample members is limited to those directly responsible for providing the sample. Once the database has been completed and quality checked, these data are destroyed.

• Access to any file linking sample identification numbers with the respondents’ identification and contact information is available to only a small number of people who need to know this information. All files are kept on a secured drive with access carefully restricted.

A.11 Sensitive Questions

The RFCC Clinical Intake Forms contain questions that may be perceived as sensitive. The forms ask for detailed health history information, including sexual and reproductive health, and alcohol and substance use/abuse. These questions are included because of their importance in understanding the health attitudes, practices, and needs of RFCC clients/patients. Completion of this form is voluntary, and respondents will be allowed to skip items they prefer not to answer.

A.12 Burden Estimate

Exhibit 1 indicates the estimated hour burden for each data collection activity as well as the total estimated burden for all data collection activities that are part of the National Evaluation. The estimated burden hours were derived by evaluating the length and complexity of each data collection tool.

Exhibit 1. Estimated Hour Burden

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response	Total Burden Hours
RFCC Leaders	RFCC Center Director/Program Coordinator Questionnaire	11	1	2 hours	22 hours
RFCC Staff	RFCC Women’s Services Registration Form	300	1	15 minutes	75 hours
RFCC Staff	RFCC Clinical Intake Form (Provider RFCCs)	200	1	20 minutes	67 hours
RFCC Partners	RFCC Clinical Intake Form (Non-Provider RFCCs)	135	1	20 minutes	45 hours
RFCC Clients	RFCC Client/Patient Questionnaire	165	1	15 minutes	41 hours
RFCC Leaders	RFCC Site Visit	11	1	15 hours (15 interviews, each lasting 1 hour)	165 hours
RFCC Leaders	RFCC Tracking Sheet	11	1	5 hours	55 hours
	TOTAL	833		34 minutes	470 hours

Exhibit 2 indicates the estimated financial burden to respondents for the time it will take them to participate in this data collection effort. The financial burden was derived by identifying average hourly wage rates for each of group of respondents. Hourly wage rates were derived from the U.S. Bureau of Labor Statistics.

Exhibit 2. Estimated Cost Burden

Type of Respondent	Total Burden Hours	Estimated Hourly Wage Rate	Estimated Total Respondent Costs
RFCC Leadership	242	$39.36	$9,525.12
RFCC Partners (Primarily RNs)	45	$27.35	$1,230.75
RFCC Staff	142	$12.58	$1,786.36
RFCC Clients	41	$7.00	$287.00
TOTAL			$12,829.23

A.13 Capital Costs (Maintenance of Capital Costs)

Time and effort are the only burden to respondents. There are no capital costs associated with this collection.

A.14 Cost to Federal Government

The estimated cost for the administration of all data collection efforts is $85,290. Exhibit 3 illustrates the breakdown of these costs for each data collection activity. With the exception of travel expenses for the RFCC site visits, costs are based on the labor hours that will be required to conduct surveys, enter and analyze data, and report findings. These costs will be incurred only once, and will not span over multiple years.

Exhibit 3. Data Collection Costs of the Proposed Study

RFCC Center Director/Program Coordinator Questionnaire	Cost
Conduct Survey	$1,580
Analyze and Report Results	$2,600
Subtotal	$4,180
RFCC Women’s Services Registration Form	Cost
Data Entry	$3,500
Analyze and Report Results	$2,600
Subtotal	$6,100
RFCC Clinical Intake Forms	Cost
Data Entry	$3,500
Analyze and Report Results	$3,600
Subtotal	$6,100
RFCC Client/Patient Questionnaire	Cost
Administer Surveys	$3,160
Data Entry and Database Maintenance	$2,000
Analyze and Report Results	$2,600
Subtotal	$7,760
RFCC Site Visit Activity	Cost
Plan and Coordinate Site Visits	$2,400
Travel to RFCC sites	$11,000
Conduct Site Visit Interviews	$35,200
Data Entry and Database Maintenance	$2,000
Analyze and Report Results	$5,200
Subtotal	$55,800
RFCC Tracking Sheet	Cost
Conduct Survey	$2,750
Analyze and Report Results	$2,600
Subtotal	$5,350
GRAND TOTAL	$85,290

A.15 Program or Burden Changes

This is a new data collection.

A.16 Publication and Tabulation Dates

The time schedule for data collection for this evaluation is four months, once OMB clearance is obtained. Exhibit 3 includes timeframes for the major milestones of the National Evaluation

Exhibit 3. Project Time Schedule

Activity	Time Period
Obtain OMB Clearance	May 2007
Data Collection (including from sources not requiring OMB clearance)	May 2006 - August 2007
Data Analysis	August 2007 – September 2007
Submit Evaluation Report	September 2007

OWH’s plan for tabulation and analysis is detailed in the Evaluation Methodology (Attachment 1). This document includes in depth narratives of the analysis plan for each data collection instrument, as well as illustrative table shells where appropriate.

A.17 Expiration Date

The OMB expiration date will be displayed on all data collection instruments.

A.18 Certification Statement

There are no exceptions to the certification statement.

B. Collection of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods

The respondent universe and sampling method varies for each data collection tool. Detail for each data collection activity is provided below.

RFCC Center Director/Program Coordinator Questionnaire

The respondent universe includes Center Directors and Program Coordinators at each RFCC. One survey will be sent to each RFCC, and the RFCC Center Director and Program Coordinator at each site will be asked to work together to complete the survey. All eleven RFCCs will be asked to participate because data on each RFCC is integral in conducting a meaningful evaluation. This is especially important because each RFCC serves a very different client population. Sampling is not appropriate for this survey. The anticipated response rate for this survey is 100 percent because participation is required for continued RFCC designation and funding from the OWH.

Universe and Sample for RFCC Center Director/Program Coordinator Questionnaire

Respondent Universe	Sample Size	Expected Response Rate
11 Center Directors and 11 Program Coordinators (one center Director and one project coordinator per RFCC)	Not Applicable – the entire universe will be surveyed	100%

RFCC Women’s Services Registration Form

The respondent universe includes all women who access RFCCs. The number of women who access RFCCs varies substantially across the 11 organizations due to geographic and demographic factors, as well as specific objectives of the individual RFCCs. RFCC staff will make an effort to obtain completed registration information on every woman who accesses their RFCC. A response rate of nearly 100 percent is expected because one of the central missions of all RFCCs is to provide access to needed services for women. In order to provide access and referrals to appropriate services, RFCCs must collect the registration information. Although some women will refuse to provide complete information, most are expected to comply since they cannot obtain needed referrals to services otherwise.

Universe and Sample for RFCC Women’s Services Registration Form

Respondent Universe	Sample Size	Expected Response Rate
All women who access RFCC services (it is expected that roughly 300 women will access RFCCs during the data collection period	Not Applicable – the entire universe will be surveyed	100%

RFCC Clinical Intake Forms

The respondent universe includes women who access RFCCs and who receive clinical care services from direct care providers (either at the RFCC or at referral organizations).

Respondent selection will vary depending on whether the RFCC is a direct clinical care provider, or whether clients/patients are referred to outside providers for clinical care.

For the two RFCCs that provide direct clinical care, clinical intake forms will be completed for all clients/patients who access clinical services at the RFCC. Each RFCC will then identify a sample of 100 women, with 25 women for each of four disease states: cardiovascular disease, cancer, diabetes/obesity, and HIV/AIDS. These clinical indicators were selected from former DHHS Secretary Thompson’s priority prevention initiatives, which are also incorporated into current Secretary Leavitt’s 500-Day Plan. The RFCCs will identify the sample of women, and extract the pertinent data from their Clinical Intake Forms for analysis.

For the nine non-clinical care provider RFCCs, women are referred to outside providers for clinical care. RFCCs will obtain the clinical intake forms from the referral organizations for a sample of 15 women. These women will be selected randomly. Because it is expected that the number of women referred to clinical services will vary substantially across RFCCs, and that some women will not provide an informed consent to have their data used for research, the actual universe size is difficult to estimate at this point. However, the selection process will continue until 15 women from each site have provided their consent and the sample size of 165 women is reached. We estimate that we will achieve an approximately 80 percent response rate.

Universe and Sample for RFCC Clinical Intake Forms

Respondent Universe	Sample Size	Expected Response Rate
Women who access RFCCs and receive clinical care services from a direct care provider	165	80%

RFCC Client/Patient Questionnaire

The respondent universe is all individuals who are clients/patients of an RFCC. Every RFCC client/patient will have an equal probability of selection, as respondents will be selected using a random sampling methodology. Participation is voluntary, and no respondent will be selected more than once for participation in the survey. Clients will be sampled, with consideration of expected response rates, in order to receive a total of 15 completed questionnaires for each RFCC. Since it is expected that some respondents will decline to participate in this survey, the RFCCs will continue to sample women for participation until 15 surveys have been completed. It is expected that 165 surveys will be completed across all eleven RFCCs. The use of random sampling will enable the survey results from each RFCC to be generalized to that RFCC’s client/patient population. We expect an overall response rate of approximately 80 percent.

Universe and Sample for RFCC Client/Patient Questionnaire

Respondent Universe	Sample Size	Expected Response Rate
All individuals who access RFCCs	165	80%

RFCC Site Visit

The respondent universe will vary slightly at each RFCC, but will include individuals who fill the following roles:

• RFCC Center Director

• RFCC Program Coordinator

• RFCC Local Evaluator

• Institutional Commitment Individual

• Prominent partners

• Key community stakeholders; and

• Individuals who can address each of the five required components for RFCC designation (research, clinical care, outreach, professional education/training, and leadership/mentoring).

The number of individuals interviewed at each site visit is expected to range from 7 to 15. Site visits will be conducted at all eleven RFCCs. Sampling is not appropriate because there are too few RFCCs, and because each RFCC serves a very different client population.

Navigant Consulting, Inc. with assistance from each RFCC, will select partners and community stakeholders to interview during site visits, based on the frequency and level of interaction with the RFCC. For each partner or stakeholder organization chosen, a senior leader or liaison will be asked to participate in an interview.

Universe and Sample for RFCC Site Visit

Respondent Universe	Sample Size	Expected Response Rate
11 RFCCs	Not Applicable – the entire universe will be surveyed	100%

RFCC Tracking Sheet

The respondent universe includes Center Directors and Program Coordinators at each RFCC. One tracking sheet will be sent to each RFCC, and the RFCC Center Director and Program Coordinator at each site will be asked to work together to complete it. All eleven RFCCs will be asked to participate because data on each RFCC is integral in conducting a meaningful evaluation. This is especially important because each RFCC serves a very different patient population. Sampling is not appropriate for this data collection effort. The anticipated response rate is 100 percent because participation is required for continued RFCC designation and funding from the OWH.

Universe and Sample for RFCC Tracking Sheet

Respondent Universe	Sample Size	Expected Response Rate
11 Center Directors and 11 Program Coordinators (one center Director and one project coordinator per RFCC)	Not Applicable – the entire universe will be surveyed	100%

B.2 Procedures for the Collection of Information

The procedure for collecting data varies for each data collection instrument. Each data collection activity will occur one time only. The procedures to collect all data required for the National Evaluation of the RFCC program (not just those requiring OMB clearance), are detailed below:

Step 1:

RFCCs will begin using RFCC Women’s Services Registration Form upon OMB clearance. If possible, registration form data will be entered into an electronic database by the RFCCs. NCI will request that RFCCs submit all de-identified data on August 31, 2007. If RFCCs are unable to enter data into an electronic format, NCI will obtain copies of the paper forms (de-identified). NCI will create a “master” database of registration form data that will include data from all 11 RFCCs. This will allow for comparisons of women across RFCCs as well as analyses of demographic characteristics and the types of services most requested by women accessing RFCCs. Other similar “registration type” data that is included in RFCC quarterly and annual reports will be collected and used to the extent possible.

Approximate Time Frame: OMB Clearance Date (Expected May 2007) – August 31, 2007

Step 2:

RFCCs that are direct care providers, as well as referral organizations that are clinical care providers will continue to collect clinical intake data as they always have. RFCCs that are direct clinical care providers will collect this data from their internal information systems and will submit de-identified data to NCI. RFCCs that are not direct care providers will request clinical intake data from a sample of 15 women who are referred to clinical care providers. The data can be either electronic or hard copy (de-identified) and must be submitted to NCI by August 31, 2007.

Approximate Time Frame: July 2006 – August 31, 2007

Step 3:

NCI will review RFCC quarterly and annual reports. Using a data collection form that was developed to extract data from these reports, NCI will answer as many of the National Evaluation research questions as possible, and prepare a document of questions and preliminary answers for each RFCC. NCI will indicate where specific research questions cannot be answered, either completely or partially, and will continue to add to this question-and-answer document as new reports are received periodically. This activity will be completed on an ongoing basis prior to and during the waiting period for OMB clearance.

Approximate Time Frame: August 2006 - August 2007

Step 4:

RFCCs will begin using the RFCC Tracking Sheets upon receipt of OMB clearance. The Tracking Sheet is in Excel format, and RFCCs should enter data in the Excel spreadsheet and maintain data electronically. NCI will obtain the Excel files from the RFCCs by August 31, 2007. The information will used to create a “master” tracking sheet that identifies all activities and initiatives sponsored by and/or implemented by the RFCCs.

Approximate Time Frame: August 2006 – August 2007

Step 5:

Patient questionnaires will be administered to a sample of 15 women who access services at each RFCC, following OMB clearance. The survey will be mailed to the women and returned directly to NCI for data entry and analysis.

Approximate Time Frame: May 2007 – August 2007

Step 6:

Upon OMB clearance, NCI will begin scheduling site visits. Site visits are expected to be completed in one or two days over approximately a three month period. A conference call will be scheduled with Center Directors and/or Program Coordinators prior to the site visit to discuss logistics, specific individuals to be interviewed and other requirements.

Approximately Time Frame: May 2007 – August 2007

NCI will conduct a thorough review of all data received in order to identify and correct errors prior to analysis. In addition, NCI will assign unique codes to collected data to assist with data analysis.

As with the data collection process, NCI’s data analysis approach for the national evaluation will also be structured in order to accommodate the variation in data that will be collected across RFCCs. As such, four “levels” of analysis will be employed in the evaluation. They are described below.

Level 1:

This is an “overall” count of women who access the RFCC and receive referrals for services. This overall count will be collected from all 11 RFCC sites and will include all women who have accessed the RFCC since the RFCC was formed.

Level 2:

In most cases, regardless of the access point (clinic, phone, email/internet), RFCC staff will be able to complete the Women’s Health Registration Form for each woman who accesses the RFCC. This form includes limited demographic and health information, as well as list of potential service(s) that are of interest. Upon receipt of OMB clearance, each RFCC will collect baseline data from women who access their Centers using these forms. As described previously, the data will be entered in a master electronic database maintained by NCI and analyzed. The analysis will include frequency distributions that will enable NCI to build “profiles” of women who access RFCC services and identify similarities and differences in the types of women seeking services across RFCCs. The analysis will also enable NCI to determine which services are requested most and those requested least.

In addition to the Registration Form data, data collected from the Activities Tracking Sheet will also be used in the Level 2 analysis. NCI will again compute frequency distributions of the various activities that are cited in the tracking sheets and quantify the participation in these activities as much as possible.

Level 3:

Using the clinical intake form, each RFCC will collect data for a sample of at least 15 women who access clinical care services. These services will be accessed either through the RFCC itself (those that are direct care providers) or a referral/partner provider. RFCCs who are not direct clinical care providers will be responsible for obtaining the data from the providers who treat the 15 women. The primary objective of the Level 3 analysis is to expand on the analysis of the registration form data, because the clinical intake form data is more detailed. Although the clinical intake forms used at each provider site will be slightly different, we expect most to include similar data elements. These data will be aggregated into a single database but analysis will be conducted for each site as well as across sites.

Aggregated data will be used to generate both quantitative and qualitative analyses. Quantitative analyses will be based on counts and frequency distributions, while qualitative analyses will be based on observations in the data such as which health issues appear to be most common among women who access services.

Level 4:

For the two RFCCs that are direct care providers (Tennessee and Utah), detailed clinical data on four specific clinical indicators will be collected for a sample of women who access the RFCC. The clinical indicators have been selected from former DHHS Secretary Thompson’s priority prevention initiatives, which are also incorporated into current Secretary Leavitt’s 500-Day Plan. They are: cardiovascular disease, cancer, diabetes/obesity and HIV/AIDS. These RFCCs have or will have electronic mechanisms for collecting data. A sample of 100 women from each RFCC is proposed, with 25 women for each of the four disease states. The individual RFCCs will need to identify potential candidates and extract the data. Data points requested may vary depending on the exact data fields collected by their individual information systems. However, the data points are expected to include some of the following data elements:

Cardiovascular Disease⁴

Risk Factors

• Blood cholesterol levels

• High blood pressure

• Diabetes

• Smoking

• Diet

• Physical inactivity

• Overweight / Obesity

• Alcohol use

• Family history

Prevention and Treatment

• Controlling cholesterol and blood pressure

• Cigarette smoking cessation

• Physical activity

• Heart healthy diet

• Weight maintenance/reduction

• Controlling and preventing diabetes

• Evaluation and treatment of depression

Cancer⁵

Risk Factors

• Age

• Family history

• Exposure to environmental hazards (e.g. radiation, asbestos)

• Behavioral risk factors

• Tobacco use

• Sun exposure

• Diet

• Weight and physical inactivity

Screening and Diagnosis

• Mammography

• CT Scans, X-Rays

• Pap test

• Colonoscopy / Sigmoidoscopy

• Lab tests

Treatment and Care

• Surgery

• Chemotherapy

• Radiation

Diabetes/Obesity⁶

Risk Factors

• Overweight / Obesity

• Race / Ethnicity

• Family history

• High blood pressure

• Physical inactivity

Treatment and Care

• Physical activity

• Healthy eating

• Blood glucose monitoring

• Insulin injections and oral medication use

• Foot care

Related Health Concerns

• High blood pressure

• Heart disease or stroke

• High cholesterol

• Eye problems (e.g. blurry vision, blindness, cataracts, glaucoma)

• Kidney disease, kidney failure, and dialysis

• Nerve problems – numbness, pain and weakness in the extremities

• Problems with teeth and gums

• Depression

HIV/AIDS⁷

Risk Factors

• IV Drug use

• Unprotected sexual activity

• Sexually transmitted diseases such as chlamydia or gonorrhea

Prevention

• Risk counseling and education

• Condom use

Treatment

• Antiretroviral medication use

Special Issues for Women

• Pregnancy

• Childbirth

• Breastfeeding

A more detailed description of data sources, the proposed data collection approach, and the data analysis plan is presented in Attachment I, the Evaluation Methodology.

B.3 Methods to Maximize Response Rates and Deal with Nonresponsiveness

We do not expect to have significant difficulties with non-responsiveness for the following reasons:

• For the RFCC Center Director/Program Coordinator Questionnaire, we expect a 100 percent response rate because participation is required for continued RFCC designation and funding from the OWH.

• For the RFCC Women’s Services Registration Form, we expect a response rate of nearly 100 percent because, as described previously, RFCCs must collect the registration form data in order to provide access and referrals to appropriate services. Women who refuse to provide the registration information will not be able to fully utilize the services of the RFCCs. RFCC staff will encourage women to provide complete and useful information in order that their needs can be appropriately addressed.

• For the RFCC Clinical Intake Forms, women will be sampled until the required number of respondents is achieved.

• For the RFCC Client/Patient Questionnaire, additional questionnaires will also be mailed out in order to continuously sample until the desired sample size is reached.

B.4 Test of Procedures or Methods to be undertaken

No pretests of data collection procedures have been conducted. This is a baseline data collection.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Statistical Aspects Contact

Wilma Tilson

Senior Health Policy Analyst

US Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

Office of Health Policy

Phone: 202-205-8841

Email: Wilma.Tilson@HHS.GOV

Data Collection/Analysis and Statistical Contact

Dr. Henry Miller, PhD

Managing Director

Navigant Consulting Inc.

Phone: 410-528-4806

Email: hmiller@Navigantconsulting.com

1 Rural Frontier Women’s Health Coordinating Center. HHS Awards $2.86M for Transformation of the National Centers of Excellence in Women’s Health Program (CoE). October 4, 2004.

2 U.S. Department of Health and Human Services. HHS Rural Task Force Report to the Secretary. One Department Serving Rural America.

3 The Association of Maternal and Child Health Programs. From Rural to Remote America, Family Health Care in Alaska, Idaho, Oregon and Washington. March 2004

4 Data points compiled from the Centers for Disease Control and Prevention’s Division for Heart Disease and Stroke Prevention website: http://www.cdc.gov/dhdsp/index.htm

5 Data points compiled from the National Cancer Institute’s website: http://www.cancer.gov/cancertopics

6 Data points compiled from the Centers for Disease Control’s FAQa page on Diabetes: http://www.cdc.gov/diabetes/faq/index.htm

7 Data points compiled from the Centers for Disease Control’s HIV/AIDS Prevention website: http://www.cdc.gov/HIV/dhap.htm