Revised Supporting Statement

HRSA Quality Toolkit Customer / Grantee Satisfaction Survey

The Health Resources and Services Administration (HRSA) currently has approval under the generic clearance, OMB No. 0915-0212, to conduct customer satisfaction surveys. These surveys help fulfill the requirements of:

1. Executive Order 12862, “Setting Customer Service Standards,” which directs Agencies to continually reform their management practices and operations to provide service to the public that matches or exceeds the best service available in the private sector.

2. The March 3, 1998, White House Memorandum, “Conducting Conversations with America to Further Improve Customer Service,” which further directs Agencies to determine the kind and quality of service its customers want and their level of satisfaction with existing services.

HRSA is requesting approval to conduct a survey under this generic clearance to get customer/grantee feedback for a web based toolkit that is going to be provided to grantees supporting the adoption of the HRSA Core Clinical Quality Performance Measures and continuous quality improvement. HRSA has begun implementation of a Clinical Quality Core Measure Set of 12 performance measures as part of a clinical quality measurement and improvement initiative.  The initiative underscores HRSA’s commitment to quality improvement and begins to measure and demonstrate the quality of care across the Agency. The Core Clinical Quality Performance Measures are available on the web at http://www.hrsa.gov/quality/coremeasures.htm. The core measures address priority health conditions of HRSA safety-net populations, cover all life cycles, are amenable for quality improvement, and were selected for their relevance to HRSA programs.

With the recent adoption of the set of 12 HRSA-wide Core Clinical Measures, a need for related technical assistance (TA) for impacted grantees is anticipated to properly and effectively implement the future use of these Core Measures and reporting to HRSA. HRSA currently supports a variety of existing vehicles and infrastructure for the delivery of TA to grantees. A planned approach was needed for the consistent alignment and supplementation of existing TA content related to the implementation and improvement of the HRSA-wide Core Clinical Measures. HRSA recognized the need for a toolkit with objective educational tools and resources to assist organizations in their quality improvement (QI) efforts. In our commitment to maintaining the high standards and usefulness of the toolkit, OMB clearance is requested to gain user (grantee) feedback regarding the information, tools, resources and functionality of the toolkit.

A. Product/Activity to be assessed:

The HRSA Quality Toolkit provides educational tools and resources to assist an organization in its quality improvement (QI) efforts focusing specifically on the HRSA Core Clinical Quality Performance Measures. The HRSA Quality Toolkit is equipped with proven strategies and techniques to effectively support an organization’s new or existing QI program. The toolkit’s wide range of practical, convenient, and useful tools were contributed and compiled from several HRSA organizations, where members have proven their usefulness in current QI health care delivery initiatives. The HRSA organizations that contributed tools and resources for this toolkit are:

• HIV/AIDS Bureau (HAB)

• Maternal and Child Health Bureau (MCHB)

• Office of Rural Health Policy (ORHP)

• Healthcare Systems Bureau (HSB)

• Bureau of Primary Health Care (BPHC)

The tools and resources were used for implementing data collection and performance measurement techniques, and the organizations that integrated their use into QI programs effectively improved their deliveries of care. These resources are adaptable and may be used to facilitate the implementation of a new QI program or fine tune an existing one. The HRSA Quality Toolkit:

• Describes various systematic approaches to QI for multidisciplinary teams

• Provides practical and actionable strategies to improve care outcomes in various settings and within diverse constraints

• Explains the importance of integrating evidence-based medicine within a QI program

• Demonstrates the correlation between cultural competency and quality health care delivery

• Provides examples and learning cases of common QI scenarios

• Points to tools and resources external to the toolkit that an organization can access to further advance its quality objectives

• Offers technical assistance for implementing the HRSA Core Clinical Measures (CCMs)

The HRSA Quality Toolkit addresses the needs of any QI leader who is responsible for effectively implementing a quality improvement program into its organization. There are several members within a health care organization that may find this toolkit particularly useful:

• Providers, including doctors, dentists, advanced practice nurses, and physician assistants

• QI directors and administrators

• Allied Healthcare Professionals, including nutritionists, social workers, dietitians, behavioral specialists, and other health care professionals

The toolkit addresses the needs of any organization that is interested in integrating QI into its delivery system. Whether it is the provider, QI director, or social worker who is at the preliminary stage of learning basic QI concepts, or a veteran QI team that is seeking the latest performance measurement strategies, the toolkit contains an array of beneficial tools to provide varying levels of assistance. The HRSA Quality Toolkit provides an organization with the tools and resources to support a successful QI program, because it:

• Introduces the HRSA CCMs and their uses for evaluating and measuring quality of care within an organization

• Explains basic QI concepts to an organization that is new to QI

• Demonstrates how to integrate a QI program with an organization’s established goals for improving quality of care

• Highlights the importance of staff involvement and provides guidance on actively involving staff in a QI program

• Helps an organization to develop an effective plan for collecting and analyzing data

The principles described in the toolkit can be generalized and applied to health conditions and performance measures other than those presented, and often they can be integrated into an organization’s existing QI program. Throughout the toolkit, there are learning cases that demonstrate realistic applications of QI approaches by organizations that integrated the toolkit’s tools and techniques into their health care delivery settings. The learning cases provide actual scenarios to help an organization to conceptualize how to effectively use these QI strategies in its own setting.

Additionally, the HRSA Quality Toolkit is designed to support the varying needs of a large body of organizations, and it provides relevant information, resources, and tools in multiple formats to accommodate those needs:

• A printable Adobe Portable Document Format (PDF) file that contains a comprehensive description of QI concepts

• Autonomous modules, suitable for training purposes, are provided in printable or web-accessible formats

• A web portal that allows users to easily navigate through the toolkit, with embedded hyperlinks to other important online tools and resources

The purpose of the proposed survey is to ensure that intended audiences find the information provided within the various modules in the HRSA Quality Toolkit easy to access, clear, informative, and useful. Additionally, the survey will examine whether the information is presented in an appropriate technological format and whether it meets the needs of users of the HRSA Quality Toolkit.

The HRSA Quality Toolkit project development team will use the survey responses to review the extent to which toolkit users are satisfied with the overall learning experience as well as satisfaction with specific aspects of the toolkit regarding format, usefulness of subject matter, usefulness of tools and resources, and needed improvements to the toolkit.

Survey findings regarding toolkit user satisfaction will provide information for the HRSA Quality Toolkit project development team that will be used to make recommendations for improvements to the refinement of content and format to modules, if necessary, and to develop plans for future modules development. The project team will review the descriptive findings and discuss how best to address matters identified by the survey.

1. Data Collection:

This survey will be available for customer/grantee participants within each module contained in the HRSA Quality Toolkit test website. The survey will be available on the website at the end of each module. Since the web site is a test web site it will not be publically available but only to those volunteering to review and provide comment on the various modules within the HRSA Quality Toolkit.

The respondent universe will be those individuals who volunteer to review the HRSA Quality Toolkit. The total number of individuals expected to volunteer to review each module is approximately 250 and the project team anticipates approximately an 80% response rate for the satisfaction survey. In an effort to encourage feedback, the survey has been designed to be extremely brief and easy to complete. This activity is not designed nor intended to be a statistical survey or to make any generalizations about a population; its intent is to get feedback on the toolkit format, content tools and resources. The instrument consists of 7 questions. The intent is that the survey can be completed in 4 minutes. No individual or personal information is requested. It is estimated that, on average, each respondent would review 3 modules. Therefore, tThe total burden for the customer/grantee survey per module within the Quality Toolkit is:

Form	Number of Respondents	Responses per Respondent	Total Responses	Burden Hour per Respondent	Total Burden Hours
Customer Survey	250	3	7250	4 minutes	5017
Total	250		7250

Note: There are a total of 26 modules within the HRSA Quality Toolkit with a total possible burden per customer/grantee of 104 minutes. Therefore, the total estimated burden for all 250 customer/grantee survey respondents is 442 hours.

2. Planned frequency of information collection:

The project plans to request grantees across HRSA's various offices on a volunteer basis to review and comment on the HRSA Quality Toolkit. As volunteers are identified, a web link will be provided to the toolkit and survey via email. The project team proposes giving grantees appropriate time to review the toolkit as they are available and provide comment. Grantees will not be required to review each module but will provide comment on the module of their choice. To assure all modules receive appropriate review and comment, grantees will be asked in advance to identify the modules they wish to review. The customer/grantee satisfaction survey instrument will be available within the web toolkit at the end of each module in order to provide participants with the opportunity for input on each module reviewed.

3. Time period over which information will be collected:

The HRSA Quality Toolkit and surveys will be available to customers/grantees for two weeks over a 6 month period. The project team envisions the identification of volunteer grantees agreeing to review the modules to be a "rolling" process. Therefore individuals will be given 2 weeks to review and comment on a module once provided the web link and pass word. However it should be noted that all volunteers may not be identified at the same time. Identification of volunteers will be an ongoing process to last no longer than 6 months for a goal of 6,500 participants reviewing 26 modules. The survey will be available following the review of each module to obtain feedback from reviewers on their satisfaction with the toolkit content, format and usefulness of the information, tools and resources provided.

4. Questionnaire Instrument

Questions were developed using models of other brief satisfaction surveys and examples from HRSA and NIH to address the various areas the broadcasts will cover. The design of the questions follows the format of previously approved customer satisfaction instruments, such as:

The Office of Rare Disease (ORD) Web Site User Survey.

National Institute on Deafness and Other Communication Disorders Web Site User Survey.

National Cancer Institute Office of Cancer Complementary and Alternative Medicine Website Feedback.

National Heart, Lung, and Blood Institute Online Survey.

Questions for feedback on the toolkit were modeled after these existing instruments, and were reviewed by individuals on the workgroup for content and clarity. The survey consists of 7 generic questions typical of customer satisfaction surveys that ask participants how useful the information provided within the toolkit was. Some of the questions have a Likert-type format, some are fixed categories and some provide comment boxes.

5. Analysis

The information obtained from this survey is not designed nor intended to yield meaningful quantitative findings; the responses provided by reviewers will provide meaningful customer/grantee input for internal use by the project team. As an opinion survey, the feedback provided will provide the project team with information on what the toolkit users think of the overall format and content shared within each module. Responses to the questions will be summarized to provide descriptive information on reviewer opinions to help determine usefulness of the toolkit format and content as the toolkit moves to its refinement stage.

The contact for this activity is:

Amber Berrian, MPH

Public Health Analyst

Center for Quality, Office of the Administrator

Health Resources and Services Administration

5600 Fishers Lane, Rm 7C-100 Rockville, MD 20857

phone: 301/443-0845