FINAL Supporting Statement B

Supporting Statement B

Teaching Health Center Graduate Medical Education (THCGME) Program Cost Evaluation

OMB Control No. 0906-XXXX

B. Collection of Information Employing Statistical Methods

If statistical methods will not be used to select respondents and the corresponding item is checked “No”, this section does not have to be completed.

The decision not to use statistical methods to select respondents to the Teaching Health Center Graduate Medical Education (THCGME) Costing Instrument was based on several factors:

• the descriptive approach of the contract;

• the small universe of potential respondents;

• the diversity of the programs, which requires a larger group of respondents to develop a comprehensive understanding of the differential costs associated with operating a THC residency program.

Because the HRSA THC Evaluation, which is being conducted under contract with the George Washington University (GW), will be the most recent effort to collect data on THC residency program costs, statistical methods will not be used to create a sample. The descriptive approach of the contract seeks to gather standardized data on direct and indirect operational expenses across the residency programs in order to compile data on the direct and indirect operational expenses associated with sponsoring an accredited graduate medical residency training program in a community-based setting. Collecting standardized data from all THC residency programs will enable GW to identify program factors that contribute to variations in DME and IME costs across programs as well as other relevant financial data. THCs currently operate with an annual Per Resident Amount set at $150,000. With comprehensive data from all THCs, GW will be able to provide HRSA with a more accurate estimate of the costs of THC resident training. As such, the estimate will help HRSA determine an accurate level of THCGME payments.

Collecting data from all THC residency programs with the THCGME Costing Instrument will increase the robustness of the final cost estimate. Data from the 56 THCs will be collected and will provide timely, accurate and useful data for estimation purposes. THCs are diverse in their organizational, financial and educational structures. Some are established residency programs that used the THCGME funding to expand, while others are new primary care residency programs that exist only because of the THCGME funding. As such, the characteristic differences of THCs may contribute to overall differences in residency training costs. Sampling from such a small universe would have the potential to yield estimates that are not representative of the full range of THCs. For these reasons, we will collect financial information from all operating THCs and will use population parameters rather than sample statistics to develop the cost estimates.

1. Respondent Universe and Sampling Methods

Residency programs receiving THCGME funding constitute the respondent universe, and will be asked to complete the THCGME Costing Instrument one time. As noted above, no sampling methods will be used. The respondents for the data collection are the individual THC residency programs because the information collected includes the operational costs and associated clinic costs at the residency program level. The expected response rate for data collection is 100% (56).

Table 1: Respondent Universe

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Total Responses
THC Program Directors and/or Financial Officers	Teaching Health Center Costing Instrument	56	1	56

The GW team will use a multi-stage approach to reach out to all THC residency programs to communicate with them about the purpose and importance of the information collected in the THCGME Costing Instrument. GW will provide information about the timing for data collection, including the distribution of the THCGME Costing Instruments, dates for technical assistance webinars, and key deadlines. GW will encourage participation from the maximum number of THCs through the following outreach strategies:

• The GW team will gather information from THC program directors to compile basic background and context information to make virtual site visits as efficient as possible.

• The GW team will distribute the THCGME Costing Instrument via e-mail to the identified program leadership of all THCs. The template for this email can be found in Appendix A.

• The GW team will conduct virtual site visits with the THCs to jointly review the THCGME Costing Instrument and support participation.

• The GW team will provide technical assistance webinars to the THCs throughout data collection to encourage participation (please see Section 3 “Methods to Maximize Response Rates and Deal with Nonresponse” for more information).

2. Procedures for the Collection of Information

The standardized THCGME Costing Instrument and an accompanying instruction guide (see Appendix B) will be distributed once to all HRSA funded THC residency programs. The THCGME Costing Instrument will be a fillable excel workbook delivered via e-mail to the identified THC leadership. Contact information for THC program leadership will be available from HRSA. THCs will determine which persons in their leadership are best suited to be contributors and the signatory for the THCGME Costing Instrument. Because of the variation in leadership team composition across THCs, multiple people in the THC program may contribute to the completion of the THCGME Costing Instrument. All completed forms must

be reviewed and signed by the THC Program Director. Completed forms will be exported to a password-protected Excel database.

2. Methods to Maximize Response Rates and Deal with Nonresponse

In previous evaluations, THCs have demonstrated strong participation. A previous study estimating the cost of residency training fielded responses from 36 HRSA-funded THC residency programs out of a potential 43 programs eligible for analysis. Programs’ leadership teams have demonstrated a high level of commitment to the THCGME initiative and widespread knowledge of the importance of collecting data for the evaluation contract. The following methods will be used to maximize response rates:

• The GW team will host technical assistance webinars to facilitate programs’ completion of the THCGME Costing Instrument and maximize response rates. GW will compile a list of likely data elements and other examples that THCs might need assistance with, and will use this information alongside fielded questions from THCs to facilitate an efficient and high-quality data reporting process. The webinar will be geared primarily toward program directors and programs’ financial analysts; programs may identify other staff who may participate as well. GW will make available supporting documents or resources from the technical assistance webinars to all programs.

• The GW team will conduct virtual site visits with individual THCs to review the entire THCGME Costing Instrument and respond to any questions or difficulties completing the worksheets or identifying the appropriate information. The GW team expects that many of the questions about completion of the THCGME Costing Instrument will be related to a THC’s unique characteristics or operations.

• The GW team will be available for additional technical assistance for individual THCs that are having difficulties completing sections of the THCGME Costing Instrument. The goal will be to facilitate completion of the data collection and to assure that the data are as comprehensive as possible. The GW team expects to interact with all programs at various points of time during completion of the THCGME Costing Instrument.

It is expected that the technical assistance to the programs described above will be instrumental in promoting THCs’ responses to the THCGME Costing Instrument. GW will periodically contact non-responders to offer technical assistance and encourage them to complete the THCGME Costing Instrument, which will help reduce overall non-response rates. At the same time, some analysis of non-responders will be possible. Non-response can be analyzed based on program characteristics, for example, if particular programs or types of programs have higher non-response than others. As participation is voluntary, there will be no penalty for non-response.

2. Tests of Procedures or Methods to be Undertaken

The THCGME Costing Instrument reflects the analysis and input of the GW evaluation team, which includes a panel of GME experts and a GME program/financial consultant who has been advising the project throughout the planning and development of the THCGME Costing Instrument. The original version of the THCGME Costing Instrument, developed for a previous cost estimate, was based on a literature review, three site visits to THCs, and extensive review from GME experts and the GW team. This updated THCGME Costing Instrument has been refined by the GW team based on its experiences with data collection and analysis of 26 THCs in a prior THC cost evaluation that demonstrated the effectiveness of the original THCGME Costing Instrument. As part of the updating process, the GW team convened a GME Panel of Experts to recommend revisions to the original THCGME Costing Instrument. In addition to minor revisions to clarify and streamline data collection, the updated THCGME Costing Instrument includes new questions to better understand THC operations to uncover the indirect costs associated with THC residency programs. The collected data from the updated THCGME Costing Instrument will be reviewed and validated by two GME financial auditors with extensive knowledge of GME and ambulatory care settings, and this will ensure that the data collected is accurate and without errors or missing information.

2. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The person primarily responsible for data collection design, collection, and analysis is:

Marsha Regenstein, tel. 1-202-994-8662, email marshar@gwu.edu, Professor, Department of Health Policy and Management, The Milken Institute School of Public Health, The George Washington University, 950 New Hampshire Avenue, NW, Washington DC, 20037.

1