HIV Workforce Survey - Supporting Statement Part B - Revised (05-25-12)

HIV Clinician Workforce Study

Supporting Statement Part B:
Collection of Information Employing Statistical Methods

Revised

March 25, 2012

HIV/AIDS Bureau

Health Resources and Services Administration

United States Department of Health and Human Services

Parklawn Building, Suite 705

5600 Fisher's Lane

Rockville, MD 20857

CONTENTS

1. Respondent Universe and Sampling Methods 1

2. Procedures for the Collection of Information 5

3. Methods to Maximize Response Rates and Deal with Nonresponse 10

4. Tests of Procedures or Methods to Be Undertaken 12

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 13

REFERENCES 14

ATTACHMENTS

ATTACHMENT A: Design Report

ATTACHMENT B-1: HIV WORKFORCE SURVEY CLINICIAN QUESTIONNAIRE

ATTACHMENT B-2: Example of Web Survey

ATTACHMENT B-3: HIV WORKFORCE survey practice questionnaire

ATTACHMENT C-1: Informational BROCHURE FOR Clinician SURVEY

ATTACHMENT C-2: Informational BROCHURE FOR Practice SURVEY

ATTACHMENT D: respondent materials

D-1: HRSA Advance Letter to Clinicians

D-2: HRSA Advance Letter to Practices

D-3: Contractor Survey Packet Cover Letter to Clinicians

D-4: Contractor Survey Packet Cover Letter to Practices

D-5: Prompt Letter #1 from Contractor to Clinicians

D-6: Prompt Letter #1 from Contractor to Practices

D-7: Prompt Letter #2 from HRSA to Clinicians

D-8: Prompt Letter #2 from HRSA to Practices

D-9: Prompt Letter #3 from Contractor to Clinicians

D-10: Prompt Letter #3 from Contractor to Practices

D-11: URL/password sheet for clinician survey

D-12: Thank You Letter to Clinicians

D-13: Thank You Letter to Practices

D-14: promotional material

ATTACHMENT E: question sources

ATTACHMENT F: pretest memorandum

TABLES

Table 1: 95 Percent Confidence Intervals for HIV Clinician and Practice Surveys 6

Table 2: Schedule of Information Collection Activities 9

1. Respondent Universe and Sampling Methods

Commercial list coverage. We intended to include a copy of the design report as Attachment A in the Part B Supporting Statement. The design report, which incorporates the feedback we received from the technical experts who attended our expert consultation meeting, provides greater detail on the design and implementation of the aggregate supply and demand model. It also explains how we will use the survey responses to develop the input parameters that more closely reflect the workforce patterns of clinicians who manage care for patients with HIV than can be obtained from the general workforce literature. We have included a copy of the design report with these responses.

To evaluate the completeness of the SDI claims and the appropriateness of the data for identifying the universe of clinicians who manage care for a significant number of patients with HIV on an on-going basis, we asked our contractor to conduct an ex-ante review of the claims database and to summarize the findings in a technical memorandum, which we received on March 25, 2011 and include with these responses. SDI collects and maintains a warehouse of both pharmacy (RX) and medical (DX) claims from all payer sources, including managed care plans, billing providers, and geographic regions. The RX database includes electronic final-action claims submitted primarily by retail pharmacies. Although the RX file likely includes a large and nationally representative group of retail pharmacies, specialty pharmacies and pharmacy claims for mail-order prescriptions are underrepresented in the database. The RX file captures approximately 50 percent of all electronically transmitted pharmaceutical records in the country and includes between 120 and 130 million covered lives. The DX file includes medical claims transmitted electronically between providers and payers via third-party “transaction houses” or medical practice management companies. The DX database captures approximately two-thirds of all electronically filed medical claims in the country, includes roughly 1.1 billion records per year, and represents about 157 million covered lives.

The 2011 RX and DX files contain 89,638 physicians in family and general practice medicine (representing 84 percent of those listed in the AMA master file); 89,845 physicians in internal medicine (representing 66 percent of those listed in the AMA master file); and 5,453 physicians in infectious disease medicine (representing 66 percent of those found in the AMA master file) (see Table 1 in technical memorandum dated March 25, 2011). Because the AMA master file includes approximately five percent of physicians who are not engaged in direct patient care, the effective treatment universe counts are likely smaller than shown in Table 1, which increases the effective coverage of the SDI claims database. The remaining tables in the March 25, 2011 technical memorandum show the number and distribution by payer and state of clinicians with at least one HIV-related claim. The RX file contains 27,885 clinicians who prescribed at least one HIV prescription, representing 536,956 projected patients, in 2010 (see Table 2). The DX file contains 70,289 clinicians who submitted at least one medical claim with an HIV-related diagnosis code (see Table 5A). We circulated these state-level counts among several program officers for our state grants and they confirmed that the counts are consistent their understanding of the number of HIV clinicians in their states. In addition, the projected number of HIV patients in care based on the RX file (536,956) is consistent with the treated prevalence of HIV nationally (see Table 2).

To assess the completeness of the source after identifying the survey frame, we asked our contractor to compare the list of HIV clinicians identified from our analysis of SDI claims with the membership lists of the two HIV medical societies in this country: the HIV Medicine Association (HIVMA), an affiliate of the Infectious Disease Society of American, and the American Academy of HIV Medicine (AAHIVM). We received the results of the ex-post analysis of the survey frame in a technical memorandum dated January 25, 2012. (A copy of the technical memorandum is also included with these responses.) Eighty-five percent of the 3,931 members of HIVMA or AAHIVM who met our medical profession and specialty criteria were identified through our analysis of SDI claims. (See Table 5 in the January 25, 2012 technical memorandum.) The percentage of HIV medical society members who were captured on the SDI claims file was similar for physicians and nurse practitioners/physician assistants. Forty-eight percent of the member physicians who were identified on our claims file fall into our high-volume group (that is, treated 10 or more HIV patients in 2010), compared with 40.0 percent among nurse practitioners and physician assistants. We shared these findings with the directors of HIVMA and AAHIVM, and they confirmed that the results of the SDI claims analysis are consistent with their understanding of the number of clinicians managing HIV care on an on-going basis. The directors explained that approximately 15 percent of their members focus on research, work in industry, serve as pharmacists, are in medical school, and/or have other roles where they do not provide or only do minimal patient care.

One limitation of the use of claims data to identify the baseline supply of HIV clinicians is that claims data will not capture clinicians who do not bill for their services under their own names, such as many nonphysician clinicians. We will address this limitation through the HIV practice survey. This survey will be sent to a sample of the practices in which the sampled clinicians practice. To estimate the number of nonphysician clinicians providing HIV services who are not billing independently, we will include the following question on the survey:

• We are interested in the number of clinician FTEs in this clinic and the share of these FTEs that is allocated to caring for patients with HIV or AIDS. In column A, please indicate the number of clinician FTEs in this clinic providing patient care in general. In column B, please indicate the number of clinician FTEs devoted to HIV patient care.

Number of FTE Clinicians in Total and HIV Care

Type of Clinician	Column A Number of FTE clinicians in total patient care	Column B Number of FTE clinicians in HIV patient care
Infectious disease specialists
Primary care physicians
Physician assistants
Nurse practitioners

Note: Primary care physicians include internal medicine, family/general medicine, pediatrics, and geriatrics.

We will use responses to this survey question to estimate the ratio of nonphysician HIV clinicians to physicians providing HIV services. We will assess the variation in this ratio across practice settings and geographic areas (for example, regions and urban versus rural areas) and incorporate it into the baseline estimate of nonphysician clinician supply. Then, the baseline supply of nonphysician clinicians nationally will be calculated for each geographic areas and practice settings and nationally, based on the number of sampled physicians multiplied by this ratio.

Clinician survey. The Health Resources and Services Administration (HRSA) will construct a sampling frame of HIV clinicians (including physicians, nurse practitioners, and physician assistants) based on claims data provided by SDI Health LLC. SDI is a national for-profit health care analytics organization offering database and data warehousing services to private- and public-sector clients. HRSA will use SDI’s medical and prescription claims data warehouse (using claims from July 1, 2010, through June 30, 2011) to identify all clinicians who provide and bill for services to and write prescriptions for patients with HIV or AIDS. For the clinician survey, the contractor will draw a random sample of 5,000 clinicians who provide and bill for services to at least approximately 20 patients with HIV or AIDS.¹ Before selecting the clinician sample, we will send the list of all eligible clinicians (using a set of unique provider identifiers, such as their national provider identifier [NPI] number or their Drug Enforcement Agency [DEA] number) to SK&A. SK&A is a national for-profit data warehousing organization providing continually updated contact information on health care professionals, including doctors, nurse practitioners, and physician assistants. SK&A will append the variables needed to stratify the sample as described below. After selecting the clinician sample, SK&A will append clinician contact information, including mailing addresses and telephone numbers, to the list of sampled clinicians. We expect to achieve a response rate of 70 percent, giving us approximately 3,500 completed clinician surveys.

We will select the 5,000 clinicians using explicit and implicit stratification to help ensure that the sample chosen is reflective of the entire sampling frame. Explicit strata are distinct groups within which samples are chosen and can be based on one or a combination of characteristics. We will use them to ensure that each subgroup’s proportion in the clinician sample is the same as its proportion in the clinician population. Implicit stratification means that we will sort the sample frame by one or more characteristics within explicit strata to further control the representativeness of the sample on those characteristics. We plan to use a sequential sampling procedure that makes independent selections within each of the sampling intervals, while controlling the selection opportunities for units crossing interval boundaries (Chromy 1979). This procedure allows for explicit and implicit stratification and offers all the advantages of a systematic sampling approach, but it eliminates the risk of bias associated with that procedure.

To ensure that we capture a representative sample of providers, we will explicitly or implicitly stratify the sample by some or all of the following clinician characteristics: (1) age group; (2) gender; (3) race/ethnicity; (4) census region; (5) urbanicity; (6) health profession (physician, nurse practitioner, and physician assistant); (7) medical specialty (internal medicine, family/general medicine, and infectious disease); and, if feasible, (8) type and size of clinic. We do not plan to oversample nurse practitioners and physician assistants because we expect to have a sufficient number of sample members to estimate input parameters for this group of clinicians separately. We will select a larger sample than we expect to need, then divide this sample into random replicates that can be released as needed in waves to achieve the desired number of responses while still maintaining a probability sample.

Relationship between Survey Data and Supply and Demand Model. The main purpose of the provider and practice surveys is to develop stable estimates that can be input as parameters into the much larger aggregate supply and demand model developed by The Lewin Group. The seven research questions listed under Part A will be answered by this larger model, and not by the survey responses among the 3,500 responding clinicians and 350 responding practices. If the sample sizes permit, the survey data at the clinician and practice levels might be used for exploratory modeling and to help determine how to subdivide the population when generating these parameters. However, the sample sizes have not been engineered for this purpose. After reviewing the clinician sample frequencies, we do not plan to oversample any clinician subgroups. As can be seen in Table 1 (see section B.2.a, Statistical Procedures), estimates from some of the smallest subgroups, such as infectious disease specialists in non-metropolitan areas, will be stable enough to produce parameter estimates for the provider model. All estimates shown on the table have a relative standard error less than 30 percent, which is a minimum standard used by federal statistical agencies, including the Bureau of the Census (U.S. Census Bureau). We also plan to select the practices with probability inversely proportional to their chance of being included in the practice frame, which eliminates any design effect that would have diminished the precision of estimates resulting from the practice survey.

Practice survey. Using the list of 5,000 selected HIV clinicians with the claims data and Provider360 characteristics appended, we will create a de-duplicated list of all primary practice settings in which the sampled clinicians work. (From our prior claims analysis, we will also know the number of clinicians and patients associated with each practice.) Although the frame for the practice survey will be defined at the clinician level, each record will include a unique identification number for the site at which the clinician treats patients, which we will use to create a de-duplicated list of clinics. The clinician records also contain the name, mailing address, and telephone number for his/her group practice. From our sample of 5,000 clinicians, we will draw a random sample of 500 practices for the practice survey. Practices will be selected with probability inversely proportional to their chance of being included in the practice frame, which eliminates any design effect that would have diminished the precision of estimates resulting from the practice survey. The measure of practice size is the number of HIV clinicians associated with the practice on the clinician survey sample frame. To ensure we obtain a representative sample of clinics, we will stratify the sample by the number of clinicians practicing in each site, its geographic location, and, if feasible, the type of clinic.

As with the clinician survey, the main purpose of the practice survey is to develop stable practice-level estimates that can be input as parameters into the much larger aggregate supply and demand model developed by The Lewin Group. The research questions listed under the Part A supporting statement of this package will be answered by this larger model, and not by the survey responses among the 350 responding practices. If the practice survey sample size permits, we might use the survey data at the practice level for exploratory modeling and to help determine how to subdivide the population of practices when generating these parameters; however, we did not engineer the practice survey sample size for this purpose.

We will conduct a systematic approach to identify a clinician’s most likely practice address. First, we will identify clinician’s who share the same individual mailing address. For these cases, we will mail the practice survey to the clinicians’ shared individual address. In cases where the clinician’s individual address is unique, we will compare it to the clinician’s group practice address listed on the frame. If the clinician’s individual and group practice addresses are the same, we will mail both the clinician survey and the practice survey to the clinician address. If the clinician’s individual address is unique and does not match his or her group practice address, we will conduct a web search to determine the clinician’s physical practice location, as the clinician address on record may be a residential address. Based on our review of the sample frame, we anticipate the smallest number of practices will be in this latter group. We describe this procedure in our revised Supporting Statement B, Item 1 ‘Respondent Universe and Sampling.’ Depending on the success of our web search, we might consider purchasing practice address information from a health care database vendor such as SK&A (www.skainfo.com). Practice address information from SK&A is publicly available, at cost.²

The design report for the HIV clinician workforce study, presented in Attachment A, provides a more detailed discussion of the respondent universe and sampling technique for both the clinician and practice surveys.

2. Procedures for the Collection of Information

a. Statistical Procedures

We described our methods for stratification and sample selection in the previous section. We plan to calculate analysis weights for purposes of estimation. These weights will account for the probability of selection of each provider and for differential nonresponse patterns, and will enable us to generalize estimates based on the responding sample to the reference population. Weighted estimates will be made using specialized survey software procedures that properly account for the complex sample design (stratification, unequal weighting, and – for the physician sample – clustering within practice). These procedures use a Taylor series linearization approach to account for design complexities when calculating the variance of estimates.

As described above, the main purpose of the clinician and practice surveys is to develop stable parameter estimates for the larger aggregate supply and demand model. We did not design the survey samples to test hypotheses having to do with a comparison of subgroups within each of these surveys. The projected completed sample size of approximately 3,500 responding clinicians will enable us to estimate parameters with the precision given in Table 1 for the full sample and for certain subgroups within the sample. We incorporated a finite population correction factor into our variance estimates resulting in increased precision.

The numbers in Table 1 relating to clinicians are based on the actual clinician sample frame that we plan to use to select our sample. The numbers in the table relating to practices are approximations based on the clinician distributions, because we have not yet reached the point of aggregating the clinician records to the practice level. Looking at Table 1, if we, for example, detect a retirement rate for the full sample of 10 percent, we are 95 percent certain that the true underlying retirement rate is 10 percent plus or minus 0.8 percentage points. Similarly, among primary care physicians, if we detect a retirement rate of 20 percent, we are 95 percent confident that the true retirement rate for primary care physicians is 20 percent, plus or minus 1.3 percentage points. There are no unusual statistical problems to be addressed.

Table 1. 95 Percent Confidence Intervals for HIV Clinician and Practice Surveys

Subgroup	Sample Population	Selected Sample	Completed Responses	FPC	p = 0.1 or 0.9	p = 0.3 or 0.7	p = 0.5
Clinician Survey
All	9,133	5,000	3,500	0.617	0.008	0.012	0.013
Health Profession
PCP	5,715	3,129	2,190	0.617	0.010	0.015	0.016
ID	2,546	1,394	976	0.617	0.015	0.023	0.025
NP or PA	872	477	334	0.617	0.025	0.039	0.042
Urbanicity
MSA	8,541	4,676	3,273	0.617	0.008	0.012	0.013
Non-MSA	592	324	227	0.617	0.031	0.047	0.051
Gender
Male	5,490	3,006	2,104	0.617	0.010	0.015	0.017
Female	3,643	1,994	1,396	0.617	0.012	0.019	0.021
Region
Northeast	2,348	1,285	900	0.617	0.015	0.024	0.026
Midwest	1,468	804	563	0.617	0.019	0.030	0.032
South	3,841	2,103	1,472	0.617	0.012	0.018	0.020
West	1,476	808	566	0.617	0.019	0.030	0.032
Interactions
MSA/PCP	5,304	2,904	2,033	0.617	0.010	0.016	0.017
MSA/ID	2,459	1,346	942	0.617	0.015	0.023	0.025
MSA/NP or PA	778	426	298	0.617	0.027	0.041	0.045
Non-MSA/PCP	411	225	158	0.617	0.037	0.056	0.062
Non-MSA/ID	87	48	33	0.617	0.081	0.124	0.135
Non-MSA/NP or PA	94	51	36	0.617	0.078	0.119	0.130
Practice Survey
All	4,567	500	350	0.923	0.030	0.046	0.050
Urbanicity
MSA	4,271	468	327	0.923	0.031	0.048	0.052
Non-MSA	296	32	23	0.923	0.121	0.185	0.202
Region
Northeast	1,174	129	90	0.923	0.060	0.091	0.100
Midwest	734	80	56	0.923	0.076	0.116	0.127
South	1,921	210	147	0.923	0.047	0.071	0.078
West	738	81	57	0.923	0.076	0.116	0.126

b. Clinician Survey and Practice Survey Design

Both the clinician and practice survey designs involve several parallel activities by HRSA and its contractors. First, HRSA will conduct an informational campaign to acquaint clinicians, practices, and federal agencies with the future surveys; to publicize their national importance; and to encourage sample members to respond to the request for information. HRSA has disseminated information about the surveys at the following venues:

• HIV/AIDS Bureau (HAB) biweekly emails

• Centers for Disease Control and Prevention (CDC)/HRSA Advisory Committee meetings (November 2010, May 2011, and November 2011)

• Department of Health and Human Services (HHS) Minority AIDS Initiative (MAI) Advisory Group meetings (October 2010, March 2011, and September 2011)

• HRSA Office of Regional Operations meeting (September 2011)

• HRSA/HAB HIV Workforce Consultation meeting (February 2011)

• CDC Prevention Conference (August 2011)

• Ryan White HIV/AIDS Program All Grantee Meeting (August 2010)

• HRSA/HAB Consultation Meeting on HIV Workforce (2008)

• Monthly electronic newsletter to all Ryan White HIV/AIDS Program grantees

• “30 for 30 Campaign” meeting at HHS with women living with HIV or AIDS

HRSA’s contractors will also identify a national universe of clinicians who provide care to patients with HIV or AIDS. To do this, HRSA will access national, all-payer medical and pharmacy claims to determine which clinicians treat and/or prescribe medications for patients with HIV or AIDS. HRSA will then select a sample of 5,000 high-volume HIV clinicians and, among the sampled clinicians, select 500 deduplicated practices. HRSA will collect clinician data from 70 percent of the clinician sample (about 3,500) using web, paper, and telephone data collection modes and from 70 percent of the practice sample (about 350) using a paper-only data collection mode. HRSA will clean the data, create an analytic data file, and use the data file to populate the HIV workforce model. Finally, the survey contractor will prepare and submit a survey and final report to HRSA.

HRSA anticipates that the data collection for both surveys will last 16 weeks, plus 2 additional weeks to clean the data. During the data cleaning process, the contractors will finalize all partial completes and retrieve missing information. We provide a copy of the clinician survey in Attachment B-1, sample screen shots of the WebSurv page layouts in Attachment B-2, and a copy of the practice survey in Attachment B-3. (We provide a list of the sources of the questions for the clinician and practice instruments in Attachment E.)

Data collection modes of the clinician and practice surveys. To maximize the cost-effectiveness of the clinician survey and minimize the mode effect, HRSA plans to conduct web/mail surveys and to do telephone follow-up interviews with the clinicians who do not complete the survey by mail or web. HRSA expects to complete 50 percent of the interviews by mail, 40 percent by web, and 10 percent by telephone. To avoid mode effects, the mail and web versions of the clinician instrument will use the same question wording and paths. If it is necessary to complete the interview by telephone, the interviewers will use the same paper questionnaire as clinicians will use to self-administer the survey. Whether the paper instrument is self-completed by the clinician or with the help of an interviewer, all responses will be keyed into an electronic database. The practice survey will be conducted using only mail survey methodology. The small number of completed practice surveys (350) does not justify the expense of developing and fielding a second web data collection tool.

Advance mailings to clinicians and practices. Using the updated addresses from SK&A and any additional pre-field locating information, HRSA will mail advance letters to clinicians (for the clinician survey) and to practice administrators (for the practice survey). The envelopes and letters will be printed on HRSA letterhead, signed by a senior HRSA official, and mailed in HRSA envelopes to indicate their importance to the recipients and the likelihood of the clinicians and administrators opening them. HRSA’s letter will briefly explain the purpose and importance of the survey, the amount of time it is expected to take (20 minutes for the clinician survey and 30 minutes for the practice survey), and the privacy of the information. The advance letters also will introduce the contractor and its role in collecting the data and will endorse its efforts. Enclosed in the introductory letters will be an informational brochure that explains the study in greater detail and provides responses to anticipated questions. We provide a copy of the informational brochure for the clinician survey in Attachment C-1 and for the practice survey in C-2.

HRSA’s advance letter to sampled clinicians will explain that the contractor will include a pre-completion gift card in the introductory survey packet and will send a post-completion gift card after the survey is returned. The value of the pre-completion gift card will be $20. The value of the post-completion gift card will depend on the mode: $40 for completing the survey online and $20 for completing it by paper or telephone. Recent research has also shown that offering physicians incentive payments both before and after completing a survey promotes a higher response rate than offering only a post-completion incentive payment (Delnevo et al. 2004).

Survey packets to clinicians and practices. The contractor will send survey packets via priority mail 10 days after HRSA’s advance letter mailing. The 10-day period will be sufficient for the return of any misdirected mail and for the contractor to begin any necessary locating. All packets will contain a brief letter from the contractor following up on the earlier letter from HRSA. The letters will note the included paper versions of the surveys, pre-paid envelopes for returning them, and explain how to contact contractor staff with any questions. The clinician letter will include the web address and a personal password for accessing the web-based instrument. It will also note the inclusion of the pre-completion $20 gift card and explain that an additional post-completion gift card (whose value will vary depending on the survey mode used) will be sent upon receipt of the completed survey.

Prompting. As HIV primary care clinicians and practices are heavily surveyed and have long and intense work days, they tend to be difficult to survey. Thus, in addition to the early information campaign to promote the survey, the advance letter from HRSA, and the pre- and post-completion incentives, we will mail three prompt letters to nonrespondent sample members requesting them to complete the survey. We provide a schedule for completing the survey in Table 2. The contractor will mail the first reminder letter to nonrespondent clinicians and practices, together with another copy of the paper survey, on March 14, 2012; HRSA will mail a second reminder letter on April 16, 2012; and the contractor will mail a final reminder letter on May 18, 2012. The first two reminder letters will be sent by first class mail. The final reminder letter will be sent by priority mail. On May 28, 2012, the contractor will begin conducting telephone prompts, an estimated 10 percent of which are expected to result in immediate telephone completions. It is important to prompt several times by multiple methods but not so many times as to be perceived as harassing the sample members. Note that the practice surveys follow the same schedule as the clinician surveys. Practice surveys, however, will not contain URL information, as they will be administered only by paper with telephone follow-up. The contractor will mail thank you letters to all respondents and will include post-completion gift cards in those sent to clinicians.

Table 2. Schedule of Information Collection Activities

Activity	Start Date	End Date
Information campaign (conference flyers, monthly electronic newsletters, and so on)	October 1, 2011	January 31, 2012
Sample selection (both clinician and practice)	November 1, 2011	December 31, 2011
Mail and web data collection (4 months with 2+ week clean-up)	February 1, 2012	June 16, 2011
HRSA mails advance letters, explaining the surveys and including the informational brochures, via first class mail	February 15, 2012	February 15, 2012
Contractor mails survey packets, containing survey questionnaire, $20 gift card, and URL link and password (clinicians only), via priority mail	February 27, 2012	February 27, 2012
Contractor mails first reminder letter to nonrespondents, including questionnaire and URL link and password (clinicians only), via first class mail	March 14, 2012	March 14, 2012
HRSA mails second reminder letter to nonrespondents, including questionnaire and URL link and password (clinicians only), via first class mail	April 16, 2012	April 16, 2012
Contractor mails third reminder letter to nonrespondents, including questionnaire and URL link and password (clinicians only), via priority mail	May 18, 2012	May 18, 2012
Contractor mails thank you letter and incentive payments (clinicians only) to respondents via first class mail (16 weeks)	February 27, 2012	June 15, 2012
Conduct telephone prompts to nonrespondents; some prompt calls might result in completed telephone interviews	May 28, 2012	June 15, 2012
Clean data and prepare survey data file	June 1, 2012	June 29, 2012

Attachment D provides additional respondent materials, including the advance letter from HRSA to sample clinicians and practice administrators (Attachments D-1 and D-2), the cover letter from the survey contractor to sample members (Attachments D-3 and D-4), the prompt letters to nonrespondents (Attachments D-5 through D-10), the URL address and personal password for accessing the web-based clinician instrument (D-11), the thank you letters from the contractor to respondents (Attachments D-12 and D-13), and promotional material (Attachment D-14).

c. Uses of the Data

We will use the clinician survey to identify the factors most likely to affect the current and future capacity of the HIV workforce to meet the growing demand for HIV-related medical care. We will also use the information from the clinician survey to forecast the future supply of HIV clinicians, independent of potential changes in productivity and capacity at the practice level. Individual-level determinants of future supply include the clinician’s age, gender, health care profession, medical specialty, number of hours spent in direct patient care, size of HIV caseload, plans to increase or decrease the number of hours spent in direct patient care, and retirement plans. In addition, we will collect information on clinicians’ perceptions about the effectiveness of various strategies for increasing the future capacity of the HIV workforce—for example, training more students, expanding Title VII and VIII programs, increasing the use of health information technology, and using more non-HIV clinicians.

We will use the practice survey to collect information on the characteristics of the practice that are most likely to affect the future capacity and productivity of the HIV clinician workforce. We will use the information from the practice survey to estimate the impact of organizational characteristics on clinician productivity, as measured by the number of patients, visits, relative value units, or revenue per clinician full-time equivalent (FTE). Organization-level determinants of workforce capacity include type and size of practice, practice specialty, practice affiliation, number and composition of FTE staff, type of staffing model and patient management strategies used, meaningful use of electronic medical record systems, appointment scheduling practices and policies, and number and acuity of patients. We will also use the practice survey to estimate current excess demand for HIV care. We will base these estimates on the number of staff vacancies, experience recruiting and retaining staff, amount of time needed to schedule initial and follow-up appointments, amount of time spent with patients during initial and follow-up appointments, and willingness to accept new patients.

We will use the results of the clinician survey (with supplemental information captured in the SK&A contact database) to forecast the “baseline” supply of HIV clinicians through 2015, independent of organization-level changes in productivity (such as adoption of health information technology). We will use the results from the practice survey to estimate the impact of staffing, patient management strategies, and other organization-level factors on the output of HIV care per clinician FTE, and then use these estimates to simulate future supply of HIV clinicians under various assumptions about the adoption of productivity-enhancing strategies at the organizational level, independent of changes in the number of practicing clinicians. To incorporate the organization-level inputs for all sampled clinicians into the workforce supply model, we will also ask individual providers on the clinician-level survey questions about the nature of the primary practice setting within which they work.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Because the expected response rate for the survey is less than 80 percent for both the clinician and the practice surveys, we plan to analyze nonresponse patterns using relevant information known about respondents and nonrespondents. We will compare the characteristics of respondents and nonrespondents to evaluate the risk of nonresponse bias. We will attempt to minimize nonresponse bias by constructing analysis weights that account for differential nonresponse patterns, as described in section B.2 above.

HRSA’s previously described data collection strategy has been designed so each step will help maximize the response rate for clinician and practice surveys and deliver high-quality data from both promptly. This strategy includes the following actions:

1. Early informational campaign. The early informational campaign by HRSA will center on conferences and meetings that are heavily attended by clinicians who serve the HIV/AIDS community. HRSA will also send information about the survey to clinicians and practice administrators who have signed up for HAB’s monthly electronic newsletter. The materials will inform clinicians and administrators about the upcoming workforce survey and encourage participation by explaining that the survey is important not only to HRSA and other stakeholders, but to HIV clinicians and practices themselves. The materials will emphasize that HRSA will use the information to help achieve the goals of the national HIV/AIDS strategy: increase access to care, improve health outcomes for people living with HIV or AIDS, and reduce HIV-related disparities and health inequities. See page 3 for a list of dissemination venues.

2. Up-to-date clinician contact information and pre-field locating. Contact information collected on a monthly basis by SK&A, together with pre-field locating by the survey contractor, will ensure the contractor has the optimal information for contacting clinicians and practices. HRSA will begin with an extensive pre-field locating process. The clinician sample will be sent to SK&A, which will return the file to the contractor with appended contact information, consisting of telephone numbers and addresses for the clinician, as well as the officer manager of the associated practice.

3. Timely in-field locating. In-field locating will be conducted to track sample members (clinicians and practices) not at the addresses identified earlier. We expect approximately 5 percent of all mailed packets will be undeliverable. The contractor’s trained locating staff, who have access to various electronic-locating means, will work all undeliverable mail or incorrect telephone numbers. These locating efforts will maximize our ability to contact and remind sample members to respond to the survey.

4. Frequent mail and telephone reminder contacts. Mail and other prompts will encourage participation. Clinicians and practice staff work long and intense work days; we believe that multiple prompts will be needed to achieve a high response rate. The telephone reminders will begin at month three (May 28, 2012) of data collection. We will train professional interviewers on the specifics of this study and on procedures for getting through gatekeepers to reach our sample members.

5. Paper survey option. The primary data collection tool for both surveys is the paper questionnaire. As previously noted, we expect that, given the age of many HIV clinicians, most will prefer paper administration. Thus, the survey packet will contain a hard copy of the survey. Respondents will be able to send the paper survey back to the contractor by fax or via a pre-paid envelope. Contractor staff will enter the data from the returned clinician questionnaires into the same web software that is used to collect the web survey and that will be programmed to identify missing or erroneous information. Practice survey data will also be keyed into a database that is programmed to check for skip and range errors.

6. Web-based survey option. The web survey instrument maximizes flexibility for clinicians who prefer to use it. The web mode will provide clinicians an option to complete the survey electronically, thereby eliminating the burden of mailing or faxing back a hard-copy instrument. In addition, the web survey will have integrated skips, making survey completion less burdensome. Respondents who are interrupted will be able to save their responses and complete the survey later at their convenience.

7. Regular and timely monitoring of the survey data. We will look for patterns in missing data or other issues and call respondents to retrieve any missing data. We will also monitor which content modules of the questionnaire are more or less complete to encourage the correct people in an organization to fill them out.

8. Incentive payments to clinicians. We will include in the introductory survey packet sent to all sampled clinicians a check of $20 for participating in the survey. We will also send in the thank you letters a $20 or $40 check, depending on the mode of completion, to clinicians who complete the survey. Incentives paid to respondents have been shown to encourage participation and increase response rates, which in turn improves the validity and reliability of the data (Abreu and Winters 1999; Shettle and Mooney 1999). Research has also shown that offering incentive payments before and after completing a survey promotes a higher response rate than offering only a post-completion incentive payment (Delnevo et al. 2004).

Basis of Response Rate Estimate. As mentioned previously, we are estimating a response rate of 70 percent for both clinicians and practices. This is based on multiple factors. First, since we are using a listed sample of clinicians and providers, we anticipate only a small number of cases in both samples will be unlocateable. Unlocateable cases often are a primary reason for low response rates. We do not believe that locating issues will be significant given our study design. Second, we are offering clinicians an incentive to complete the survey. Incentives timed as we have planned and of the magnitude we are proposing have been shown to facilitate participation among clinicians. Third, since the practice sample is small (n=500), we will be able to conduct personalized and repeated follow-up to encourage participation. Fourth, clinicians and practices that focus on the care and treatment of patients with HIV generally share a social mission to improve access to care for people living with HIV/AIDS. Studies also show that they currently face significant workforce challenges (HRSA 2010). Based on our previous work with this provider group, we believe that HIV clinicians and clinic administrators have a vested interest in responding to a survey intended to strengthen HIV workforce capacity and to help them address the growing demand for care. Finally, HRSA will emphasize the importance of survey participation through the field period through listerv announcements and inclusion of information about the survey (as presented in the promotional material in Attachment D-14) in topic-relevant correspondence such as electronic newsletters.

Basis for Mode Preference Assumptions. In terms of mode, we anticipate that 100 percent of practice surveys will be completed by the mail, as that is the only mode offered. If a practice prefers to complete the survey over the phone, our contractor will provide this accommodation. For the clinician survey, HRSA expects to complete 50 percent of the interviews by mail, 40 percent by web, and 10 percent by telephone. This distribution is based on two assumptions. First, clinicians are difficult to reach by telephone because of having to go through gatekeepers (that is, office staff) or are simply unavailable because of their schedules. Thus, we expect minimal participation by telephone. We typically estimate that between 80 and 85 percent of completions would be by the mail. However, we also offer the survey by web and will attempt to steer clinicians to that mode by offering a differential incentive ($40 rather than $20). To date, a differential incentive has not been attempted with clinicians. However, our contractor has successfully employed this methodology with a survey of recent college graduates and found that over 90 percent of the completed surveys from this group were completed via the web (NSRCG 2010). What we learn from the proposed survey effort with the differential incentive payments for clinicians will have important implications for the broader field of survey research.

4. Tests of Procedures or Methods to Be Undertaken

HRSA pretested the clinician instrument with seven clinicians and the practice instrument with three practice administrators. The pretest offered valuable information for changes needed to increase the clarity and appropriateness of the questions and decrease the amount of time it takes to complete the survey. Attachment F includes a copy of the pretest memorandum, summarizing the changes we made after pretesting the instruments.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Barbara Carlson, M.A. in Statistics (Boston University), Mathematica Policy Research

REFERENCES

Abreu, D. A., and F. Winters. “Using Monetary Incentives to Reduce Attrition in the Survey of Income and Program Participation,” Proceedings of the Survey Research Methods Section of the American Statistical Association, 1999.

Bureau of Labor Statistics, Department of Labor. “National Occupational Employment and Wage Estimates, United States.” Washington, DC: DOL, BLS, May 2010.[not cited in Part B]

Chromy, J. R. “Sequential Sample Selection Methods.” Proceedings of the Survey Research Methods Section of the American Statistical Association, 1979, pp. 401–406.

Delnevo, Cristine D., Diane J. Abatemarco, and Michael B. Steinberg. “Physician Response Rates to a Mail Survey by Specialty and Timing of Incentives: Brief Report.” New Brunswick, NJ: University of Medicine and Dentistry of New Jersey, School of Public Health, March 10, 2004.

Shettle, Carolyn, and Geraldine Mooney. “Monetary Incentives in Government Surveys.” Journal of Official Statistics, vol. 15, 1999, pp. 231–250.

1 We will finalize the cutoff threshold for selecting HIV clinicians for this study after examining the distribution of claims and patients across clinicians.

2 The burden hours for the telephone screening call were not included in the initial burden table. However, since the time of the original PRA submission, we have received the sample frame file and, after reviewing the contents, we now believe we can proceed without the use of telephone screener calls, thus minimizing the burden on practices. The sample frame suggests that many clinicians work at satellite clinics; their group practice address of record is a centralized location where administrative activities occur. Thus, making calls to the group practice will not have the intended consequence of helping us affirm that the clinician indeed works at that site (the target for the practice survey).