Nonsubstantive change request and justification

TO: Jennifer Park, Office of Management and Budget

Statistical and Science Policy,

Office of Information and Regulatory Affairs

FROM: John Pender DATE: 10/30/2014

Economic Research Service

SUBJECT: OMB CONTROL NUMBER: 0536-0072

Non-substantive changes to burden estimate and key informant interview script for the Survey on Rural Community Wealth and Health Care Provision

Based upon key informant telephone interviews conducted during the pilot phase of these interviews for the Survey on Rural Community Wealth and Health Care Provision, we have revised the burden estimate and propose some non-substantive changes to the key informant interview script.

Burden Estimate

The burden estimate included in the original Information Collection Request (ICR) for this survey conservatively assumed that the maximum of four respondents per community (600 total in 150 communities) would be interviewed in the key informant interviews, and that the response rate would be 67%, implying some burden also on 300 non-respondents. In the pilot study, we attempted to interview 54 potential respondents in 12 communities and completed interviews with 23 respondents.¹ We attempted to interview a smaller number of potential respondents in the pilot study than planned for the full study (4.5 per community rather than 6 per community) because of the limited time available to complete the pilot study.² During the remainder of the key informant interviews, we plan to interview on average about three respondents per community.³ Given the ratio of non-respondents to respondents found in the pilot phase (31/23)⁴, we estimate that to achieve this average number of responses we will need to attempt to interview 1057 potential respondents, including 450 respondents and 607 non-respondents. This is a smaller number of respondents than estimated in the original ICR but a larger number of non-respondents. The number of administrative staff contacted per community and the average amount of time spent talking with administrative staff, respondents and non-respondents in the pilot phase turned out to be substantially less than estimated in the original ICR, resulting in a substantially lower current estimate of the total burden of the key informant interviews in all 150 study communities (390 hours compared to 975 hours in the original ICR). As a result, our estimate of the total burden of the entire study is substantially reduced (to 1,613 hours valued at $82,950 from 2,198 hours valued at $103,650 in the original ICR).

The range of times spent interviewing respondents in the pilot phase key informant interviews (20 to 60 minutes) is consistent with our expectations as expressed in the original ICR. The burden estimates in the original ICR were conservative in using the maximum expected length of interview (60 minutes) to estimate the burden.

Implications of a Smaller Number of Respondents than Estimated in the Original ICR

As mentioned above, our estimate of the respondent burden in the original ICR assumed conservatively that the maximum number of respondents (four) would be interviewed in each community in the key informant interviews. Our current estimate of the average number of respondents per community (three) is the midpoint between the maximum and minimum number of interviews expected, and a more realistic estimate. Although we did not interview as many respondents per community as this during the pilot phase, that was due to time constraints for the pilot phase that will be less of a concern when completing the remainder of the key informant interviews. Hence, we do not anticipate significant implications of a change in the number of respondents in the key informant interviews for the precision of any estimates derived from the data, as the average number of responses expected has not changed. Furthermore, our analysis of the key informant interview data is only intended to provide qualitative indications of the perceptions of community and health care leaders related to the study questions, and not quantitative precision. Since the underlying population represented by the respondents in the key informant interviews is not well defined (which was the reason to replace the proposed survey of community leaders by semi-structured, qualitative key informant interviews), there is no basis to report the precision of any estimates derived from these data. Analysis of these data will focus on tabulating the simple frequencies of responses and will make no claims of representing the parameters of an underlying population.

Implications of a Lower Response Rate than Estimated in the Original ICR

Using the AAPOR formula for the unweighted response rate (OMB 2006, Guideline 3.2.2, p. 14), we estimate the response rate in the pilot study key informant interviews to be 57%.⁵ This is lower than the response rate estimated in the original ICR (67%). However, since we are only using the key informant interview data for qualitative analysis and not for precise estimation of any well-defined population parameters, the issue of non-response bias does not arise for the key informant interviews. The only reason the response rate is of interest in the key informant interviews is because it affects the number of non-respondents that need to be contacted and hence the burden estimate.

We will conduct a similar analysis of the response rate in the pilot phase of the health care provider survey when that is completed, using the appropriate AAPOR formula.

Proposed Non-Substantive Changes in the Key Informant Interview Script

Based on experience with the pilot key informant interviews, we propose three non-substantive changes to the key informant interview script.⁶ These changes and the rationale for the changes are shown in Table 1.

Table 1. Proposed Key Informant Interview Script Changes

Question #	7
Original Q:	How knowledgeable are you about issues related to health care provision in this community? On a scale from 1 to 5, if 1 means you are not knowledgeable at all and 5 means you are very knowledgeable, which number would you choose?
Rationale for change:	Respondents have been answering open text questions and seemed to have trouble switching to the numbered scale. We propose maintaining 5 response options but labeling them with words rather than a numbered scale.
Proposed Revision:	How knowledgeable are you about issues related to health care provision in this community? Would you say you have “No Knowledge”, “A Little Knowledge”, “Some Knowledge”, “Quite a Bit of Knowledge”, or “A Great Deal of Knowledge”?
Question #	32
Original Q:	In your opinion, how important is it for your community to actively try to recruit and/or retain health care providers? (Please explain your answer.)
Rationale for change:	As open text, this question is redundant for many respondents. Providing 5 response options would provide variety and be more useful for comparison purposes.
Proposed Revision:	How important is it for your community to actively try to recruit and/or retain health care providers? Would you say it is “Not at All Important”, “Slightly Important”, “Moderately Important”, “Quite Important”, or “Very Important”?
Question #	35
Original Q:	Those are all the specific questions I have for you. But I have one more request. We would like to contact (some) health care providers who work in your community, and we want to make sure that we have accurate information about who is currently working there.
Rationale for change:	This item is a request for assistance in verifying the list of health care providers that SBRS has developed for the community. The request needs to include the definition of primary health care providers used for the project so respondents can respond more appropriately.
Proposed Revision:	Those are all the specific questions I have for you. But I have one more request. We would like to contact (some) health care providers who work in your community, and we want to make sure that we have accurate information about who is currently working there. We are using the Medicare definition of Primary Health Care Providers, so this list includes Physicians with a specialty of General or Family Medicine, Internal Medicine, Pediatrics, or Geriatrics. We also include Dentists, Physician’s Assistants, Nurse Practitioners, and Nurse Midwives.

Summary

The completion of Key Informant interviews in the Pilot Study of 12 communities indicates that the project procedure specified in the ICR is sound. The process used for identifying knowledgeable key informants appears to be effective, and the development of sample frames of primary health care providers for each selected community has been verified as complete by knowledgeable individuals in the 12 Pilot communities. The semi-structured interview questions are providing the data required to effectively address project goals. The operational processes ran smoothly and the time required to complete the interviews was consistent with expectations. The burden was significantly less than that estimated in the ICR, although the response rate was somewhat lower that estimated in the original ICR. We interviewed fewer respondents per community in the pilot study than anticipated in the original ICR due to time constraints, but the number that we plan to interview in the full set of communities is not significantly changed (though our burden estimate is now less conservative than in the original ICR in not assuming the maximum number of respondents per community). The lower response rate in the key informant interviews does not have any implications for the precision or bias of estimates to be derived from these interviews, because the analysis of these data will focus on simple frequency tabulations of the responses to indicate the range of qualitative perceptions found, and will make no claims concerning precision or that they represent any well-defined parameters of an underlying population. A few minor changes to the semi-structured questionnaire instrument are proposed based upon the pilot study.

1 See Table 1 in the pilot phase report for key informant semi-structured interviews from the USDA Economic Research Service (ERS) and Iowa State University (ISU) Survey & Behavioral Research Services, dated October 2014. The two pending cases in that table resulted in completed surveys.

2 The contract between ERS and ISU ends September 30, 2015, and all survey work by ISU for all phases of the survey must be completed by that date.

3 See the revised supporting statement Section A of the ICR, dated October 2014.

4 The 31 non-respondents in this ratio include 3 people who were not eligible, 7 who referred the enumerator to other people who were more qualified to answer the questions, 15 people who we were never able to interview and were dropped after reaching a maximum number of calls or deciding that we had sufficient responses for the pilot study from that community, and 6 refusals.

5 The AAPOR formula is RRU = C/(C+R+NC+O+eU), where C = number of completed cases or sufficient partials, R = number of refused cases, NC = number of noncontacted sample units known to be eligible, O = number of eligible sample units not responding for reasons other than refusal, U = number of sample units of unknown eligibility and not completed, and e = estimated proportion of sample units of unknown eligibility that are eligible. Based on the results in Table 1 in the key informant interview pilot study report, we estimate RRU with C = 23, R = 6, NC = 0, O = 0, U = 15, and e = 29/39. We assume that referrals were ineligible.

6 See also the revised version of Annex G to the ICR, which incorporates all of these proposed changes.

2