NASS statistical review comments and ERS responses

5/2/12

Michael Jacobsen

OMB DOCKET FOR THE 2012-2014 RURAL COMMUNITY WEALTH AND HEALTH CARE PROVISION STUDY

I have reviewed this OMB docket and have the following comments. This is a very well-organized docket, with a strong hypothesis and a highly detailed plan for capturing the data. However, I have questions about some parts of the study, most of which concern the purpose of the study and the sampling design.

In Part A, I did not understand why the USDA needs these data. While there is strong justification for conducting this study, there was very little rationale for the USDA to collect the data. As question 1 is currently written, it suggests that no government programs or initiatives require or use these data. Therefore, why should a federal agency spend the resources to conduct this study if there is no direct benefit to the agency? The statues outlined in question 1 and annex B only show the authority of the USDA to carry out this study, not the need. Then, is this study only being conducted for the sake of research? This response would be strengthened if the program managers can give specific concrete examples of how the USDA or other agencies are going to use these data. What specific program or initiative or law requires the data?

Yes, this study is being conducted for research purposes only, and is not designed to respond to the requirement s of a particular law or program. The mission of ERS is to anticipate economic and policy issues related to agriculture, food, the environment, and rural development, and conduct economic research that broadly and specifically informs public program and policy decisions. In many cases, this involves collecting primary survey data on topics related to these issues, such as on the linkages between rural wealth and health care services. As explained in the ICR, this topic is of high priority for rural communities and for USDA and has not been addressed by other surveys. The information will also be useful for other government agencies, such as the Office of Rural Health Policy in the Department of Health and Human Services (documented by a letter from the head of that office), and other government and non-government organizations concerned with the well-being of rural people and rural development.

The response to question 2 in part A is lacking several facts on the plan for conducting the study and does not provide a strong plan for using the data. Question 2 in part A wants the program managers to outline their plan for conducting their study and how they will use the data. This includes a few sentences outlining from whom the data will be collected, how the data will be collected and what options are available for collection, how frequently the data will be collected, and with whom the data will be shared. The response to this question does not provide these details. Note that this does not mean that the entire docket needs to be put in this question. However, an outline of the full plan is expected.

We have added information in the response to question 2 indicating briefly from whom the data will be collected, how it will be collected, and how frequently it will be collected. These and the other issues raised are discussed in more detail elsewhere in the ICR.

As for the use of the data, the plan given is too weak to be acceptable. What research reports are going to be generated from the results and who will be using them? What are the purposes of the peer-reviewed articles? Are those peer-reviewed articles even necessary? Which “other research organizations” will use these data? How else will these data be used (a website, an interactive map, an educational plan, etc.)? Will any federal programs use these data for their work? There need to be specific examples of the uses for these data.

Peer-reviewed research reports and journal articles are standard outputs of the research that ERS conducts. Anyone can use them; that is the point of publishing in widely available outlets. We provide detailed information in the response to question 16 about the research reports that will be generated from the data. We have added a footnote in the response to question 2 indicating this.

I have a similar concern about the response to question 6. Who exactly will be affected by this study? The current response does not provide any concrete examples of agencies, policies, or political activities that will be affected by not conducting this study. This suggests that the data are not really needed by anybody, which I doubt.

We have already responded to this question above.

The detailed plan given in Part B is strong but some details are missing.

1. What is the target population? This was not made fully clear in questions 1 and 2.

The target population is stated in the first sentence in response to Part B, question 1: “The potential respondent universe will include health care providers, community leaders and other local stakeholders involved in recruitment and retention of health care providers in rural small towns of the nine states in the study (Arkansas, Louisiana, and Mississippi – Lower Mississippi Delta (LMD) region; Kansas, Oklahoma, and Texas – Southern Great Plains (SGP) region; and Iowa, Minnesota, and Wisconsin – Upper Midwest (UMW) region).” Elsewhere in our response to question 1, we explain how, in this study, we are defining rural small towns (with population between 2,500 and 20,000) and health care providers (physicians, dentists, physician assistants or associates, nurse practitioners, certified nurse midwives, and pharmacists), and the reasons behind those definitions. We also explain how the population of health care providers will be identified in each sample community and used as the sample frame. For community leaders and other stakeholders involved in recruitment and retention of rural health care providers, we explain that this population cannot be precisely defined or listed in advance. Thus, as explained in the response to this question, the sample of respondents for this population will be based on screening interviews with knowledgeable informants in each sample community. We do not claim that this sample will be a representative sample because we have no way to reliably identify the full population of community leaders and other stakeholders that are involved in recruiting and retaining health care providers in the sample communities.

2. The sample numbers listed in part A, question 12 need to be put in part B, question 1. These are the results of the sampling procedures; therefore, they need to be here.

These numbers are now given at the end of the response to question 2, on the sample selection approach (as well as in Part A, question 12, where they are necessary to estimate the respondent burden). They fit here better than in the response to question 1, because the response to this part of question 2 explains how and how many respondents of each type will be selected in each sample community.

3. 150 towns will be sampled but will all 150 towns be included in the final dataset? The final sample for the actual study (after the pilot study) was not obvious.

We have added a paragraph to the response to question 2 explaining that the data from the 10 pilot phase communities will be combined with the data from the remaining 140 sample communities for the analysis, except for survey questions that are revised or dropped based on the pilot phase results.

4. Why will you only select a maximum of 8 providers to survey in your sampled towns? There was no support for this number.

We provide further explanation for a maximum number of respondents per town in the response to question 2. The reasons for using a maximum are to limit the costs of the survey and the burden on members of any particular town, while allowing a sufficient number to represent the different types of providers in any town. Due to likely non-independence of the information collected from different providers in the same town, it would be inefficient to insist on a complete census of all providers. The maximum number selected (8) is inevitably somewhat arbitrary since we do not have information on the variances and covariances of the variables of interest within and across the study communities, which would be necessary to estimate the optimal allocation of the sample among the study communities.

5. Where did the 80% response rate come from? What similar studies have supported the idea that the response rate for this study will be 80%?

We have added a paragraph to the response to question 3 explaining the basis for the target 80% response rate.