0354 Ss B 20190215

B. Collections of Information Employing Statistical Methods

1. Describe the potential respondent universe and any sampling or other respondent selection methods to be used.

The potential respondent universe of the Goat 2019 study is all operations with 5 or more adult goats in 25 States¹. The study population is limited to operations with at least 5 adult goats because the goal is to survey operations with a commercial goat component; operations with fewer than 5 goats are likely to have goats only for pet/hobby purposes based on analysis of data from the NAHMS Goat 2009 data.

The 25 specific States were chosen because they account for 82.7 percent of the goats on operations with 5 or more adult goats and 77.9 percent of goat operations with 5 or more adult goats in the United States – and at least 70.0 percent of goat inventory and goat operations within each of the goat type categories (dairy, meat/other, and unknown types of goats). States were chosen based on their percentage of goat inventory and goat operations, for their geographic representativeness, and expected response burden. See Table A.1 in Appendix A for percentage of goat inventory and goat operations estimates used for State selection.

For Phase I of the study, a total sample of 4,770 operations will be chosen from the National Agricultural Statistics Service (NASS) List Frame, which will have been updated with information from the NASS 2017 Census of Agriculture, so will have the best chance of avoiding out of scope, out of business, or otherwise operations with no animals present. This will be a stratified random sample, where the selection strata are defined by State, operation type (dairy, other), and size category (5-19, 20-99, and 100 or more adult goats). Operations that complete the Phase I questionnaire will be eligible to participate in Phase II of the Study, which entails a second questionnaire, plus two optional questionnaires, depending on the operation type. Operations that complete the second questionnaire will have the opportunity to participate in biological testing and an opportunity to submit feedback on the Study.

2. Describe the procedures for the collection of information.

0. Statistical methodology for stratification and sample selection:

Eligible operations in the 25 states on NASS’ List Frame will be stratified by State, operation type, and size category. The total sample of 4,770 will be allocated to strata based on a weighted average of inventory and number of operations in each stratum. Dairy operations will be sampled with a higher selection probability to allow reporting of estimates by operation type and larger operations will be sampled with a higher selection probability to ensure they are captured in the study. Within each stratum, a simple random sample will be chosen.

Phase I of the Study will be implemented by NASS. Selected operations will be mailed a pre-survey letter and then contacted by telephone. Amenable operations will be visited in person by a NASS enumerator, who will administer the Phase I questionnaire. At the same time, operators who complete the Phase I questionnaire will be asked to participate in Phase II of the survey. Operators who express an interest in Phase II will be asked to sign a consent form which allows NASS to turn over the operator’s name and contact information to a designated Animal and Plant Health Inspection Service (APHIS)-designated data collector.

APHIS-designated data collectors will contact the consenting operations by telephone to request personal visits to administer the Phase II questionnaire(s). The questionnaire(s) and any biologic testing will be conducted by a VS veterinarian or animal health technician.

0. Estimation procedure:

The sampling design is a stratified random sample with unequal probabilities of selection. The statistical estimation will be undertaken using either SAS survey procedures and/or SUDAAN. Both software packages use a Taylor series expansion to estimate appropriate variances for the stratified, weighted data.

0. Degree of accuracy needed:

NAHMS’ goal is to produce descriptive statistics (proportions or means) with a coefficient of variation (CV) of less than 20 percent. If possible given adequate response rates, estimates will be produced for region (East, West) by operation-type (dairy, other) cells, and for size category (5-19, 20-99, and 100 or more adult goats) cells.

In order to meet the precision criteria within each of the given stratification cells, we require an overall sample size of approximately 1,600 assuming that a simple random sample with a perfect response rate is taken. However, due to practical considerations, we must account for the expected response of 55.0 percent and an expected design effect of approximately 1.5 to obtain estimates with appropriate standard errors. An overall sample size of approximately 4,370 is required after adjusting for these factors. In order to produce estimates at the State level for the State of California, we require an additional 400 samples after accounting for the expected response rate and design effect. This means that a total sample size of 4,770 will be required to produce estimates with CV not exceeding 20 percent within each of the stratification cells and at the State level in California.

Tables B.1 and B.2 in Appendix B shows estimates of precision based on the total sample of 4,770, for region by operation-type cells, and nationally for Phase I and Phase II. Response rates from previous studies (see Table C.1. in Appendix C for response rate information from the NAHMS Goat 2009 and Sheep 2011 Studies) and a design effect of 1.5 were assumed. Most of the estimated CVs and confidence intervals for Phase I are expected to be within the desired range with the given allocation.

Since there are three size strata, precision estimates will not exceed those presented in Tables B.1 and B.2 because the number of expected respondents in each of the three size categories will exceed the expected number of respondents in the region by operation-type categories. Because the CV estimates will be smaller, precision estimates under this scheme are not reported here.

Reporting strata for Phase II estimates may be adjusted depending on the number of respondents, i.e., estimates reported by region or operation-type (not crossed). In general, if sample sizes are too small or CVs too large for any estimates, those estimates are not published.

• Unusual problems requiring specialized sampling procedures:

There are no unusual problems requiring specialized sampling procedures and data collection cycles.

3. Describe methods to maximize response rates and to deal with issues of non-response.

Questionnaire Design and Training:

0. The study minimizes collection of data to that which is absolutely necessary to meet the stated objectives. Questionnaires are extensively reviewed by NAHMS staff, NASS staff, and experts in industry and in academia.

0. Pretesting activities were exercised both at NASS and NAHMS (see the answer to question 4). Results from the pretesting activities were incorporated into the final questionnaire design.

0. For Phase I data collection, NAHMS staff will develop training materials for NASS enumerators that explain the purpose of the study and address anticipated difficulties with questions. NAHMS representatives will participate in the NASS enumerator trainings.

0. For Phase II data collection, NAHMS State Coordinators will receive specialized training via NAHMS headquarters staff and in return train the other VS data collectors in their State.

0. The goat specialist for NAHMS has made numerous contacts and collaborative efforts to identify the information needs of the industry and the best way to ask for that information via questionnaire.

Contacting Respondents:

0. A pre-survey letter will be sent along with an informational brochure to both notify potential participants of the upcoming study and to provide education on the study efforts.

0. The study has been announced and is supported by the American Goat Federation, and American Dairy Goat Association (ADGA).

0. Producers will be called by the NASS enumerator up to seven times followed by an on farm visit before they are listed as a refused or inaccessible operation. NASS enumerators have gone through specific training to help them answer questions of reluctant producers so as to maximize response rates.

0. The APHIS-designated data collectors will contact farms that have consented to continue in Phase II of the study and set up a convenient time for the producer to complete the questionnaire and conduct biological sampling. Training for the APHIS-designated data collectors will include specific suggestions from the NASS trainers based upon their experience in avoiding refusals.

Nonresponse adjustment:

• Baseline response rates are expected to be approximately 55.0 percent for Phase I of the study. If the respondents differ substantially for the non-respondents there is potential for bias. NASS’ List Frame data may be available for both respondents and non-respondents to allow for examination of potential differences in type of responding and non-responding operations.

• Selection weights will be adjusted for nonresponse. Weights of eligible non-respondents will be transferred to responding operations that are most similar based on available data. The nonresponse adjustment will use the method of propensity scores, whereby a logistic model is constructed to model the probability of responding. The inverse of this probability is the nonresponse adjustment.

• A method to increase response rates for this study at the Phase I level will be to sample from NASS’ List Frame after the NASS 2017 Census of Agriculture data have been incorporated. This will help reduce the number of out of scope operations in the Phase I sample. In the NAHMS Goat 2009 Study, 15.3 percent of the Phase I sample were out of scope operations. Disregarding the out of scope operations in the NAHMS Goat 2009 Study resulted in a response rate of 61.8 percent compared to a response rate of 52.4 including these operations. Therefore, still accounting for some proportion of the Phase I sample to be out of scope, we expect approximately a 55.0 percent response rate at Phase I. See Table C.1 in Appendix C for examples of historic response rates to similar studies.

• We plan to implement a face-to-face enumerated Phase II questionnaire. This is expected to increase response probability compared to the mail-out Phase II questionnaire implemented in the NAHMS Goat 2009 Study. The mail-out Phase II questionnaire resulted in a 44.1 percent response rate in the NAHMS Goat 2009 Study, while the face-to-face enumeration response rate, adjusted for out of scope operations was 61.8 percent. Also, in the NAHMS Sheep 2011 Study, Phase II response rates were approximately 61.4 percent (see Table C.1 in Appendix C). Therefore, we expect Phase II response rates of approximately 61.4 percent because face-to-face enumeration has historically been shown to produce response rates in NAHMS studies greater than those in studies implementing mail-out or phone-based surveys.

4. Describe any test procedures or methods to be undertaken.

The proposed questionnaires will be pretested prior to field enumeration, involving fewer than 10 respondents. The NASS questionnaire will be pretested under a previously approved generic OMB clearance on 20 operations. Results of these pretests will be utilized to refine the questionnaires in order to reduce respondent burden and improve the usefulness of the information. Many revised questions from the previous goat study will be used. Final questionnaires will have been reviewed by a variety of experts, including academic researchers, individual representatives, extension agents, veterinarians, health specialists, and epidemiologists.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and /or analyze the information for the agency.

The statistical aspects of the design were coordinated by:

• Mr. Matthew Branan (970-494-7349), Mathematical Statistician, NAHMS, USDA APHIS, VS, CEAH, M&M, Fort Collins, CO.

For questionnaire design and methodology, NAHMS coordinated with:

• Ms. Rachel Sloan, Methodology Division Standards and Survey Development Methodology Branch, USDA, NASS, Washington, D.C. (202-720-3159).

The actual data collection will be conducted by NASS enumerators (Phase I) and VS-designated data collectors (Phase II). Contact persons for data collection are:

• Mr. Gerald Tillman, Chief, Survey Administration Branch, USDA, NASS, Washington, D.C. (202-720-3895).

• Dr. Bruce Wagner, Director, Centers for Epidemiology and Animal Health (CEAH), USDA, APHIS, VS, Fort Collins, CO (970-494-7256).

• Dr. Jack Shere, Deputy Administrator, Veterinary Services, USDA, APHIS, Washington, D.C. (202-799-7146).

Analysis of the data will be accomplished by NAHMS veterinarians, epidemiologists, and statisticians under the direction of:

• Dr. Amy Delgado, Director, Monitoring and Modeling (M&M), USDA, APHIS, VS, CEAH, Fort Collins, CO (970-494-7302).

Appendix A: State selection

State selection was performed based on the percentage of operations and of goats by State and by goat type. State-level percentage data by operation type were taken from the NASS “Sheep and Goats”, January, 2016 report and the NAHMS “Goat and Kid Predator and Nonpredator Death Loss in the United States, 2015” report data.

Table A.1. Percentage of operations and percentage of goats on operations with 5 or more adult goats, by State and by goat type. Note, columns may not sum to totals due to rounding. Bolded rows indicate States chosen for participation in the study.

	Percent Operations				Percent Inventory
Operation type	Dairy	Meat/Other	Unknown	All	Dairy	Meat/Other	Unknown	All
State
AK	0.1	0.0	0.1	0.1	0.1	0.0	0.1	0.0
AL	1.4	3.1	3.0	2.9	0.7	1.9	2.4	1.9
AR	2.0	2.1	2.1	2.1	1.0	1.5	1.8	1.5
AZ	1.1	3.5	0.7	2.4	0.8	2.0	0.6	1.5
CA	4.0	2.3	2.7	2.6	9.0	3.7	4.7	4.6
CO	2.6	1.4	2.4	1.9	2.5	1.2	2.2	1.6
CT	0.8	0.2	0.2	0.3	0.5	0.1	0.2	0.1
DE	0.1	0.1	0.1	0.1	0.0	0.1	0.1	0.1
FL	3.1	3.0	3.7	3.2	1.7	1.8	3.3	2.1
GA	1.6	3.6	4.9	3.8	1.0	2.3	4.2	2.6
HI	0.3	0.4	0.3	0.3	0.2	0.7	0.4	0.5
IA	3.1	1.6	1.6	1.8	8.8	1.5	2.2	2.6
ID	1.4	0.5	1.0	0.8	0.8	0.6	1.1	0.7
IL	2.0	1.5	1.8	1.6	1.8	1.0	1.7	1.3
IN	3.8	2.0	2.1	2.2	2.5	1.4	1.7	1.6
KS	1.6	1.6	1.6	1.6	1.1	2.2	2.1	2.0
KY	3.0	3.5	4.3	3.7	1.5	2.3	3.3	2.5
LA	0.6	1.1	1.4	1.1	0.3	0.5	0.8	0.6
MA	1.0	0.3	0.4	0.4	0.6	0.2	0.2	0.3
MD	1.0	0.7	0.7	0.7	0.8	0.4	0.5	0.5
ME	1.0	0.2	0.5	0.3	0.6	0.1	0.3	0.2
MI	3.9	1.1	1.5	1.6	2.3	0.6	1.1	0.9
MN	2.4	1.1	1.1	1.2	3.2	0.9	1.2	1.2
MO	4.0	3.3	3.8	3.5	3.3	2.8	3.9	3.1
MS	0.8	1.4	1.8	1.5	0.4	0.9	1.3	0.9
MT	1.1	0.3	0.3	0.4	0.6	0.4	0.3	0.4
NC	2.8	3.5	3.1	3.3	2.1	2.2	2.5	2.2
ND	0.4	0.2	0.2	0.2	0.2	0.2	0.2	0.2
NE	1.1	0.7	0.8	0.8	0.8	0.8	0.9	0.8
NH	0.7	0.1	0.2	0.2	0.4	0.1	0.1	0.1
NJ	0.9	0.7	0.5	0.6	0.5	0.4	0.4	0.4
NM	1.1	1.9	1.1	1.6	0.7	1.2	1.1	1.1
NV	0.2	0.2	0.3	0.2	1.9	0.1	0.2	0.4
NY	3.7	1.3	1.5	1.6	3.8	0.9	1.4	1.4
OH	4.2	3.1	3.7	3.4	2.5	2.0	2.6	2.2
OK - E	1.2	1.8	2.0	1.8	0.7	1.4	2.2	1.5
OK - w	1.6	2.1	2.5	2.2	0.9	1.9	2.6	2.0
OR	3.6	1.5	1.8	1.8	2.5	1.2	1.3	1.4
PA	3.6	2.4	2.2	2.5	3.2	1.5	2.1	1.9
RI	0.2	0.0	0.0	0.1	0.1	0.0	0.0	0.0
SC	1.4	2.4	2.1	2.2	0.6	1.3	1.8	1.3
SD	0.8	0.4	0.4	0.4	0.7	0.6	0.6	0.6
TN	2.9	6.0	4.7	5.2	1.6	4.3	4.4	4.0
TX - E	3.4	6.7	6.5	6.2	2.1	4.7	5.7	4.6
TX - W	5.6	18.3	13.0	15.2	3.9	39.7	19.6	30.4
UT	0.9	0.6	0.6	0.7	0.5	0.5	0.5	0.5
VA	2.4	2.5	2.6	2.5	1.4	1.7	2.1	1.8
VT	0.8	0.1	0.2	0.2	1.5	0.1	0.2	0.3
WA	2.8	1.3	2.4	1.8	2.1	0.7	1.5	1.0
WI	4.3	1.0	1.3	1.5	17.8	0.6	2.6	3.3
WV	1.4	1.3	1.7	1.5	0.6	0.8	1.2	0.9
WY	0.6	0.2	0.3	0.3	0.4	0.5	0.4	0.5
Percent for 25 chosen States	76.7	77.8	78.9	77.9	83.2	83.4	81.3	82.7

Appendix B: Expected estimates of precision

Estimates of percent of operations and percent of animals will be reported at the national level, for region by operation-type categories, and by size categories.

Estimates of precision are shown for proportions of 0.5 and 0.1. As an example, for the West-Dairy category and an expected proportion of 0.5, the CV is 6 percent. None of the estimated CVs in the examples presented here exceed 20 percent.

Table B.1. Precision of estimates by reporting class and by expected proportion, at 95 percent confidence for Phase I.

					Phase I
Region – Operation type	Approx. Sample size*	Number of respondents at 55.0% response rate	Effective sample size (DE = 1.5)	Expected proportion	Half-width of confidence interval	Coefficient of variation (%)
West – Dairy	804	442	295	0.5	0.06	6
				0.1	0.03	17
West - Other	1,132	623	415	0.5	0.05	5
				0.1	0.03	15
East – Dairy	1,313	722	481	0.5	0.04	5
				0.1	0.03	14
East - Other	1,521	837	558	0.5	0.04	4
				0.1	0.02	13
Total	4,770	2624	1749	0.5	0.02	2
				0.1	0.01	7

* Approx. sample size was computed by allocating the overall sample size to each of the strata, proportional to the weighted number of operations and goat inventory in that stratum, while accounting for oversampling in Dairy and oversampling in the West strata by sampling a higher rate than suggested by proportional sampling for those strata.

Table B.2. Precision of estimates by reporting class and by expected proportion, at 95 percent confidence for Phase II. Only two of the estimated CVs in the examples presented here exceed 20 percent, both for estimated proportions of 0.1.

					Phase II
Region – Operation type	Approx. Sample size at 78.4% consent rate*	Number of respondents at 61.2% response rate	Effective sample size (DE = 1.5)	Expected proportion	Half-width of confidence interval	Coefficient of variation (%)
West – Dairy	347	213	142	0.5	0.08	8
				0.1	0.05	25
West - Other	488	300	200	0.5	0.07	7
				0.1	0.04	21
East – Dairy	566	348	232	0.5	0.06	7
				0.1	0.04	20
East - Other	656	403	268	0.5	0.06	6
				0.1	0.04	18
Total	2057	1263	842	0.5	0.03	3
				0.1	0.02	10

*Approx. sample size was taken as a percentage of consenting operations from Phase I respondents from Table B.1.

Appendix C: Response rates

Table C.1. Response rates from the NAHMS Goat 2009 and the NAHMS Sheep 2011 Studies.

Survey	Sample	Complete	% Complete	Out of scope operations1	% Complete - Adjusted
2009 General Goat Management Report	3,501	1,835	52.4	534	61.8
Goat 2009 VS Initial Visit Questionnaire Consent	1,835	1,438	78.4
Goat 2009 VS Initial Visit Questionnaire2	1,438	634	44.1
Sheep 2011 VS Initial Questionnaire3	1,241	761	61.4

¹This includes operations that were out of scope for the study design definitions, out of business, inaccessible, and those that had no animals at the time of the study.

²This questionnaire was a mail-out only, no face-to-face enumeration at this phase was performed like with the first two items. Face-to-face enumeration for NAHMS studies have historically resulted in higher response rates than mail-out surveys.

³The VS Initial Questionnaire from the NAHMS Sheep 2011 Study was enumerated face-to-face, unlike the same questionnaire from the NAHMS Goat 2009 Study, and thus serves as an estimate of the response expected during the second phase of the NAHMS Goat 2019 Study.

Appendix D: Burden estimates

Table D.1. Response burden estimates from the NAHMS Goat 2009, NAHMS Sheep 2011, and NAHMS Swine 2012 Studies (in minutes).

Survey	Average burden for respondents (min.)	Average burden for nonrespondents (min.)
2009 General Goat Management Report	55.8	11.8
Goat 2009 Initial Visit Questionnaire*	26.6
Sheep 2011 General Management Report	64.1	22.8
Sheep 2011 VS Initial Visit Questionnaire	89.6
Sheep 2011 Individual Animal Fecal Collection Record	126.0
Sheep 2011 Blood and Swab Collection Record	150.0
Swine 2012 Blood Collection Record	96.0

^*Mail-out questionnaire

1 Alabama, Alaska, California, Colorado, Connecticut, Florida, Georgia, Indiana, Iowa, Kentucky, Michigan, Minnesota, Missouri, New York, North Carolina, Ohio, Oklahoma, Oregon, Pennsylvania, Tennessee, Texas, Vermont, Virginia, Washington, Wisconsin

Page 10 11/1/2018