B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
The universe for this study is comprised of all registered users of the two online tools as described above. The universe consists of a self-selected sample of the target group–farmers and Extension educators–who have expressed an interest in learning through the internet how to farm organically.1
Users of the FarmSelect tools will be recruited through promotional activities and presentations at professional meetings, farmer conferences, e-mail newsletter campaigns, e-mail contacts with farmer organizations members, advertising campaigns at Schools of Agriculture and other professional events and activities for farmers and Extension specialists that will approximate a purposive sampling process.
Research on the dynamics of transitioning to organic agriculture is very limited. This lack of information constrains our ability to accurately estimate usage rates of the FarmSelect tools and the size and composition of the universe. While a growing number of studies have documented the characteristics and preferences of consumers who buy organic foods, we know very little about the demographics or the information and training needs of U.S. farmers who start farming organically and the professional needs of the agriculture educators who support them. This study is therefore exploratory and will make a unique contribution to the body of knowledge.
Recent ERS reports estimate the number of certified organic farms in the United States at around 8,000 in 2003 with an average annual growth of about 10% during the last decade (see ERS, 2005). Other sources estimate the number of organic farmers in the U.S.–including non-certified operations and subcontracted organic farms, as high as 20,000 (see NBJ, 2003). Based on the findings of these studies we estimate that about 1,400 farmers will start farming organically in 2007 in the U.S. and about 560 of them will seek certification for their operations.
The USDA’s National Agricultural Library (NAL) website lists 159 US Colleges and Universities offering Educational and Training Opportunities in Sustainable Agriculture2. However, only a handful of them offer organic classes or have faculty involved in organics. Furthermore, while interest in organic education and curricula development is growing, less effort is directed at hands-on farmer training as compared to organic research (see Silverman, 2005). There is no reliable information about what proportion of the over 6,500 Agriculture and Natural Resources Extension FTEs within the Cooperative Extension Service3 are involved in organics.
A pilot study of a draft version of the FarmSelect™ calculator during 2005 generated, on average, about 490 unique visitors a month. In addition, about 2,000 farmers and 500 agriculture educators have subscribed to The Rodale Institute’s Newfarm.org newsletter–an online source of information about organic agriculture. The Rodale Institute® received about 4,000 online requests for information about organic practices in 2005. Given that both the pilot study and the newsletter were minimally advertised or promoted, we believe that these participation rates represent the lower range of the potential level of interest in the online tools.
Thus, based on information from the pilot studies and within the context of national trends and patterns in U.S. organic agriculture and organic education, we estimate that at least 10,000 individuals will access the FarmSelect calculator and online course annually and about 2,000 of them will register to take full advantage of all features of the online tools.
Based on survey results from the Organic Farming Research Foundation (OFRF) national surveys of organic farmers and The Rodale Institute’s web-based surveys of sustainable farmers, we believe that the majority of registered users will be: (1) conventional farmers who are interested in transitioning to organic production, followed by (2) organic farmers who seek to improve, diversify, or certify their operation, and (3) Extension specialists who receive farmer requests for information and training in organic agriculture.
Recent research findings indicate no consistent differences in response rates between surveys conducted via e-mail or the Web and more traditional mail or phone surveys (see Shonlau et al, 2002). While Internet-based surveys, and research about them, are still in their infancy, studies suggest that traditional methods used to increase response rates for mail and phone data collections are relevant to this new mode as well (see Dillman, 2000). We therefore expect a response rate of about 80%.
The second component of this data collection is the Beta test of the FarmSelect calculator and online course. Information and assumptions about the universe of the evaluation surveys and the target population of this project outlined above form the strategy for the Beta test of the two online tools.
This project will involve a targeted Beta test with a judgment sample of sixty individuals representing the three categories of the target population. An expert panel with representatives from The Rodale Institute®, the Office of Extension and the Department of Instructional Design and Development—World Campus, Outreach, at Pennsylvania State University will select participants in the Beta test. We expect an initial contact with 75 individuals–25 each--conventional farmers interested in transitioning to organic, organic farmers, and Extension specialists responding to farmer requests in organic agriculture, will result in recruiting 60 participants.
Beta test participants will be given a token payment for their time testing the products and submitting feedback to the design team. Data collection will be done using the web forms enclosed in this package. We estimate an 80% response rate for Beta testing participants.
2. Procedures for the collection of information
This is an electronic data collection that will use e-mail and the Web.4 The surveys will be designed and administered using Perseus Survey Solutions Enterprise software. This product has advanced Survey Panel Management capabilities that allow tracking users over time, keeping updates on individual responses, scheduling and e-mailing reminder notes, and other features that will increase the efficiency and effectiveness of data collection and will greatly improve survey data quality.
This study uses fourfive data collection instruments, namely (1) Beta test survey questionnaires; (2) User registration forms; (3) User baseline surveys; and (4) User satisfaction surveys. The survey forms are customized for each tool (OTSM and training course) and for farmers and Extension specialist as two distinct target user groups. With the exception of the Beta test, all information collections are in compliance with the original pretest-posttest single group design of the evaluation of the project (see Appendix 1, Section VII, Task 7). A schematic describing the information collection process appears in Appendix IV. Copies of the data collection instruments appear in Appendix V.
Evaluation
As noted above, the universe for this evaluation study is comprised of all registered users of the FarmSelect calculator and online course. We will use a mixed mode (Web, e-mail, telephone) approach to contact participants and to collect the survey information. Initial contacts, invitations to register, and completion of the baseline surveys and satisfaction surveys will be conducted on the Web and will be integrated with the training tools.
The first step in the process will be to invite users of FarmSelect to register. Three registration form options–for farmers, Extension specialists, and a general “Other” category will collect contact information and some basic demographic information about the users of the tools. Participants will obtain user IDs and passwords for login into the system. Information for each user will be stored and saved as an individual record. To register the user is only required to provide name, address and e-mail address, all other information is optional.
At login, farmers and Extension specialists will be invited to complete a brief baseline survey. Two Web-based forms–one for each user group--will collect information about the participant’s interest, knowledge, and practice of organics prior to using the FarmSelect training tools. The survey will be designed as an integral part of the training tools and will take approximately 10 minutes to complete. Non-respondents will be sent reminder e-mails one week after registration.
Users will be invited to complete a satisfaction survey following their first complete use of the course and calculator. This survey will take approximately 7 minutes to complete. Separate forms will be used for the calculator and for the online course, individually tailored for farmers and Extension workers. Non-respondents will be sent reminder e-mails one week after complete use of the training tools. After four e-mail contacts and four weeks after completion a decision will be made whether to follow up by phone.
Beta Test
We will use a mixed method approach for the Beta testing process and will establish first contacts by phone and/or e-mail. Participants will be asked to submit a formal agreement to participate via e-mail. Contacts and reminder messages will be sent by e-mail and the data collection will be conducted using two Web-based questionnaire forms–one for each of the FarmSelect training tools. Links to the questionnaire forms will be included in the invitation e-mails along with links to the test versions of the tools. The invitation e-mails will provide e-mail addresses and telephone numbers that the participants can call if they have any questions or concerns about any aspects of the study. Participants in the Beta test will be provided with IDs and passwords to access the Web based questionnaire forms and submit their feedback.
3. Describe methods to maximize response rate; information collected adequate for intended uses
The Rodale Institute® has significant experience addressing the information and training needs of farmers and Extension specialists interested in organic methods. They will apply a Web adaptation of the Tailored Design Method (Dillman, 2000). Multiple contacts, respondent-friendly formats, communications emphasizing the surveys usefulness, and the importance of each individual response are of primary importance in this design.
We expect that the most effective factor in achieving a high response rate will be the multiple personalized contacts with respondents. Registered users of the FarmSelect tools will have multiple contacts with the FarmSelect product team via e-mail and the Web as part of the process of using the tools. In addition, sponsorship from The Rodale Institute, a well-respected organic farming research organization is an important factor in establishing trust with the respondents. Furthermore, the time burden for taking the surveys is negligible compared to the overall time commitment for using the tools and the economic benefits (farm plan and customized budget) the tools offer.
There are several factors that we expect to bear favorably on the response rate for the evaluation surveys, including: (1) Topic saliency. The shared characteristic of users of the FarmSelect tools is interest in learning how to farm organically. Meaningful use of the tools requires time commitment and a goal-oriented attitude. We expect registered users of the tools to be very motivated and more willing to share their feedback via a survey. (2) Incentive/ Economic benefit to respondents. The FarmSelect tools provide registered users with the ability to develop personalized budgets and a Farm Plan to be submitted directly to a certification agency – two resources that are not available to non-registered users. The value of these resources to a transitioning farmer is significant and will be a strong incentive to register and use the tools. (3) Respondent-friendly design. One of the advantages of web-based surveys is that they are easy to use and impose a minimal burden to the respondents–through drop-down menus, easy click-to-choose patterns, and question branching and skip-patterns. (4) Survey context. All survey questionnaires for this data collection will be designed and administered as integral components of the two online tools. Minimal time burden and a seamless integration with the FarmSelect calculator and course modules will also increase the response rate. (5) Questionnaire length. The survey questionnaires are designed with a minimalist approach. Surveys will be administered at different points in time (baseline, midterm, and final) with a one-time burden of up to 10 minutes–this is a negligible time commitment compared to the estimated 24 hours or more to complete the online course.
In addition to the factors listed above, we will be taking several steps in questionnaire design and administration to achieve high response rates. More specifically we will:
(1) Follow a multiple contact strategy–web and e-mail contacts with invitation and links to the web survey form; phone follow ups for Beta testers.
(2) Personalize e-mail contacts so that none show as part of a mass mailing.
(3) Keep cover e-mails brief to allow respondents quick access to the survey form link.
(4) Include links and login information with each reminder e-mail message.
(5) Design Web questionnaire forms to minimize the likelihood of wrap-around text.
(6) Introduce Web questionnaire with a welcome screen that is motivational and inviting.
(7) Use consistency in visual appearance, color, and formats.
(8) Provide specific instructions on how to respond to each question.
(9) Use scroll-down rather than multiple-page forms whenever appropriate
(10) Provide a progress indicator if multiple page formats are used.
The Rodale Institute® has comprehensive experience in assessing the educational and training needs of farmers and Extension involved in organic agriculture. The team has specific expertise in evaluating the effectiveness of online tools and applications and knowledge of the information requirements for this analysis. The survey instruments were designed specifically for this research.
As indicated above, the data collection procedure was designed to achieve a response rate of about 80 percent. To evaluate non-response bias we will compare respondents to non-respondents across background demographic and attitudinal variables available for all registered users. Given the exploratory nature of this research, we will compare respondent information to data from public sources that provide relevant proxy measures including USDA (ERS, NASS, NAL) data on trends in organic agriculture, Organic Farming Research Foundation bi-annual surveys of organic farmers, The Rodale Institute’s web surveys of sustainable farmers, and web traffic data about The Rodale Institute’s online tools and resources for organic agriculture.
4. Test of procedures or methods.
The procedures and questions to be used in this data collection have been tested in mail, telephone, and online surveys of farmers and Extension conducted by The Rodale Institute® during the last four years. The survey instruments have been informally tested with a small number of farmers and agriculture educators, users of The Rodale Institute online informational resources and reviewed by Pennsylvania State University extension specialists.
5. Individuals consulted on statistical aspects of survey.
Lynn M. Daft, Ph. D.
Executive Vice President
Promar International, Inc.
Alexandria, VA
Theodore R. Alter Ph. D.
Professor of Agricultural,
Environmental & Regional Economics
College of Agricultural Sciences
The Pennsylvania State University
204 Armsby
University Park, PA 16802
Jeffrey Hyde, Ph.D.
Assistant Professor of Agricultural Economics
The Pennsylvania State University
210-B Armsby
University Park, PA 16802
Evelina Panayotova, Ph.D.
Monitoring and Evaluation Manager
The Rodale Institute
611 Siegfriedale Road
Kutztown, PA 19530-9749
Paul Reed Hepperly, Ph.D.
Research Manager
The Rodale Institute
611 Siegfriedale Road
Kutztown, PA 19530-9749
Jeffrey W. Moyer
Farm Manager
The Rodale Institute
611 Siegfriedale Road
Kutztown, PA 19530-9749
Maria Pop
Training Coordinator
The Rodale Institute
611 Siegfriedale Road
Kutztown, PA 19530-9749
Lawrence C. Ragan, Ph.
D.
Director- Instructional Design & Development
The
Pennsylvania State University
115 Mitchell Bldg.
University
Park, PA 16802
Greg Bowman
Online Editor–newfarm.org
The Rodale Institute
611 Siegfriedale Road
Kutztown, PA 19530-9749
Bill Klepeiss
Project Manager
The Rodale Institute
611 Siegfriedale Road
Kutztown, PA 19530-9749
References
Cochran, W. G. 1977. Sampling Techniques. New York: John Wiley and Sons.
Dillman, D.A. 2000. Mail and Internet Surveys. The Tailored Design Method. New York: John Wiley and Sons.
Dimitri, Carolyn, and Catherine Greene. 2002. “Recent Growth Patterns in the U.S. Organic Foods Market.” U.S. Department of Agriculture, Economic Research Service, Market and Trade Economics Division and Resource Economics Division, Agriculture Information Bulletin No. 777, Sept. 2002.
Greene, Catherine and Amy Kremen. 2003. “U.S. Organic Farming in 2000-2001: Adoption of Certified Systems.” U.S. Department of Agriculture, Economic Research Service, Resource Economics Division, Agriculture Information Bulletin No. 780, Feb. 2003.
Nutrition Business Journal (NBJ). 2003. “The NBJ/Spins Organic Foods Report 2003.” Penton Media, Inc., Apr. 2003.
ERS. 2005. “Organic Production, 1992-2003.” U.S. Department of Agriculture, Economic Research Service. Updated 16, Nov. 2005. Retrieved 6 Dec. 2005 <http://www.ers.usda.gov/Data/Organic/index.htm>
Schonlau, Matthias, Ronald D. Fricker, Jr., and Marc N. Elliott. 2002. Conducting Research Surveys via E-Mail and the Web. Santa Monica, California: RAND.
Silverman, Julia. 2005. “Academics propose including organic farming in curriculum”, Capital Press, Salem, OR, July 29.
Walz, Erica. “Final Results of the Fourth National Organic Farmers’ Survey: Sustaining Organic Farms in a Changing Organic Marketplace.” Organic Farming Research Foundation, 2004.
1 The 2002 Census of Agriculture (NASS 2002) shows about 50% of U.S. farms reporting to have access to the Internet. National surveys conducted by the Organic Farming Research Foundation (OFRF) show the percentage to be much higher for organic farmers at about 72%. All Extension offices have access to the Internet.
2 Retrieved from http://www.nal.usda.gov/afsic/AFSIC_pubs/edtr.htm#Colleges
3 Ahearn, M., Yee. J., and Bottum, J. Regional trends in Extension System Resources. Agriculture Information Bulletin No. 781. U. S. Department of Agriculture, Economic Research Service. April 2003. Retrieved from: http://www.ers.usda.gov/publications/aib781/aib781.pdf
4 We will use telephone contact for follow-up with non-respondents if the response rate is lower than anticipated.
| File Type | application/msword |
| File Title | OTSM Supporting Statement B |
| Author | Shannon Persetic |
| Last Modified By | Shannon Persetic |
| File Modified | 2007-09-25 |
| File Created | 2007-09-25 |