Comments/responses Q: 8

Comment # 1: Dr. William Borrie, Professor of Park and Recreation Management,

University of Montana. June 26, 2008.

As I read through the supporting statement, I get the sense that the main focus of the study is examination of the 'Preferred & trusted means of communication among minority groups'. This is a very necessary and relevant topic, and one that should be of great utility to the USDA

Forest Service as they work with increasingly diverse U.S. populations.

Although outside my direct area of expertise, I suspect there is already some work in the area of cross-cultural communications that could provide a solid theoretical foundation for this work. I would imagine that different ethnic and racial groups have distinct preferences and uses for various forms of media, and thus marketing targeted towards different groups can be quite distinct. In addition, I suggest that some concurrent research examine how natural resources (and outdoor recreation in particular) is/are portrayed in 'ethnic' media. More work on documenting Latin/Hispanic, Asian American, and African American environmental ethics is also foundational to the interests of the current study. Building off existing understandings allows for more receptivity towards, and cohesion/stickiness of, communication and marketing messages.

I applaud the implementation of a communication needs analysis prior to any substantial marketing effort. As was the case with my work on public-purpose marketing (Borrie et al, 2002. Journal of Park and Recreation Administration, Vol. 20, No. 2, pp. 49-68), it is important

to segment the general population (based, say, on prior levels of trust, commitment to the environment, values, etc) and not just focus on current recreation visitors to the National Forests. That way, more targeted messages can be utilized for each of the identified segments.

Further, any attempts to change behavior of minority groups, such as towards greater visitation and use of National Forests, should be based on solid educational and psychological theory. Simply raising awareness of recreation opportunities (i.e. which behavior is

encouraged/recommended) is often insufficient. In my work on Low-Impact Recommendations (eg. Harding, Borrie & Cole 2000 Wilderness science in a time of change conference- Volume 4: Wilderness visitors, experiences, and visitor management; 1999 May 23-27; Missoula, MT.

Proceedings RMRS-P-15-VOL-4., p. 198-202) I have argued that educational efforts must also target i) identification of the need for a behavior change, ii) ready cognitive accessibility of desired behavior, iii) personal motivation for choosing desired behavior , and iv) social or

normative reinforcement for desired behavior. For instance, following this model communication efforts would focus not just on awareness of recreational opportunities but also on i) how outdoor recreation can fit within the culture and lifestyle of different minority groups, ii) familiarization / practice of different recreation opportunities, iii) benefits of outdoor recreation, and iv) creation of new norms and patterns of behavior (established through existing, influential cultural groups such as church, Boy Scouts, schools, sports clubs, etc.).

I am a little unclear how some of the telephone survey questions are based on, or build on, previous work. I know the authors are familiar with the extensive literature on leisure constraints (such as the work of the Diversity Research Laboratory at the University of Illinois) and with the extensive literature on barriers to outdoor recreation participation (indeed, much of the work has been with the PSW and Southern research stations of the USDA Forest Service), but there wasn't much reference to that work in this supporting statement. A more complete study plan and literature review would probably have helped demonstrate the clear contribution of previous work and the extension that this survey will provide, I guess.

Methodologically, I support the use of telephone surveys but am increasingly frustrated in their implementation. It is getting increasingly hard to get acceptable patterns of response, including

adequate response rates when surveying the general public. It may be time for us all to make better use of internet surveys, as well as door-to-door surveys. Dillman's latest edition (2007 update) seems to suggest as much, particularly given the prominence of cell phones, unlisted numbers, and do-not-call lists.

Additionally, you might want to give a little more time to the sampling approach. I think you mention quota samples, and if you mean continuing random-digit-dialing until specific group quotas (such as 500 responses for each of the four major ethnic/racial groups) are met, then

I think that is an efficient sampling approach for the objectives of this study (which is not claiming to be representative of the overall population). I would also think you need to present a clearer definition of what is a metropolitan area, and what qualifies as an urban area adjacent to National Forests.

Forest Service Response:

The literature review contained in this form has been enhanced to include the appropriate literature, although Dr. Borrie’s interests extend beyond the purposes of this study in regards to a number of additional suggested areas of inquiry. This is a proposed telephone survey and must be kept brief in order to secure a positive response rate. His suggestion for door-to-door research is impractical and would severely impact ability to work in distant areas and further challenge limited resources. Limitations in access to the Internet are well documented and continue to challenge recommendations to conduct research via the web. Sampling is well laid out in our opinion and appropriately describes our research plan. We have added a citation for metropolitan area for purposes of clarification.

Comment # 2: Dr. Carolyn Ward, Professor, Humboldt State University

Everything looked fine to me. The only thing that caught my eye (which was brought to my attention on an earlier study of mine) was a line somewhere in the introduction that (if it is true!) says that all responses and respondents are anonymous and confidential and that they have the right to stop the interview at any time during the phone call. Everything else looked fairly textbook.

I have also attached a recent AP story that you might be able to reference for this...

Forest Service Response:

The respondents are assured their responses are anonymous and they can stop the interview at any time. A reference to the report that Dr. Ward cites is included in our summary of literature.

Comment # 3: Dr. Corliss Wilson-Outley, Assistant Professor, Texas A & M University, July 9, 2008.

Dr. Wilson-Outley’s remarks appear in detail on the attached documents. On the draft supplemental statement she raised several questions and had suggestions for areas that should be extended. We have added to our literature review and clarified wording to address her concerns. She also encouraged a commitment to distribution of results among communities of color, which we have added to our research plans. We feel that each methodology we might employ has its own unique constraints, but a telephone survey offers the best option for our area of interest.

Comments on the draft survey were also addressed through modification where appropriate. Based on prior work of others we have found that asking respondents to report television channel watched is counter productive. Channels vary based on delivery of television signal (cable, antenna, satellite) and programs are sometimes geographic specific. Without a readily reliable resource outlining a standardized listing of channels and programs from each area/service we feel it best to omit this question. Radio stations however are much more reliable and we have found an external listing for those. We already know language of programming without needing to gather this information from respondents.

Federal Reviews

Reviews were gathered from Dr. Haiganoush Preisler (Research Statistician), and David Dillard (Commodity Section; USDA / NASS / SD / SMB). Their comments and Agency responses follow.

Comment # 4: Dr. Haignaoush Preisler, Research Statistician, USDA Forest Service, Pacific Southwest Research Station. June 17, 2008.

1. Page 7: You might want to add a reference (or just state ‘from previous experience’) to the claim that 500 individuals are expected to decline and 2000 are expected to participate etc.

2. Page 13 table: I think this table is simply obtained using the formula

which is largest when p=.5. So the assumption here is that we have binary response (Yes/No questions). You might want to add this statement to the first line on page 13. That is: assuming a sample of respondents of size N out of a population of 5000 or more, then the 95% error rate for questions with binary response is proportional to one over the square root of the sample size. (Or some such sentence).

3. In the Questionnaire introduction statement. How about finishing the intro with ‘study we are conducting on [fill ..] County communities regarding National Forest use.’

I feel I will be more open to respond if I knew that it is about National Forests.

4. Same comment for question 2). ‘We are interested in studying possible ethnic differences that exist in LA county regarding NF …’

I might not want to talk about ethnic difference in general but will be more willing to discuss National Forests which seems to me less ‘threatening’!

5. Questions 14 – 22 may be better if moved after question 2. If I was being surveyed and the questions started with hard ones like how many hours of TV etc, I am going to lose interest and get annoyed very quickly. While questions 14-22 seem easier to answer.

Forest Service Response: These recommendations were followed and the attached form reflects the changes made.

Comment # 5: Dr. David Dillard, Commodity Section; USDA / NASS / SD / SMB. October 21, 2008.

This document analyzes the Forest Service’s OMB request to conduct a phone (CATI) survey to collect information about media sources certain race/ethnic groups (White, Hispanic, Asian, and African American) use to gather information. Once the Forest Service identifies specific media sources used by these groups, it plans to use these media to acquaint the groups with information on recreational uses of national forests located close to urban areas.

The request provided ample documentation of differences among the four ethnic groups of interest with respect to how they obtain information. The documentation was also well supported by subject matter experts contacted by the Forest Service for advice on how to proceed with the survey.

Since this survey is not a probability survey, I believe the steps outlined to identify potential respondents are sound. They are obviously well thought out and, like other documentation in the request, well supported. One of the subject matter experts, Dr. William Borrie, discussed weaknesses to phone surveys and suggested more extensive use of Internet surveys. I cannot disagree with Dr. Borrie’s statements. However, for this survey, I believe phone data collection is adequate and will provide the information the Forest Service desires.

The Forest Service request states that it expects an 80 percent response, but doesn’t say how it arrived at that estimate. It also plans to make up to 12 follow-up calls to non-answers, which seems ample. The Forest Service said it had pre-tested the questionnaire on nine individuals, collecting data in person and asking respondents about possible confusion with the questions. I applaud the Forest Service for asking respondents to identify wording problems in the instrument, but I suggest that it conduct more pre-testing using operational (i.e., phone) procedures.

Overall, the Forest Service request is well documented, well supported by research results and advice from subject matter experts, and consistent with the legislation mentioned in PL-95-307 and EO 12898.

My final comments concern the questionnaire (CATI instrument) and data collection procedures. When I collected survey data, I avoided offering potential respondents a chance to discontinue the interview. (However, we are required by law to tell the respondent that cooperation is voluntary.) I’ve noticed that phone solicitors have become more aggressive in their techniques. Instead of asking me whether I would like to contribute, they ask if I’m comfortable giving a specific amount which they designate. In keeping with that philosophy, I would drop the introductory question “Would you like to participate?” At the least, I would re-phrase it as “Would you be willing to participate?” I would also drop the statement “I promise I’m not trying to sell you anything,” which could arouse suspicion. If the Forest Service is interested in collecting data from 18-year-olds, it should replace the screening question “Are you over 18 years of age?” with “Are you 18 years of age or over?” Instead of asking “Would you mind telling me your ethnicity, please?” I would ask “What is your ethnicity?” or “Which ethnic group do you most closely identify with?” Question 6 asks respondents which media source they would trust the most to obtain information about outdoor recreational opportunities. I wonder whether “trust” is the best word to use in that question. Finally, the skip instruction in Question 7 could cause confusion. In a CATI instrument, the interview should automatically branch to the correct continuation based on the response to the question. However, there should be a branch after Question 20 so the interviewer doesn’t ask Question 21, which is identical to Question 19. Finally, the documentation alluded to obtaining income data for categorization purposes, but no income questions appear in the draft questionnaire.

Forest Service Response:

The NASS review was insightful and appreciated. The 80% projected cooperation rate is based on our plan to complete up to 12 calls to any one selected household, the topic of focus for this survey, and the use of skilled interviewers. Prior experience suggests these are favorable steps that help ensure a higher response rate. Finally, access to Spanish language interviewers will ensure that our Spanish speaking population is well served and respected in our study. The wording changes suggested were made, and question branching has been modified to flow correctly.