Corps of Engineers Civil Works Questionnaires Generic Clearance

PARTICIPANT QUESTIONNAIRE TO ASSESS

THE BENEFITS OF COLLABORATIVE MODELING

(Personal Interview)

OMB 0710-0001

Expires: 30 September 2012

The public report burden for this information collection is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this data collection, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Executive Services Directorate, Information Management Division, and the Office of Information and Regulatory Affairs, Office of Management and Budget, Washington, D.C. 20503: Attn.: Desk Officer for U.S. Army Corps of Engineers. Respondents should be aware that notwithstanding any other provision of law, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Please DO NOT RETURN your completed form to either of these offices.

Participant Questionnaire to Assess

the Benefits of Collaborative Modeling

1. Please identify the name of the collaborative process for which you are completing this survey.

2. Which category best describes the interest or organization you represented in this process?

Check the most appropriate answer only

 a. Federal Government

 b. State Government

 c. Local/Regional Government

 d. Tribal Government

 e. Environmental/Conservation

 f. Recreational

 g. Industrial/Resource Extraction

 h. Business/Commercial

 i. Community or Private Citizen (e.g., neighborhood association, local resident)

 j. Special Advocacy Interests (please describe):

 k. Other (please specify):

3. Please identify the concerns that were considered during the collaborative process, and use the scale below to rank these concerns based on what you believe were the highest priorities. More than one concern may be identified in the same category (e.g., “highest priority”). Please identify and rate any additional categories that are not listed.

	Highest priority	Relatively high priority	Relatively low priority	Not a consideration
Agriculture				
Air Quality				
Archeology or Historic Preservation				
Coastal Zone or Marine Management				
Ecosystem Management				
Endangered Species and/or Critical Habitat				
Energy				
Facility (dam) Reoperation				
Forest and Timber Management				
Flood Damage Reduction (Flood Control)				
Stream Flow Regime				
Land Use and Urban Development				
Mining				
Native American, Alaska Native, Native Hawaiian Issues				
Navigation				
Parks and Refuges				
Recreational Use and Access				
Solid or Hazardous Waste				
Transportation				
Vegetation/Riparian Management				
Watershed/River Basin Management				
Water Demand Management				
Water Quality				
Water Supply				
Wildlife Management				
Other Category #1 (please describe)				
Other Category #2 (please describe)				
Other Category #3 (please describe)				

4. Using the scale below, please identify the different aspects of the process in which you were involved and rate your level of involvement. (Note that for the purposes of this questionnaire, a distinction is being made between the “planning process,” and the “modeling process,” where the latter describes the subset of activities focused on development and use of a quantitative model to evaluate alternatives. Note that the term “model development” as used herein refers to the process of identifying important variables, relationships among variables, and output requirements; it does not refer to the task of actually programming the model)

	Not involved	Involved very little	Somewhat involved	Very involved	Not Applicable
a. Planning process (overall)			*	*	---
b. Modeling process (overall)			**	**	---
c. Planning process design					---
d. Modeling process design					---
e. Problem formulation					---
f. Fact finding					---
g. Discussions/negotiations with other parties regarding alternatives					---
h. Coordination of other stakeholder input					---
i. Model development					---
j. Evaluation of alternatives					---
k. Development of recommendations					---
l. Monitoring the implementation of the agreement					
m. Other (please describe)					---

*4A. If you answered that you were “somewhat involved” or “very involved” in the planning process overall, please provide the following descriptive information about the planning process:

a. What was your role in the process?                                                                       

b. What was the duration of the process?                                                                  

c. How many meetings/calls were convened and at what frequency?                                                                       

**4B. If you answered that you were “somewhat involved” or “very involved” in the modeling process overall, please provide the following descriptive information about the modeling process:

a. What was your role in the process?                                                                       

b. What was the duration of the process?                                                                  

c. How many meetings/calls were convened and at what frequency?                                                                       

4C. If you answered either Question 4A or 4B, please rate your agreement with the following statements about the interaction of the planning and modeling processes:

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know	Not necessary
a. The planning and modeling processes were closely integrated								---
b. Some individuals were involved in both the planning and modeling processes								---
c. The scope of issues addressed by the model were defined by the planning process								---
d. Information used in the model was gathered through the planning process								---
e. The alternatives to be evaluated in the model were defined by the planning process								---
f. The model was modified as necessary to address feedback from the planning process								

5. As a representative of your organization, what was your role in communicating with others who were not as directly involved in the process.

Check all that apply

 a. I was responsible for conveying information among stakeholders within my organization

 b. I was responsible for conveying information among stakeholders outside of my organization (please describe)

 c. I had authority to make some decisions on behalf of my organization

 d. I had authority to make all decisions on behalf of my organization

 e. Other (please describe)

6. Please indicate the extent to which agreement was reached. (To answer this question, think about what it was that the group was charged to come up with at the end of this collaborative process. The term “agreement” as used herein refers to final recommendations made by the group to the decision maker or decision-making authority as a direct result of the collaborative modeling process.)

please choose only one

 Agreement reached on all key issues

 Agreement on most key issues

 Agreement on some key issues

 No agreement on any key issues, but progress was made towards addressing the issues or resolving the conflict (skip to Question 8)

 No agreement, we ended the process without making much progress (skip to Question 8)

 I was involved in the modeling process only and am not sure of the agreement status (skip to Question 8)

 The process is ongoing, and an agreement has yet to be reached (skip to Question 8)

Use this space to comment:

7. Using the scale below, rate the following statements regarding the agreement (as referred to in Question 6).

	Not at all	To a very little extent	To a little extent	To some extent	To a great extent	To a very great extent	Don’t know
a. The extent to which you feel the agreement reached takes account of all key interests							
b. The extent to which you feel that the agreement reached takes account of your key interests							
c. The extent to which the agreement reached will effectively solve the problem/resolve the conflict							
d. The extent to which you are confident the agreement can be implemented.							

7A. If you rated any of the above statements “not at all,” “to a very little extent,” or “to a little extent,” please explain.

                                                                                                                                                        

                                                                                                                                                        

8. If you had not participated in this collaborative process, what would have been the most likely process(es) (or mechanism(s)) for the issues to be addressed or resolved?

Please choose all that apply

 a. Unassisted negotiation

 b. Judicial settlement conference

 c. Litigation

 d. Lobbying or working to achieve legislative action

 e. Rulemaking

 f. Arbitration

 g. Administrative proceeding (e.g., agency appeals process, contested process hearing, agency order)

 h. Unilateral decision by single party

 i. Wait for a better time to take action

 j. Maintain the status quo with no plans for future action

 k. A combination of the above (please specify)                                     

 l. Don't know (skip to Question 12)

 m. Other (please describe)

9. Please consider how the collaborative process you completed compares with the alternative that you identified in the Question 8, and then check the most appropriate of the following:

please choose only one

 I feel the collaborative process was less expensive….and this level of expenditure was appropriate.

 I feel the collaborative process was less expensive…but additional resources were probably needed.

 I feel the collaborative process cost more…. and the extra costs were worth the investment.

 I feel the collaborative process cost more…and the extra costs were not worth the investment.

 Don’t know

10. Please consider how the collaborative process you completed compares with the alternative that you identified in Question 8, and then check the most appropriate of the following:

please choose only one

 I feel the collaborative process took less time…and this duration was appropriate.

 I feel the collaborative process took less time… but additional time was probably needed.

 I feel the collaborative process took more time… and the extra time was worth the investment.

 I feel the collaborative process took more time…and the extra time was not worth the investment.

 Don’t know

11. Using the scale below, how do you think the collaborative process you completed would compare with the alternative that you selected in Question 8? (Although it may be hard to know what would have happened with the alternative you chose in Question 8, please give us you thoughts on the following)

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know
a. The results of the collaborative process better served the interests of the participants.							
b. The collaborative process made me more aware of other stakeholders’ interests and objectives.							
c. The results of the collaborative process are less likely to be challenged.							
d. The participants are more likely to be able to work together in the future on matters related to this case or project.							
e. The collaborative process we participated in more effectively addressed the issues or resolved the conflict.							
f. The collaborative process we participated in led or will lead to a more informed public action / decision.							

* 11A. If you rated any of the above statements “strongly disagree,” “disagree,” or “somewhat disagree,” please explain.

                                                                                                                                                     

                                                                                                                                                     

12. Using the scale below, please rate the extent to which the following conditions were in place (1) when the process began and (2) as a result of the process.

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know
The participants were able to work together cooperatively before the process began.							
The participants were able to work together cooperatively as a result of the process.							

13. Using the scale below, please rate the extent to which the following conditions were in place (1) when the process began and (2) as a result of the process.

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know
The participants trusted each other before the process began.							
The participants trusted each other as a result of the process.							

14. Using the scale below, please rate your level of agreement with the following:

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know
a. I had the resources (e.g., time, money) needed to participate effectively in the process.							
b. The participants, as a group, felt they were appropriately engaged in designing the process.							
c. I was involved as needed in selecting the facilitator(s).							
d. The participants, as a group, represented all affected concerns.							
e. Participants were involved in appropriate roles in the process							
f. The absence of participants had a negative effect on the collaborative process.							
g. The participants had sufficient authority to make commitments on behalf of their organizations.							
h. Changes in leadership during the process had a negative effect on the collaborative process							
i. The participants continued to be engaged so long as their involvement was necessary.							
j. The overall duration of the process had a negative effect on the collaborative process							
k. The process helped me gain a better understanding of the all of the issues to be addressed.							
l. The process helped me gain a better understanding of the other participants’ views and perspectives.							
m. The process helped me identify and focus on the key issues that had to be addressed.							
n. The process helped the participants, as a group, effectively engage to work on the key issues.							
o. The participants, as a group, sought options or solutions that met the common needs of all participants.							

15. Please identify your level of familiarity and/or involvement in the model development, testing, and/or application:

please choose only one

 I was involved in at least one aspect of the modeling process (e.g., identifying objectives, identifying variables and relationships, testing the model, etc.)

 I was not directly involved but am familiar enough with the modeling process to answer questions about it

 I was not involved and am not familiar enough with the modeling process to answer questions about it (skip to Question 17)

16. Using the scale below, please rate your level of agreement with the following based on your participation in the modeling process:

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Not applicable	Don’t know
a. Participants in the modeling process were involved in defining the overall purpose and use of the model.								
b. Participants in the modeling process were involved in identifying data sources, variables, and assumptions to be used in the model.								
c. Participants in the modeling process were involved in defining and/or structuring the causal relationships among variables in the model.								
d. Participants in the modeling process understood how their questions would be addressed by the model.								
e. Participants in the modeling process were involved in characterizing the status quo (i.e., current approach to the resource management issue) and the assumptions used for the status quo.								
f. Participants in the modeling process were involved in formulating alternatives to the status quo to be tested.								
g. Participants in the modeling process were involved in validating/testing the model.								
h. Participants were encouraged to directly interact with the model via the interface.								
i. Modeling results were available in a short enough timeframe to meet the needs of the collaborative process.								

17. Using the scale below, please rate your agreement with the following. (note that this question should be answered by those who were directly involved in the collaborative modeling as well as those who were less involved in the modeling but, nonetheless, experienced its impacts on the overall process)

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know
a. The modeling process improved the extent to which relevant information was integrated into the process (relative to what would have been the case in the absence of the model).							
b. The model addressed all key interests.							
c. The model addressed your key interests							
d. The model balanced participants’ key interests in an unbiased way.							
e. The model presented a realistic portrayal of the relative impacts of different resource management alternatives.							
f. I trust the technical information used in the model.							
g. The model was accessible by all participants, regardless of their technical background.							
h. Participants had adequate opportunities to evaluate scenarios of interest to them using the model.							

18. Using the scale below, please rate your level of agreement with the following:

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know	Not applicable
a. We worked effectively to identify information needs.								---
b. All participants had full access to relevant information they needed in order to participate effectively in this collaborative process.								---
c. The quality of the information used was good enough for the process.								---
d. Key information was missing which raised important doubt regarding some aspect of the model.								---
e. Relevant information was effectively integrated into the process (e.g., a project web site was used to share information, spatial analysis and decision support tools were used).								---
f. As a group, participants gained a better understanding of the nature and magnitude of impacts of different resource management alternatives on their interests other than their own.								---
g. As a result of my involvement, I was better able to convey relevant information among the stakeholders and/or decision-makers who I represented.								
h. The agreement(s) reached was (were) improved as a result of information integrated into the process.								

19. Using the scale below, please rate your agreement with the following:

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Not applicable	Don’t know
a. The evaluation of alternatives was better informed as a result of the collaborative fact finding process.								
b. The evaluation of alternatives was better informed as a result of the collaborative modeling process.								
c. The trade-offs among stakeholder interests associated with different alternatives were more clearly articulated as a result of the collaborative modeling process.								
d. The comparison of alternatives was more credible as a result of the collaborative modeling process.								
e. The modeling process took into account implementation factors and helped the process focus on implementable options.								
f. The modeling process helped to address the problem/resolve the conflict.								
g. The alternatives evaluated using the collaborative modeling process contributed to the development of the recommended plan.	*	*	*					
h. Actions taken (or planned) to address the resource management issue are consistent with the recommended plan.	**	**	**					

*19A. If you rated Question 19g as “strongly disagree,” “disagree,” or “somewhat disagree,” please identify the reason(s) for the lack of continuity between the modeling process and recommended plan.

please choose all that apply:

 There was little overlap between the individuals participating in the planning process and the individuals participating in the modeling process

 One or more participants in the planning process felt that their interests were not adequately represented in the modeling process

 The model rationale and/or results were not clearly understood by those who developed the recommended plan

 The model did not account for key information, interests, and/or implementation considerations (e.g., legal constraints)

 Model results were not available at the time of the negotiation of the recommendation plan.

 Other (please describe)                            

**19B. If you rated Question 19h as “strongly disagree,” “disagree,” or “somewhat disagree,” please identify the reason(s) for inconsistency between actions taken (or planned) and the recommended plan.

 Alternative recommendations that came from sources outside of the collaborative planning process were used (or will be used) as the basis for action

 One or more participants in the collaborative planning process developed alternative recommendations after the process was complete that were used (or will be used) as the basis for action

 New information and/or key interests were identified but the process was not reconvened

 Legal constraints were identified that required a different planning approach and different actions

 Agreements required by the recommended plan were too complex

 Too much time passed between the conclusion of the planning process and action

 Other (please describe:                                              )

20. How many “practitioners,” including facilitators and modelers, supported the process? For the purpose of this questionnaire, these roles are defined as follows:

• Facilitator – a neutral party who seeks to assist a group of individuals or other parties to constructively discuss a number of complex, potentially controversial issues

• Modeler – a neutral or trusted party who is responsible for developing the model through technical coding.

Please focus on the principal practitioners in the process - e.g., lead facilitator; if more than three principal practitioners supported the process, choose the three with whom you had the most experience, and answer Questions 21 and 22 for those three practitioners.

 One

 Two

 Three

 More than three

21. Please identify the facilitators and/or modelers involved in the process by entering their initials in the space provided and identifying the choice(s) that best describe their role and/or roles in the process. (note that for some processes, one person may serve more than one role – if this is the case, select all of the roles that the person served)

Facilitator’s / Modeler’s Initials	Roles Served (Check all that apply)
	Facilitator	Modeler	Not Sure
			
			
			

22. Using the scale below, please rate the following for each of the facilitators / modelers identified in Question 21.

Note that some of the following statements refer to a specific role – facilitator or modeler. If a statement does not mention one of the roles served by the practitioner that you are rating, select “not applicable.” For example, if you were rating a practitioner who served as a facilitator only, you would provide a rating for statement “b” and answer “not applicable” in response to statement “c.” If the practitioner served both the facilitator and modeler roles, you would provide a rating for both statements “b” and “c.”

22-1. Initials of first facilitator / modeler identified above ___

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Not Applicable
a. On reflection, this was the right facilitator / modeler to guide the planning and/or modeling process.							
b. The facilitator kept us on track and proceeding in a timely manner.							
c. The modeler was able to explain to participants with varying levels of experience how the modeling process would work							
d. The facilitator helped us manage technical discussions efficiently.							
e. The facilitator dealt with all the participants in a fair and unbiased manner.							
f. When things got tense, the facilitator was able to help us find ways to move forward constructively.							
g. When necessary, the facilitator / modeler brought in neutral topical experts to help address technical questions and/or resolve differences in interpretation of factual information.							
h. The facilitator made sure that the views and perspectives of all participants were considered in the process.							
i. The modeler made sure that all participants had adequate opportunity to participate in decisions about model structure and data inputs							
j. The facilitator made sure that no one dominated the process or other participants.							
k. The modeler was able to explain in an intuitive way how the model input was reflected the model output (e.g., how different input assumptions affected the output)							
l. The modeler was able to work effectively with the group to create an interface and/or to display results in a manner that was useful to participants.							
m. The facilitator helped the participants test the practicality of the options under discussion.							
n. The facilitator was helpful in documenting our agreement.							
o. The group could not have progressed as far without the help of the facilitator / modeler.							

If you answered “one” for Question 20, skip to Question 23

22-2. Initials of second facilitator / modeler identified above ___

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Not Applicable
a. On reflection, this was the right facilitator / modeler to guide the planning and/or modeling process.							
b. The facilitator kept us on track and proceeding in a timely manner.							
c. The modeler was able to explain to participants with varying levels of experience how the modeling process would work							
d. The facilitator helped us manage technical discussions efficiently.							
e. The facilitator dealt with all the participants in a fair and unbiased manner.							
f. When things got tense, the facilitator was able to help us find ways to move forward constructively.							
g. When necessary, the facilitator / modeler brought in neutral topical experts to help address technical questions and/or to resolve differences in interpretation of factual information.							
h. The facilitator made sure that the views and perspectives of all participants were considered in the process.							
i. The modeler made sure that all participants had adequate opportunity to participate in decisions about model structure and data inputs							
j. The facilitator made sure that no one dominated the process or other participants.							
k. The modeler was able to explain in an intuitive way how the model input was reflected the model output (e.g., how different input assumptions affected the output)							
l. The modeler was able to work effectively with the group to create an interface and/or display results in a manner that was useful to participants.							
m. The facilitator helped the participants test the practicality of the options under discussion.							
n. The facilitator was helpful in documenting our agreement.							
o. The group could not have progressed as far without the help of the facilitator / modeler.							

If you answered “two” for Question 20, skip to Question 23

22-3. Initials of third facilitator / modeler identified above ___

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Not Applicable
a. On reflection, this was the right facilitator / modeler to guide the planning and/or modeling process.							
b. The facilitator kept us on track and proceeding in a timely manner.							
c. The modeler was able to explain to participants with varying levels of experience how the modeling process would work							
d. The facilitator helped us manage technical discussions efficiently.							
e. The facilitator dealt with all the participants in a fair and unbiased manner.							
f. When things got tense, the facilitator was able to help us find ways to move forward constructively.							
g. When necessary, the facilitator / modeler brought in neutral topical experts to help address technical questions and/or resolve differences in interpretation of factual information.							
h. The facilitator made sure that the views and perspectives of all participants were considered in the process.							
i. The modeler made sure that all participants had adequate opportunity to participate in decisions about model structure and data inputs							
j. The facilitator made sure that no one dominated the process or other participants.							
k. The modeler was able to explain in an intuitive way how the model input was reflected the model output (e.g., how different input assumptions affected the output)							
l. The modeler was able to work effectively with the group to create an interface and/or to display results in a manner that was useful to participants.							
m. The facilitator helped the participants test the practicality of the options under discussion.							
n. The facilitator was helpful in documenting our agreement.							
o. The group could not have progressed as far without the help of the facilitator / modeler.							

23. Using the scale below, please rate the following statements for processes that involved a modeler/modeling team who worked with a separate facilitator:

 Not Applicable, roles were not separate (skip to Question 24)

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree
a. There was a clear distribution of roles and responsibilities between the modeler and facilitator						
b. The modeler and facilitator understood and respected each others’ roles.						
c. The modeler and facilitator supported each other and worked effectively together.						

24. Think back to the start of the process and please rate the following using the scale below:

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Not Applicable
a. At the start of the process, I was willing to work cooperatively with other participants in this process.							

25. Using the scale below, please rate your agreement with the following statements:

	Strongly disagree	Disagree	Somewhat disagree	Somewhat agree	Agree	Strongly agree	Don’t know
a. I would recommend this type of process to my colleagues in a similar situation without hesitation.							
b. We could not have progressed as far using any other process of which I am aware.							
c. The process would not have achieved as much without the use of collaborative modeling.							

26. At this point in time, in very general terms what did this collaborative process accomplish?

Check all that apply

 a. A potentially costly or divisive dispute was likely avoided.

 b. An impasse (stalemate) was broken.

 c. A crisis was averted.

 d. Conflict didn’t escalate.

 e. Costly or protracted litigation was avoided.

 f. Participants gained a greater understanding of the complexities and uncertainties associated with the issue

 g. Participants gained a greater appreciation for all of the interests involved

 h. Relationships among parties in this process were improved.

 i. The process resulted in timely decisions and outcomes

 j. Participants agreed to work together to address future problems/resolve future conflicts in a collaborative manner

 h. Nothing was accomplished.

 i. The process made the issues or dispute worse.

Use this space to comment:

27. What is your top suggestion on how this collaborative process could have been improved?

Please write “none” if you feel this process could not have been improved

28. From your perspective, what will be the long-term effect(s) (e.g., positive or negative impacts) of the collaborative process?

please identify effect(s) in each of the categories listed below that were addressed by the collaborative process and/or add your own categories as appropriate. If a category was addressed but no effects are expected, please write “none.” If a category was not addressed, please write “NA.”

Effect(s) of the process

Natural resources and

environmental conditions

Historic and cultural

resources

Community and social

Conditions

Economic conditions

Recreational Uses

Other, please specify

Thank you for taking the time to complete this questionnaire.

Your assistance in providing this information is very much appreciated.

17