Form 1 DD Council Executive Director Interview

DEVELOPMENTAL DISABILITIES PROGRAM INDEPENDENT EVALUATION

STATE COUNCILS ON DEVELOPMENTAL DISABILITIES

EXECUTIVE DIRECTOR AND COUNCIL STAFF INTERVIEW

[Interviewer]: Thank you for taking the time to meet with us today. My name is [interviewer’s name] and this is [second interviewer’s name]. We both work for _______, a private research company in _______.

As we’ve already discussed, the Administration on Developmental Disabilities (ADD) is conducting an independent evaluation of Developmental Disabilities Network programs and collaboration among them. The purpose of this evaluation is to examine the impact of the Developmental Disabilities (DD) Network programs on the lives of people with developmental disabilities and their families. This evaluation is intended to provide feedback to the ADD to help improve the effectiveness of its programs and policies, to serve as a decision-making tool for ADD, and to promote accountability to the public.

Your program is one of ____ State Councils on Developmental Disabilities (DD Councils) in ____ states that has been randomly selected to participate in the evaluation. The evaluation will consist of interviews with key stakeholders (e.g., Executive Directors and staff, DD Council members, policy makers and DD Council collaborators, recipients of DD Council self-advocacy and leadership development training, and recipients of DD Council community capacity development training). We will also be conducting the evaluation at UCEDDs and P&As. We will summarize the information provided by interviewees (yourself included) in all programs to help us to examine the impact of DD Councils at the national level.

As the DD Council executive director, you are an important source of information regarding the operations and achievements of this program. We have some specific questions to ask you about the different key functions of your Council. The interview should last approximately 3-4 hours.

Before we start, we would like to ask your permission to record this interview so that we do not miss any of your responses to our questions. The recording will be used by ________ only; it will not be provided to ADD or anyone else.

Do you have any questions before we start?

PART A. STATE PLAN DEVELOPMENT

As you know, DD Councils are required by law to develop and have an approved 5-year State Plan.

Process for developing goals and priorities

Let’s start with the process your DD Council uses for developing the State Plan, thinking about what happened during the last planning cycle.

1. What methods did you use to obtain input on the needs and priorities of people with developmental disabilities and their families? Please describe.

Probe:

• Public meetings

• Focus groups

• Surveys

• Analysis of telephone calls to the DD Council

• Input from staff and contractors

• Other _________________________________________________________

2. Whom did you contact to participate in providing input?

Groups/individuals	Contacted ()	Provided input ()
People with developmental disabilities
Family members
Advocates
Service providers
Policy makers
P&A
P&A Board of Directors/Advisory Council
UCEDD(s)
UCEDD Consumer Advisory Committee (CAC)
Other, please specify

3. What groups/organizations/individuals provided your DD Council with input for the last planning cycle?

4. What did you do to accommodate people with developmental disabilities so they were able to participate in the planning and priority setting process? Please describe.

Probe:

• Methods of contact

• Materials used

• Accessibility at meetings

• Assistance with materials

• Other, please specify ____________________________________________

5. How did you make sure you provided opportunities to participate to those who had never participated before in the process? For example, please describe how you sought input from:

Probe:

• People from both urban and rural communities;

• People throughout all parts of the state;

• People from ethnic/minority groups in your state;

• People who have difficulty understanding English;

• Others, please specify____________________________________________

6. What did you do to provide accommodation to those who needed it in order to participate? Please give examples.

7. What type of existing data or reports did you use to identify the needs and interests of people with developmental disabilities? Please give examples.

Probe:

• Census data

• Findings from UCEDD reports [Obtain copy or references.]

• Findings from published literature [Obtain references.]

• Findings from P&A reports [Obtain copy.]

• Findings from DD Council-supported demonstration projects [Obtain copy.]

• State agency reports [Obtain copy.]

• Other _________________________________________________________

[Obtain copy or references.]

8. In the last planning cycle, how did you use feedback about current activities to inform the planning and priority setting process? Please describe.

1. What kind of information did you collect about current activities (e.g., information on recipients’ satisfaction with DD Council-supported programs)?

2. How did you collect it?

3. From whom (e.g., staff and contractor feedback)?

4. How did you use it? Give examples.

Results of planning process

9. What is the process you use to move from stakeholder input to statements of goals and priorities in your SGP? Please describe.

1. What do you do?

2. Who is involved?

Probe:

• Staff

• DD Council members

• State officials

• People with developmental disabilities

• Family members

• Representatives from other DD Network programs

• Other _____________________________________________________

3. How do you make sure the end result is consistent with both stakeholder input and existing data regarding the needs and priorities of people with developmental disabilities? Please explain.

d. How do you make sure the end result is truly reflective of the findings from the planning process? Please describe.

I see the goals and objectives in the State Plan are _______________________ (Refer to State Plan).

10. How do these relate to the feedback you received from stakeholders? Please give examples.

Use of State Plan to shape DD Council activities

We’ve been told that the State Plan and amendments are used to shape the activities of the DD Council.

11. How does your DD Council use the State Plan? Please describe.

12. How did your DD Council deal with new and emerging issues that arose in the past year? Please give examples.

13. What was the rationale for taking on these new and emerging issues not covered in the State Plan? Please explain.

PART B. SELF-ADVOCACY AND LEADERSHIP DEVELOPMENT

According to the DD Act [Sec. 125(c)(5)(C)], DD Councils “may support and conduct training for persons who are individuals with developmental disabilities, their families, and personnel (including professionals, paraprofessionals, students, volunteers, and other community members) to enable such persons to obtain access to, or to provide, community services, individualized supports, and other forms of assistance, including special adaptation of generic community services or specialized services for individuals with developmental disabilities and their families.”

We’d like to talk a little bit about self-advocacy and leadership development activities supported by your Council. The kinds of activities we mean are those that are targeted toward people with developmental disabilities (and sometimes their families) and that provide them with knowledge and skills to obtain supports and services for themselves and advocate for systems change so that their choices can meet their desired lifestyle goals and will be included in the community.

Self-advocacy and leadership training programs and supports

14. What self-advocacy and leadership activities did the DD Council support or conduct last year? Please describe.

Probe:

• Types

• Content

• Skills taught

• Target audience

14. In the past year, did you identify the unserved and underserved populations and communities in your state? Please describe.

Probe:

• Process for identification

• Groups/populations identified

14. How did you or your grantees go about recruiting these populations and communities for participation in self-advocacy and leadership activities? Please describe.

14. Did you keep track of the number that participated in DD Council-supported education, training, and technical assistance in the past year?

Council supports to State self-advocacy organizations led by people with developmental disabilities

Apart from helping people with developmental disabilities to develop their advocacy and leadership skills, DD Councils, according to the DD Act, are also called upon to “establish or strengthen a program for the direct funding of a State self-advocacy organization led by people with developmental disabilities” [Sec. 124(c)(4)(A)(ii)(I)].

14. Would you please talk about what your Council does to comply with this section of the Act?

1. To which organization(s) did you provide support in the past year? Please describe organization(s).

Probe:

• Name of organization(s)

• Mission

• Participants

2. What type of support did you provide?

Probe:

• Provided funding

• Provided training or technical assistance on grant writing

• Identified public or private funding opportunities

• Provided opportunities for members to participate in Council activities

• Identified opportunities for members to participate in other advocacy activities of other organizations

• Other _________________________________________________________

3. How did these organizations use the support you provided? What were the results in the past year? Please give examples.

Probe:

• Submission of grant applications

• Receipt of grant and contract awards

• Increase in te number of organizations throughout the state

• Increase in size of the organization

• Achievement of financial independence

• Other

Results of self-advocacy and leadership training programs

We would like to get an idea of the impact of your Council-supported advocacy and leadership training programs on people with developmental disabilities and their families.

19. How does the DD Council (or contractors) evaluate the effectiveness of advocacy and leadership activities? Please describe.

Probe:

• Satisfaction surveys

• Followup surveys

• Other ___________________________________________________________

20. How do participants in Council-supported advocacy and leadership activities use the information and skills they obtained?

1. Among your current Council members, how many are former recipients of Council-supported self-advocacy and leadership training?

2. Among those who participated in DD Council-supported advocacy and leadership training within the past 5 years, did any become members or leaders of community organizations that advocate on behalf of people with disabilities? Please give examples.

3. Among those who participated in DD Council-supported education and training within the past year 5 years, did any participate in Council-supported advocacy activities? Please give examples.

Probe:

• Approximate number

• Types of activities

• Attendance at meetings with policymakers

• Letter writing or sending emails to policymakers

• Provision of personal testimony to legislators

• Speaking at Council-sponsored conferences

• Other (please specify)

4. Among those who participated in Council-supported self-advocacy and leadership training in the past 5 years, did any take part in other advocacy activities organized by the DD Council or another organization? Please give examples.

Probe:

• Approximate number

• Types of activities

• Attendance at meetings with policy makers

• Letter writing or sending emails to policy makers

• Provision of personal testimony to legislators

• Speaking at disability-related conferences

• Other (please specify) ________________________________

5. How do participants in Council-supported advocacy and leadership training use the information and skills they obtained in their personal lives? Please give examples.

Probe:

• Became an employee or volunteer of a disability advocacy organization

• Used the knowledge and skills they obtained to enhance their lives and become self-sufficient

• Obtained supports and services for oneself

• Obtained supports and services for others

• Obtained or retained a job

• Mentored others

• Other

PART C. COMMUNITY CAPACITY DEVELOPMENT

Now we would like to talk about Council-supported activities that educate the public in order to build community capacity to better serve people with developmental disabilities, provide them with choice and help them to be independent and included in the community.

We’ll start with the ways you support educational activities, the target audiences of these activities, and how you make sure that these activities are accessible, culturally competent, and appropriate to the target audiences for which they are intended.

Educational Activities (Training, Demonstration Projects, Technical Assistance)

21. Within the past year, what educational activities (including training, demonstration projects, and technical assistance) did you conduct or support?

Probe:

• Purpose

• Types of activities

• Format

• Content

21. Who implemented these activities? Please describe.

Probe:

• DD Council

• Grantees

• Other _____________________________________________________________

21. What audiences were targeted? Please give examples.

Probe:

• The media

• Policy makers

• Service providers

• Disability-related organizations

• Non-disability related organizations that interact with the public

• Employers

• Health care providers

• Teachers

• Others _______________________________________________________

24. Did you (or your grantees) keep track of the number that participated in DD Council-supported educational activities (including training, demonstration projects, and technical assistance)?

Results of educational activities

Now we’d like to ask you some questions on the results of the educational activities you supported and how you evaluated them.

25. How did the DD Council (your contractor) evaluate the effectiveness of its educational activities?

1. What was done? Please describe.

Probe:

• Satisfaction surveys

• Followup surveys

• Other __________________________________________________________

2. How did participants in DD Council-supported capacity-building efforts use what they learned? Please give examples.

Probe:

• Transferred the knowledge and skills they obtained to their work

• Improved their effectiveness in providing services to people with developmental disabilities

• Improved the effectiveness of the community in including people with developmental disabilities in all aspects of life

• Provided accommodations for employees with developmental disabilities

• Made use of tax incentives for assistive technology for employees

• Made public buildings more accessible

• Other _________________________________________________________

PART D. IDENTIFICATION OF PROMISING AND EFFECTIVE PRACTICES

We would like to learn about your Council’s activities with regard to the identification of promising and effective practices for providing services to people with developmental disabilities and their families and including people with developmental disabilities in the community. We define promising practices as new approaches to serving people with developmental disabilities and their families and including people with developmental disabilities in the community. Promising practices are practices that have not yet been tested (evaluated) for effectiveness.

Identifying promising practices

25. Did your DD Council identify or test¹ any promising practices in the past 5 years?

1. What promising practices did your Council identify or test within the past 5 years? Please give examples.

Probe:

• Description of practice

• Intended target audience

2. Why are these considered promising practices?

3. What did you find/conclude?

[See table.]

25. How did your DD Council go about identifying and/or testing promising practices in the past 5 years? Please describe.

Probe:

• Issued requests for proposals to identify promising practices

• Issued requests for proposals to test promising practices using DD Council funds

• Secured external funding to identify or test promising practices

• Assisted community organizations in obtaining funding to identify or test promising practices

• Partnered with other agencies or organizations to identify or test promising practices

• Other _________________________________________________________

Results of identifying and testing promising practices

Now let’s talk about how your Council uses the results from these efforts. .

25. For each of the promising practices identified and/or tested in the past 5 years, how were findings disseminated? Please describe.

Probe:

• Placed on DD Council web site

• Mailouts (or email blasts) to specific target audiences

• Presentations of findings at conferences and meetings

• Descriptions in newsletters

• Other _______________________________________________________________

25. For each of these, to which organizations/individuals were these findings disseminated? Please be specific.

Probe:

• Council members

• DD Network sister agencies

• Disability-related organizations in the community

• Other _________________________________________________________________

25. How were the results used? Please give examples.

Probe:

• For future Council planning

• In development of new requests for proposals

• Used by service providers and others in the community

• Used to increase inclusion and choice in the community

• Other ____________________________________________________

Identifying and Testing Promising Practices

Description of Practice (Q26a)	Target Audience (Q26b)	Results/Conclusions (Q26c)	Method of Identifying/Testing (Q27)	Followup of Results/ Conclusions (Dissemination and Use) Q28-31)

31. Are any of your DD Council’s current activities the result of findings/conclusions that were found from the process of identifying or testing promising practices in the past? Please give examples.

PART E. SYSTEMS CHANGE THROUGH ADVOCACY

According to the DD Act, DD Councils are called upon to support and conduct activities related to barrier elimination, systems design and systems redesign [Sec. 125(c)(5)(H)] – that is, system change activities. The next section of this interview has to do with systems change through advocacy.² Questions in this section will address the types of systems design projects you implemented in the past few years, collaboration with other organizations and individuals as part of the advocacy process, participation of people with developmental disabilities and their families in these activities, and the specific outcomes of DD Council advocacy efforts.

32. First, let’s start with a systems change project in which you used advocacy strategies in the past 3 years. Please describe.

[See table.]

Probe:

• Issue

• Strategies used

• Wrote position papers

• Obtained press coverage

• Informed policy makers

• Gave public testimony

• Drafted legislation

• Gave funding to support systems change (or maintenance) efforts

• Other _______________________________________________________

• DD Council role (e.g., lead, supportive)

Examples of DD Council system design and improvement activities

Systems Change/Project Issue (Q32)	Strategies Used (Q32)	Collaborators (Q33)	Contributions of Collaborators (Q34)	DD Council Role (Q35)

[INTERVIEWER: REPEAT QUESTION 32-35 FOR OTHER ISSUES.]

Collaboration to Support Advocacy Efforts

In order to achieve advocacy goals, it is often beneficial to collaborate with other groups who have similar goals. Thinking about the issues you’ve described, let’s talk about the groups and individuals with whom your DD Council collaborated to achieve your goals and the nature and extent of the collaboration.

32. What groups/individuals were involved?

• People with developmental disabilities

• Family members

• Legislators

• Other policy makers

• Advocacy groups

• Coalition (disability network) members

• Others

32. What was each group’s/individual’s role? Please describe.

32. What role did the DD Council play? Please describe.

Outcomes of systems change (advocacy) activities

We realize that a large part of systems design and improvement is educating policymakers, so we’d like to talk about your efforts to educate policymakers.

We realize that it may take a long time to see the impact of DD Council-supported systems design and improvement efforts on the lives of people with developmental disabilities and their families. However, we’d like to know if any changes were made (or not made) as a result of your Council’s system design and improvement activities.

36. Can you think of any examples in which system changes were made (or not made) as a result of Council-supported advocacy efforts in the past year 5 years? Please describe.

1. How were legislation and policy changed (or maintained) to address gaps in and barriers to supports and services for people with developmental disabilities and their families as a result of Council activities that took place in the past 5 years? Please give examples.

2. How did DD Council efforts in the past 5 years support changes to community practice? Please give examples.

3. How did DD Council efforts in the past 5 years lead to inclusion in the community, independence and increased choice? Please give examples.

4. How do you know these changes were a result of Council efforts? Would they not have occurred but for Council efforts? Please explain.

PART F. GOVERNANCE AND MANAGEMENT

Capacity of the DD Council to fulfill its roles and responsibilities

Let’s start by looking at the composition of the Council.

[Use list of Council members from PPR or web site.]

37. As a whole, what areas of expertise does the Council represent? Please describe.

Probe:

• Knowledge of the needs of people with developmental disabilities and family members

• Legal expertise

• Financial and business expertise

• Leadership qualities

• Policy expertise

• Knowledge of community supports and services available for people with developmental disabilities and family members

• Knowledge and understanding of the state population

• Other _________________________________________________________

38. How do Council members become familiar with the principles and goals in the DD Act and the DD Council? Please describe.

Probe:

1. Orientation of new members

2. Attendance at national meetings

3. Participation in or chair of a DD Council sub-committee

4. Attendance at a community meeting on behalf of the DD Council

5. Participation in or observation of Council activities

6. Other ______________________________________________________________

39. How does the DD Council assess its own efficiency and effectiveness in meetings its goals and objectives? In the past 5 years, did the Council:

1. Review itself;

2. Review the Designated State Agency;

3. Review the Executive Director; and

4. Other? __________________________________________________________

Please describe.

Supports provided to DD Council members

40. What kinds of supports do DD Council members receive so that they can meaningfully participate in meetings and fulfill their responsibilities?

Probe:

• Reimbursements for travel expenses

• Stipend

• Receipt of relevant meeting materials in advance

• Assistance from a support person to aid in preparation and understanding of materials before and during meetings

• Other ______________________________________________________

Procedures for selecting grantees

41. Does your Council have a policy or practice to identify and select competent and experienced grantees? Please describe.

Probe:

• Requirement for review committee

• Stated review criteria

• Methods for handling unsolicited proposals³

• Other _________________________________________________________

[Obtain a copy of policy or practice.]

42. Is there a requirement for a review committee or sub-committee? Please describe.

Probe:

• Method of selection

• Composition

• Council membership

• Knowledge of issues affecting people with developmental disabilities and their families

• At least one person with a developmental disability

• Evaluation/research expertise

• Understanding of subject matter being discussed

• Other____________________________________________________

43. How does your Council handle unsolicited proposals? Please describe.

a. In the past year, were any unsolicited proposals funded?

b. What process was used to review and make decisions on unsolicited proposals?

PART G. COLLABORATION

According to the DD Act, DD Network programs are required to report the extent to which they collaborate with one another. We’ve talked a little bit about collaboration as part of your systems change/advocacy activities. Now we would like to focus on your collaborative efforts with the DD Network programs on all types of activities besides systemic advocacy.

For this evaluation, we define collaboration as a well-defined relationship entered into by two or more individuals, groups, and/or organizations to achieve common goals of the Developmental Disabilities Assistance and Bill of Rights Act of 2000 (the DD Act).

Let’s start first with how the DD Network programs identify issues and common goals on which to collaborate.

43. What process do the programs use for identifying, planning, and implementing collaborative efforts? Please describe.

Probe:

• Nature of process

• Participants

• Meeting frequency

• Formal vs. informal

• Other_______________________________________________________________

45. What are the issues and goals on which the DD Network programs are currently collaborating? Please describe.

[See following table.]

[Obtain written goals/objectives.]

Collaborations

No.	Project/Issue (Q45a)	Goals/Objectives (Q45b )	Collaborators (including P&A) (Q46)	Collaborators’ role (including role of DD Council) (Q47)	Leadership (Single/joint) (Q48)	Resources Contributed (Q49)
a.
b.
c.

[INTERVIEWER: REPEAT QUESTIONS 81 THROUGH 86 FOR TWO MORE EXAMPLES.]

1. What project or issues are you collaborating on with DD Network programs? Would you please explain each issue?

2. What are the goals you and your collaborators were trying to achieve?

46. Which organizations or groups (including other DD Network programs) are you collaborating with on these issues?

47. What is the role of each collaborator, including the P&A and other DD Network programs?

48. How would you characterize the type of leadership for each project?

Probe:

• Single

• Joint

• Other ____________________________________________________________

Resources for Collaboration

48. Collaboration suggests shared resources being available and used. What resources did each group or organization contribute? Please describe.

Probe:

• Funding

• Staff time

• Meeting place

• In kind contributions

• Other _________________________________________________________

Process of collaboration – regular communication

Let’s talk about the process of communication by using the same examples.

48. What were some of the ways you achieved regular communication during your collaborative efforts? Please describe.

1. Sharing information and progress on ongoing efforts

Probe:

• Examples

• Frequency

• Recipients

• Other

2. Regular updates to participants in the collaborative effort

Probe:

• Examples

• Frequency

• Recipients

• Other

3. Reports on milestones achieved

Probe:

• Examples

• Frequency

• Recipients

• Other

4. Discussion of issues that arose

Probe:

• Examples

• Frequency

• Participants

• Other

• Debriefing at the end of a collaborative effort (if appropriate)

Probe:

• Examples

• Frequency

• Recipients

• Other

48. In the past year, were there any issues that arose in the community on which the DD Network programs responded together? Please give examples.

48. What was the joint response of the DD Network programs?

48. What is the process the DD Network programs use to monitor progress and barriers of collaborative efforts? Please describe.

1. Please give examples of ways in which the programs monitor and identify barriers.

2. For collaborative efforts implemented in the past 2 years, what process was used to identify leadership? What decisions were made on leadership of each effort?

3. What barriers have been identified?

4. Have goals and objectives been changed as a result of monitoring and identifying barriers? Please give examples.

Results of collaboration

Finally, let’s talk about achieving your goals and objectives -- the results of your collaborative efforts.

48. Were there any products derived from your collaborative efforts? Please give examples and describe?

Probe:

• White papers

• Wording for legislation

• Brochure on joint program

• Program enrollment form

• Other _________________________________________________________

[Obtain examples of products developed in past year, if applicable.]

48. What has been the result of your collaboration with DD Network programs and other organizations? Please give examples from the past 5 years.

Probe:

• Changes to (or maintenance of) State and local systems (e.g., changes or maintenance in practices and legislation)

• System changes (or maintenance) that affected people with developmental disabilities and their families, including people who are traditionally unserved and underserved (e.g., racial or ethnic minorities, people with little or no use of English, or people who live in remote or rural areas or live in poverty)

• Increases in the capacity of the community to provide supports and services to people with developmental disabilities and their families (e.g., accommodations are provided in public services such as transportation, education, and employment)

• Ways participants in DDN collaborative efforts have improved their knowledge or skills in ways that benefit people with developmental disabilities and their families

• Other _________________________________________________________

PART H: DEBRIEFING [AT END OF SITE VISIT]

48. Before we leave, we would like to give you the opportunity to ask any questions you might have, raise any issues you might want to raise, or talk about anything related to your program that you would like to expand upon.

1. First, do you have any general comments to make?

2. Issues?

3. Further explanation of your DD Council?

48. There are also a few things I want to make sure we don’t forget.

Obtain materials requested, if easily available.

Confirm products to be sent. Leave FedEx form.

Obtain contact information for additional individuals who need to be contacted.

Confirm process for telephone interviews and availability of interviewees.

Leave business card.

We’d like to thank you again for taking the time to meet with us these past 2 days. We also appreciate you making your staff available and arranging for us to meet with many of your stakeholders.

1“Testing” refers to evaluation with both rigorous methodology and less rigorous methodology (e.g., observation of trying out a promising practice).

2Advocacy is an umbrella term for organized activism related to a particular set of issues. It is the act or process of defending or maintaining a cause or proposal. In the context of the key function of advocacy and leadership development, advocacy includes advocacy for self and others.

3 An unsolicited proposal is one that is independently developed and submitted rather than in response to a solicitation or request for proposal (RFP) of State Councils on Developmental Disabilities.

S1D-13