Principal Investigator Interview Protocol

CPATH Principal Investigator Interview Protocol March 2010

Principal Investigator Interview Protocol

CPATH Project Title:

Grantee Institution(s):

Interviewee Name/Title/Contact Information:

Interview Date:

Interviewer(s):

Introductory Remarks:

My name is ______ and I work for SRI International, a non-profit research firm that is responsible for the evaluation of the NSF CPATH Program on behalf of the National Science Foundation. The purpose of this interview is to learn more about your CPATH project, how it is being implemented, the organizations and stakeholders involved, and the effect that it has been having on various groups and organizations. Beneficiaries of this project may include students, faculty members, administrators, institutions of higher education, K-12 school districts, professional associations, government and non-profit organizations, and businesses and industry. Throughout our discussion, please feel free to interrupt me to ask any questions you may have or include information that you believe we should know.

Before we start, I need to have you review and sign this consent form. SRI International’s Institutional Review Board (IRB) as well as the U.S. government’s Office of Management and Budget has approved this project. Your signature on this form indicates that your participation in this interview is voluntary and that you understand that we will never directly identify or quote you based on what you tell us today.

After they review consent form: Do you have any questions before we get started?

Introduction and Background

1. Please tell me about yourself. Probe for:

1. What roles do you have at this institution?

2. How long have you taught/worked at this institution?

3. What other positions have you previously held?

4. Describe your role and responsibilities related to the CPATH project.

2. Prior to the CPATH project, were you engaged in undergraduate reform activities at your institution? If so, please describe those activities.

1. What do you view as pressing issues in improving undergraduate education? (Probe using the following: enrollment and graduation trends for STEM and other disciplines, new demands on education to align more closely to industry or workforce needs, concepts related to computational thinking, as you understand it, in the different disciplines)

3. How did the CPATH project come about? Probe for the following:

1. What was the motivation for initiating the project? How does the project align with other initiatives at your institution? Who was involved in shaping the project?

2. How did you come to be the PI?

Project Strategies

4. Please describe the nature of the teaching/learning environment for computing you are trying to create through the project. What is it designed to do at your institution (and other partner sites, if applicable)? Probe using the following:

1. How might this environment look different from traditional settings for computing?

2. How will you know you’ve been successful in creating this environment?

3. What are the core strategies you’re using to create this environment?

5. What are the core computing concepts that the CPATH project focuses on? Probe using the following:

1. How has participating in the CPATH program helped you identify and refine these concepts?

2. Define the concept of computational thinking.

6. How has your project integrated these core concepts into courses outside of traditional computing disciplines (i.e. biology, chemistry, mathematics, etc.)? Probe using the following:

1. What disciplines are involved?

2. If applicable, how has this integration played out at other participating sites/institutions?

3. What have been the challenges the project has faced in efforts to integrate these core concepts outside of traditional computing disciplines?

7. Who do you think are the primary beneficiaries of your project? Probe using the following:

1. Which groups of students are you targeting for participation?

2. Are you targeting any underrepresented groups for participation? If yes, please explain. (Probe for traditionally underrepresented groups including minorities, females, disabled and non-traditional computer majors.)

Implementation Factors

If project is at multiple sites, probe for information on each participating institution using the probes provided with questions in this section.

8. How has project implementation gone thus far? Probe using the following, asking the respondent to provide reasons for their statements:

1. What have been the highlights or successes? How do you know?

2. What failures has the project experienced? Please describe.

3. What have been the challenges to the implementation of the project? Probe about the relevant actors – students, faculty, administrators, partners, etc.

4. What lessons have been learned from addressing these challenges?

9. What support do faculty at your institution have to help with curriculum development or devising pedagogical strategies for teaching computational thinking (as you understand it)? Probe using the following:

1. Describe the current incentive and awards structure at participating institutions.

2. What incentives are you offering through your CPATH project to faculty to develop curriculum, to devise pedagogical strategies and/or to participate in your project?

3. Are there any additional incentives to encourage faculty to develop innovative ways to teach computational thinking (as your understand it)?

10. What are some of the factors that have supported project implementation? Probe using the following:

1. Culture of committed faculty involvement and participation

2. Strong institutional support from department head/deans/administrators

3. Innovative curricular and pedagogical strategies

4. Student demand for course changes

11. How has the institution leveraged funding from other sources to provide additional support to the project? Probe using the following:

1. What are the specific funding sources, if any (e.g., state, federal, private)?

2. What other ongoing initiatives are focused on computing reform?

3. What other projects, if any, do you envision emerging from this project? Please explain.

Outputs

11. Overall, what influence has the CPATH project had on students and faculty? How do you know? Please describe any influences as well as supporting evidence. Probe using the following asking the respondent to describe the influences as well as to provide supporting evidence:

1. To what extent has the CPATH project influenced student engagement in computing and how? Please describe.

2. How has the project influenced student enrollment in computing courses? Have there been any changes in the demographics or population of students enrolled in computing courses? How do you know?

3. How has the CPATH project prepared students for STEM and other careers? How do you know?

4. How has the project influenced faculty members? Has it led to changes in faculty culture? How do you know?

5. Have faculty members published any articles related to computational thinking (as you understand it) in peer-reviewed journals? (Note: this may or may not be related to the CPATH project)

11. Are there any institutional changes that you would attribute to the CPATH project? How do you know this? Please describe any institutional changes as well as supporting evidence. Probe using the following, asking the respondent to describe changes as well as supporting evidence:

1. Has the CPATH project integrated computational thinking (as you understand it) into other disciplines?

2. Has the CPATH project influenced the rewards/incentive structure of your institution? If yes, please explain.

3. Are the changes formal or informal?

4. In your view, who or what is driving these changes?

11. Do you think the CPATH project has created a model that could be used at other institutions? If yes, please describe the model and how you are supporting its implementation at other institutions. Probe using the following:

1. Are other institutions implementing your models?

2. Are you providing materials or guidance?

3. What mechanisms are in place to sustain the relationship between you and the institutions that have adopted your model?

Community Building and Partnership Development

11. Who are the stakeholders for your project? How is information shared with this community of stakeholders? Note: Stakeholders may include other CPATH grantees, faculty within computing-related disciplines (e.g., computer science, informatics), faculty in other disciplines, university administrators, other higher education institutions, K-12 teachers, professional associations, industry and businesses, and government and non-profit organizations.

Probe using the following:

1. How inclusive is this group of stakeholders?

2. Would you say there is a shared understanding about computing competencies among stakeholders in this community?

3. How do stakeholders communicate with and learn from each other? (How is the project supporting learning and sharing of best practices around computational thinking (as you understand it) and education amongst these stakeholders?) Probe for online sharing and communication, conferences and colloquia, informal networks and other venues for sharing best practices.

11. Are there other organizations with which your project has a significant ongoing relationship as a part of the CPATH project? Probe using the following:

1. Post secondary institutions

2. K-12 school districts

3. Government offices

4. Private companies/non-profit organizations

5. Professional membership organizations

6. Any other groups that have had or will have a significant connection to the project. Probe if there is a key person in this organization that we should talk to.

(IF YES to Q16, then ask questions 17-20; IF NO, then go to Q21)

11. To what extent has NSF funding (for the CPATH project) opened new opportunities for partnerships between multiple sectors (e.g. industry, K-12, professional associations) around computing? Please describe these new opportunities.

12. To what extent have these partnerships leveraged pre-existing relationships or new opportunities in the local science, technology and economic development environment? Probe using the following:

1. What elements outside each organization were initially perceived as needed to enhance their ability to accomplish goals?

1. How much did members believe that partnership activities would be reciprocally helpful and involve “give and take”?

2. How much did members believe that their respective stand-alone identities would be improved through a continual partnership operation and attainment of goals?

11. How much does the work done by each partner depend on the work of other members? Probe using the following:

1. To what extent is the work shared equally among the partners?

2. How do you assess the effectiveness of the partnership? What measures are you using to make that assessment?

3. How would you describe any challenges to building and maintaining effective partnerships?

11. How can multi-sector partnerships support the replication of promising models of computational thinking (as you understand it) over the long term? What are the barriers that would inhibit sustaining these partnerships?

Conclusion

11. Is there anything else you would like to add that might help us get a better understanding of the impact the CPATH program is having on computing education at your institution (and others with which you may be involved)?

Thank you for your time.

1