Developing Teaching Materials

OMB Control Number: 0970-0401

Expiration Date: June 30, 2024

Developing Teaching Materials:

Usability Review Feedback, Instructor Semi-Structured Panel Interview Protocol

Introduction:

Hi, I’m [NAME] working with MEF Associates. Thanks for meeting with me today to talk about the teaching materials developed for the Office of Planning, Research, and Evaluation (OPRE) at the Administration of Children and Families.

  

Today, we will gather any feedback you have about the teaching materials. All together this won’t take more than 90 minutes. 

We want to use your ideas to make the teaching materials better, so please don’t hesitate to share feedback. Also, we would like to include the names of individuals who contributed to the development of the teaching materials in a list of acknowledgements when we publish the materials. Please let us know if you would like to be included or remain anonymous. You can let me know now or send me an email after this call. [PAUSE FOR A RESPONSE] 

My colleague [NAME] is here to take notes, and I’d like to record our conversation so that we don’t miss anything. Do I have your permission to record? [IF YES, Great, thank you. IF NO, Okay, we will take notes as we go along] 

Do you have any questions before we start? 

Semi-Structured Panel Interview:

I’d like to start by asking about your general thoughts on the teaching materials [Note to interviewer: It will be helpful to have the relevant materials open on your screen].

1. What are your initial thoughts on the usefulness of the [name of materials] materials?

   1. Probe for thoughts on each component [instructor guide, lecture slides, (activity/handout/questions for further learning)]

2. We’ve identified these disciplines for which we think this set of materials will be most appropriate: [list disciplines; consider sharing on screen from the instructor guide]. Do you think instructors in those disciplines are likely to use these materials? Why or why not?

Now, I’d like to ask questions about how you or other instructors might use these materials.

3. If you were to use these materials in a course, how would you use it?

   1. Would you use each component of the materials? Why or why not?

   2. Probe: Do you think other instructors would use them in a similar way? Do you think instructors with more or less experience teaching courses would use the material differently? How about instructors teaching at different levels?

4. Would you adapt any of part of the materials? If so, how would you modify the materials and why?

5. How would you use the instructor guide?

   1. Do you think the ‘recommended use’ guidance on page(s) X is helpful? [Note to interviewer: it might be helpful to display the ‘recommended use’ guidance on screen]

   2. What could make the instructor guide more useful?

   3. Do you think instructors would need additional resources or guidance to use these materials in their courses successfully?

6. How would you use the slides?

   1. Which sections of the slides do you find most useful? Least useful?

   2. What changes could make the slides more useful for instructors?

7. [For Behavioral Science, CCDF, and Self-Regulation materials only] Is the activity something you can imagine an instructor using in their course? Would they need more guidance or additional resources to implement it successfully?

8. [For Head Start materials only] Are the discussion questions something you can imagine an instructor using in their course? Why or why not?

9. If you could choose only one component (e.g., slides (or a subset of slides), activity, handout) of the materials to use in your course, what would it be? Why?

10. What component(s) of the materials seemed least useful to you and why? What would you change about it to make it more useful?

My next set of questions are about the formatting and features of the materials.

11. What did you like about the format of the materials? What did you dislike?

    1. Probe for thoughts on each component [instructor guide, lecture slides, (activity/handout/questions for further learning)]

    2. Probe: Are the materials easy to read/digest? Are they visually engaging?

12. What aspects of the materials do you think make them easy to use?

    1. Probe for thoughts on each component [instructor guide, lecture slides, (activity/handout/questions for further learning)]

       1. Probe for thoughts on instructor notes in the lecture slides

13. What aspects of the materials do you think make them difficult to use?

    1. Probe for thoughts on each component [instructor guide, lecture slides, (activity/handout/questions for further learning)]

14. Are there any features of the materials that might make you or other instructors not want to use them?

    1. Probe for thoughts on each component [instructor guide, lecture slides, (activity/handout/questions for further learning)]

    2. Probe for thoughts on the length of materials

Finally, I have some closing questions.

15. How useful are these teaching materials compared to other teaching materials you have previously used or come across?

    1. Probe: are there other types of teaching materials you can think of that would be more useful (e.g., a teaching dataset)?

16. Would you be interested in using OPRE teaching materials in the future? Why or why not?

17. Would you recommend these teaching materials to other instructors? Why or why not?

18. Given when instructors begin planning course content and materials, when and where do you think would be the best time and place to announce the availability of OPRE teaching materials?

19. Is there anything else that came to mind as you reviewed these materials that you think is important for our team to consider?

[Thank interviewee for time/Honorarium instructions]

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN:  The purpose of this information collection is to gather feedback on the usability of the instructional materials developed as a part of the Develop Teaching Materials Based on OPRE’s Research and Evaluation project. Public reporting burden for this collection of information is estimated to average 2.5 hours per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information.  This is a voluntary collection of information. agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number.  The OMB # is 0970-0401 and the expiration date is 06/30/2024.  If you have any comments on this collection of information, please contact Kate Stepleton, project director at MEF Associates. She can be reached at 206-653-0222 or kate.stepleton@mefassociates.com