National Center on Early Childhood Quality Improvement Feedback Surveys

National Center on Early Childhood Quality Assurance

Generic Data Collection Bank of Questions

Overview: This document, using constructs and format developed by the Office of Head Start and NORC to facilitate data aggregation and analysis across events of the national training and technical assistance centers and reviewed by the Office of Child Care (OCC), outlines a bank of data collection questions to assess customer feedback to technical assistance. It identifies four constructs and related survey questions to be used by the National Center Early Childhood Quality Assurance (NCECQA) to evaluate the delivery of universal, targeted, tailored, and intensive technical assistance (TA). These constructs are Content Quality, Content Fit, Increased Knowledge, and Intended Application of Knowledge.

This bank of questions is for poll, discussion, pre and post surveys and other customer feedback surveys for the NCECQA’s training and TA offerings. Most universal and targeted TA engagements will include a poll question and a post-survey. A pre- and post-survey will likely be used for intensive, and individualized TA engagements. NCECQA will select the most appropriate questions from this bank of questions based on the specific TA event.

All information collection requests will include the follow statement:

PAPERWORK REDUCTION ACT OF 1995 (Public Law 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to obtain feedback regarding technical assistance received. Public reporting burden for this collection of information is estimated to average 5 minutes per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0401 and the expiration date is 5/31/2027. If you have any comments on this collection of information, please contact Carol Hartman at Carol.Hartman@icf.com.

Construct	Survey Question	Optional or Recommended	Responses	Example Type of TA for Use of Question
Content Quality	The purposes and objectives are clear.	Recommended	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	The experience of being in the [Event] is relevant to my current work (i.e. pertinent to your current work).	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	The technical assistance provided was useful to my work.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	Resources are provided as needed.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	The identified goals and outcomes of the technical assistance/session/event were met.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Targeted, Intensive
	I am increasing my awareness and knowledge by participating in the [Event}	Recommended	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
Content Fit	The experience of being in the [Event] is useful (i.e. provides you with practical information or a practical perspective to inform your work.)	Optional	Scaled item Far too advanced (1) A bit too advanced (2) About right (3) A bit too simple (4) Far too simple (5)	Individualized, Targeted, Intensive
	What outcome or desired result, if any, was achieved with the support of this technical assistance?	Optional	Open Response	Individualized, Targeted, Intensive
	The experience of being in the [Event] is influential (i.e. influenced your thinking: gave you “a-ha” moments; enabled you to think in a different way about your system(s), your partnerships, or other critical aspects of your work; and/or helped you analyze, synthesize, or integrate information in a new way.)	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted, Intensive
	The materials for use with child care providers are easy to use and easy to understand.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted
Increased Knowledge and Skill	How much did the event increase your knowledge of (specific topics) presented?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Universal, Individualized, Targeted, Intensive
	Work with the technical assistance staff increased my knowledge on potential tasks, plans, or directions for [TA focus].	Recommended	Scaled item NA Unsure Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Intensive
	Work with the technical assistance staff helped me identify priorities and clear action steps for [TA focus].	Recommended	Scaled item NA Unsure Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Intensive
	Work with the technical assistance staff connected me to resources (internal or external) that will help me more easily access answers/solutions in the future.	Optional	Scaled item NA Unsure Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Intensive
	Work with the technical assistance staff influenced my overall approach to improving the quality of early care and education.	Optional	Scaled item NA Unsure Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Intensive
	Work with the technical assistance staff was valuable to my work supporting [TA focus].	Optional	Scaled item NA Unsure Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Intensive
	In a few words, describe up to 3 goals you were hoping to accomplish through your work with the NCECQA technical assistance specialist.	Optional	Open response	Individualized, Intensive
	In a few words, describe up to (x) goals you hope to accomplish through your work with the NCECQA technical assistance specialist.	Optional	Open response	Individualized, Intensive
	Please indicate the extent to which the technical assistance helped you accomplish your goals.	Optional	Scaled item I’m not sure (1) The TA did not help me achieve my goals (2) The TA somewhat helped me achieve my goals (3) The TA helped me achieve my goals (4)	Individualized, Intensive
	[If it did not help is selected] Please describe how the technical assistance could better support you in achieving your goal.	Optional	Open response	Individualized, Intensive
	[If helped or somewhat helped is selected] What resources or supports provided by the NCECQA technical assistance specialist were most helpful?	Optional	Open response	Individualized, Intensive
	I feel ready to apply new resources or ideas shared to my work.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Targeted, Intensive
	Overall, the experience is relevant and fits my needs.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Targeted, Intensive
	I significantly improved my understanding of the content [needed to offer this training or to train other trainers].	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted
	How much has your understanding or awareness of [insert concept] improved?		Scaled item Not Improved (1) A Little Improved (2) Moderately Improved (3) Significantly Improved (4)	Universal, Targeted
	I feel ready to [offer this training or to train other trainers].	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted
	How much did the technical assistance increase your agency’s compliance with CCDF rules and policies?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Universal, Individualized, Targeted, Intensive
	How much did the technical assistance increase integration of your agency’s functions (e.g., administrative, technological, policy)?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Targeted, Intensive
	How much did the technical assistance increase your agency’s ability to establish and leverage partnerships to improve coordination and collaboration with other agencies?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Universal, Individualized, Targeted, Intensive
	BEFORE this technical assistance, my knowledge of (specific content/topics addressed) can best be described as … AFTER this technical assistance, my knowledge of (specific content/topics addressed) can best be described as …	Optional	Scaled item No knowledge (1) Minimal knowledge (2) Moderate knowledge (3) High level of knowledge (4)	Individualized, Targeted, Intensive
	BEFORE this technical assistance, my comfort with communicating about (specific content/topics addressed) can best be described as … AFTER this technical assistance, my comfort level with communicating about (specific content/topics addressed) can best be described as …	Optional	Scaled item No comfort (1) Minimal comfort (2) Moderate comfort (3) High level of comfort (4)	Individualized, Targeted, Intensive
Intended Application of Knowledge	I learned something during this event that I plan to use in my work.	Recommended	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	How have you used- or how do you plan to use- what you learned from this technical assistance in your work?	Optional	Open response	Individualized, Intensive
	How will you use the content covered today?	Optional	Open response	Universal, Individualized, Targeted, Intensive
	How likely will you use at least one of these strategies or practices?	Optional	Scaled item Won’t (1) Probably not (2) Will consider (3) Definitely (4)	Universal
	How likely are you to use the resources that were shared in the technical assistance?	Optional	Scaled item Very unlikely (1) Unlikely (2) Somewhat likely (3) Already use this resource (4)	Universal, Individualized, Targeted, Intensive
	What has your agency done/is your agency doing (around this topic)?	Optional	Open Response	Individualized, Targeted, Intensive
	Add your example of (this practice) to (the mural board/chat box).	Optional	Open Response	Individualized, Targeted, Intensive
	BEFORE this technical assistance, my comfort with implementation efforts related to [specific content/topics addressed] can best be described as … AFTER this technical assistance, my comfort level with implementation efforts related to [specific content/topics addressed] can best be described as …	Optional	Scaled item No comfort (1) Minimal comfort (2) Moderate comfort (3) High level of comfort (4)	Individualized, Targeted, Intensive

Optional Constructs: The following four constructs and related questions are optional, designed for planning future technical assistance, providing internal feedback to presenters, and improving technical assistance delivery. These constructs are Satisfaction, Presenter Quality, Formative Assessment, and Barriers to Application of Knowledge.

Construct	Question	Type	Responses	Type of TA for Use of Question
Satisfaction	I would recommend this technical assistance/session/event to my colleagues	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted
	The script and resources provide me with the information I need to offer the training.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted
	What worked well about today’s session?	Optional	Open Response	Universal, Individualized, Targeted, Intensive
	What didn’t work well in today’s session?	Optional	Open Response	Universal, Individualized, Targeted, Intensive
	What is your preferred way of learning virtually?	Optional	Choices Small Group Discussions Large Group Discussions Presentations Working Session Combination of Above	Universal, Individualized, Targeted, Intensive
	What is your preferred technical assistance session length?	Optional	Choices Less than 60 minutes 60 minutes 90 minutes More than 90 minutes	Universal, Individualized, Targeted, Intensive
	What is your preferred meeting schedule for ongoing technical assistance?	Optional	Choices Every Two Weeks Once per Month Quarterly Other _____	Universal, Individualized, Targeted, Intensive
Presenter Quality	The presenters had robust knowledge and experience with the content.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	The facilitator is well prepared	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	The facilitator helps the group value the contributions of each member.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	The facilitator helps guide discussions and share activities about our shared interest.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive
	The presenters were able to respond appropriately to my questions.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted,
	The NCECQA Specialist’s approach to working with our team on the technical assistance is:	Optional	Scaled Item Poor (1) Fair (2) Good (3) Excellent (4)	Individualized, Intensive
	The NCECQA Specialist’s ability to convey important concepts effectively is:	Optional	Scaled Item Poor (1) Fair (2) Good (3) Excellent (4)	Individualized, Intensive
	The NCECQA Specialist’s expertise relative to the challenges your state is experiencing is:	Optional	Scaled Item Poor (1) Fair (2) Good (3) Excellent (4)	Individualized, Intensive
	Please write any comments you have about your engagement with the technical assistance specialist.	Optional	Open response	Individualized, Intensive
	Please write any comments you have about the NCECQA Specialist	Optional	Open response	Individualized, Intensive
Formative Assessment	What worked well about today’s session?	Optional	Open response	Individualized, Targeted, Intensive
	What questions do you have about the content covered today? What, if anything, is confusing or needs clarification?	Optional	Open response	Individualized, Targeted, Intensive
	What recommendations do you have for future sessions?	Optional	Open response	Universal, Individualized, Targeted
	Are there other webinar topics that would be useful to you?	Optional	Open response	Universal, Targeted
	How will you use the content covered today?	Optional	Open response	Individualized, Targeted, Intensive
	What additional comments do you have about the curriculum/content covered today?	Optional	Open response	Individualized, Targeted, Intensive
	What is the likelihood you or your staff would participate in a peer learning opportunity on this topic?	Optional	Scaled item Very unlikely (1) Unlikely (2) Likely (3) Very likely (4)	Universal, Individualized, Targeted, Intensive
	What additional T/TA opportunities would help you further improve your policies or practices?	Optional	Open response	Individualized, Targeted, Intensive
	Is your state, territory, or Tribe currently engaged in an initiative around [subject]?	Optional	Yes (1) No (2)	Universal, Individualized, Targeted, Intensive
	Would you be willing to share the work of your Lead Agency related to this technical assistance opportunity?	Optional	Yes (1) No (2)	Universal, Individualized, Targeted, Intensive
	How did you find out about this event?	Optional	Open response	Universal, Individualized, Targeted, Intensive
	I would like to learn about (topic #1) (topic #2) (topic #3) (topic #4) and/or (topic #5)	Optional	Choices	Individualized, Targeted, Intensive
	What are you most interested in learning about (this specific topic)?	Optional	Open response	Universal, Individualized, Targeted, Intensive
Barriers to Application of Knowledge	What factor(s), if any, may prevent you from using what you learned? (Select all that apply.)	Optional	I have not faced any barriers; I have used or intend to use what I learned Lack of time  Not enough staff  Need for more TA and/or professional development  Competing priorities  Limited funds or other resources to support this effort  Lack of state policies or processes to support this effort  Lack of support/guidance from state leadership  Need more time to build readiness among state leaders to support this effort  Need more time to build buy-in from other partners   What I’ve learned is not applicable to my work  I don’t have the authority or influence to gain support for this effort   Other (Please describe):______________	Individualized, Intensive
	To what extent did the technical assistance address how to overcome this factor as a potential barrier to using what you’ve learned in your work?	Optional	Scaled item Did not address this barrier (1) Briefly addressed this barrier (2) Addressed this barrier, but I was left with a few questions (3) Fully addressed this barrier (4)	Individualized, Intensive
	What supports or resources would you need to overcome this barrier? How could we improve this work to better meet your needs?	Optional	Open response	Individualized, Intensive

26