NEW PII Research Matrix

ATTACHMENT A: PII RESEARCH MATRIX FOR IMPLEMENTATION, KANSAS, WASHOE, AND ILLINOIS STUDIES

RESEARCH MATRIX FOR IMPLEMENTATION, KANSAS, WASHOE, AND ILLINOIS STUDIES
Instrument	Description	Frequency	Target Population	Annual Number of Respondents	Annual Burden Hours	Research Question
Implementation Study:
Survey of Organization/ System Readiness	Investigates the extent to which PII grantees are positioned through experience and organizational culture to make adjustments that support PII’s two main goals: To improve permanency outcomes for children who have the most serious barriers to permanency through the development of evidence-based, replicable child welfare service interventions To conduct rigorous evaluations to produce high-quality, credible evidence that the PII interventions are effective in decreasing the duration of stays in foster care and potentially replicable by future adopters. The survey will gauge whether grantee organizations support the PII goals by exploring individual respondents’ willingness to use and interest in using evidence-based interventions to address barriers to permanence for children and youth most at risk of LTFC. In addition, the questionnaire will explore respondents’ perceptions of organizational climate as it relates to readiness to change and individual and organizational interest in supporting rigorous evaluation.	Administered once to participants in all grantee sites.	Staff and partners employed by direct service providers for the PII initiative	60	18	What organizational and systemic factors are associated with reaching and sustaining full implementation?
Implementa- tion Drivers Web Survey	Will track processes that sites use to implement interventions. While the intervention and size/scope of implementation varies across sites, there are implementation processes and strategies that are common across projects. Grounded in the National Implementation Research Network framework of organizational and systemic factors that facilitate successful implementation, these measures track progress building implementation infrastructure capacity. The driver assessment survey comprises eight sections that correspond to implementation driver domains that have been the focus of training and technical assistance provided to grantees to strengthen implementation capacity. These domains are: practitioner selection, training, supervision/coaching, performance assessment, decision support data systems, facilitative administration, systems intervention, and leadership.	Completed for the first time 6 months after the first clients to receive the full intervention are served by the first trained workers. Administered annually thereafter to determine the progress made toward the development of the drivers.	Grantees, partners, and stakeholders who are a part of the PII leadership and implementation structure	150	240	What organizational and systemic factors are associated with reaching and sustaining full implementation?
Grantee Case Study Protocol	The protocol allows for a qualitative examination of the key implementation activities; interim products and milestone events that occur during exploration, installation, and initial implementation; and the stages of implementation that set a foundation for achievement of full implementation. In addition, the implementation case study allows for examination of the context in which a PII intervention is being developed and implemented: How do external factors and broad systems issues influence implementation progress and success? In addition, the case study will explore the resources and organizational structure that support the implementation of PII. In essence, the implementation case study allows PII-ET to document the individualized story of each grantee’s implementation experience including enablers and barriers encountered during the implementation process.	Completed for years 1 and 2 and then updated on a yearly basis.	The Evaluation Team completes the case study using existing documentation. Once completed, the ET meets with grantee staff to address missing information and to ensure the case study accurately reflects site specific PII experiences.	30	240	What organizational and systemic factors are associated with reaching and sustaining full implementation? What costs were incurred in reaching and sustaining full implementation?
Fidelity Data (Implementation Quotient Tracker)	A reporting format used to collect data on the percent of caseworkers/practitioners that are implementing the prescribed intervention as intended and with model integrity. This tool captures the primary dependent variable: the proportion of caseworkers/practitioners at a given point in time that are meeting performance assessment criteria. For each practitioner allocated to the program the following will be answered: Is the practitioner in position; Has the practitioner been trained; Has the practitioner being coached as intended; Did the worker meet the performance assessment standards for the most recent assessment; Has the practitioner met the performance assessment standards over the last 4 quarters.	Administered for the first time 6 months after summative evaluation begins, then quarterly thereafter. Enables quantification of a site’s overall progress towards full implementation of the PII initiative.	Site’s evaluation liaison compiles the information for each allocated practitioner using data already available for site-specific evaluation and/or implementation requirements.	2	24	To what extent have grantees reached and sustained full implementation? How many calendar months were needed to reach and sustain full implementation?
Kansas Evaluation:
Family Assessment Battery	(1) Families are videotaped and experts rate them on dimensions such as problem solving, planning a fun activity, teaching, monitoring, and parenting processes; (2) parent-informant items assess social skills, problem behaviors, and academic competence of children and youth.	Completed three times: baseline for all cases, again when parents have completed the PMTO curriculum (6 months in most cases) or at 6 months for control cases, and finally at 12 months	Children age 3-16 with serious emotional disturbance and their parents	300 families	930 (including completion of Caregiver Initial Info Form	Do children age 3-16 in foster care who meet criteria for SED achieve more timely and stable permanence if their families receive early, intensive home-based parent management training compared to children in foster care whose families receive usual services?
North Carolina Family Assessment Scale (NCFAS)	Used by caseworkers to assess and rate family functioning (problems and strengths) in the domains of Environment, Parental Capabilities, Family Interactions, Family Safety, Child Well-being, Caregiver/Child Ambivalence, Social and Community Life, Self-Sufficiency, Family Health, and Readiness for Reunification. It is used at intake and again at case closure and the change score (the difference between the Intake and Closure ratings) indicates the amount of change achieved during the service period.			45 caseworkers respond for an average of 7 study cases each	945
Child and Adolescent Functional Assessment Scale (CAFAS)/ Preschool and Early Childhood Functional Assessment Scale (PECFAS)	Used to assess the day-to-day functioning (impairments and strengths) of children ages 5-19 in the domains of School/Work Role Performance, Home Role Performance, Community Role Performance, Behavior Toward Others, Moods/Emotions, Self-Harmful Behavior, Substance Use, Thinking, and Risk Behaviors. The PECFAS is a version used for younger children. The assessor completes the scale based on information collected during a typical clinical assessment.
Washoe Evaluation:
Overarching research question: What caregiver and child characteristics are associated with length of time in foster care?
Family Assessment Battery	Rates a child's problem behaviors and competencies; reports stresses related to parenting; reports measures of mental health symptoms in various domains; assesses parenting and childrearing attitudes of parents; assesses whether parents are at risk of child abuse or neglect; assesses caregiver’s readiness to accept change; assesses resiliency on eight subscales; examines how respondent’s social relationships provide social support; and provides indication of how stable a caregiver’s housing situation is. At baseline only (not at 6-month follow-up), assesses a caregiver’s history of abuse and trauma in childhood. At 6-month follow-up only, assesses the strength of the helping relationship between social workers and their clients.	Baseline and at 6-month intervals until the case is closed. All cases have an exit assessment at the time of case closure.	Two target populations: (1) new cases involving children aged 17 ½ or younger coming into the system, who are deemed unsafe, are living with a caregiver, and at risk of foster care placement; and (2) families with children who have been in foster care at least 12 months and who have one or more of the identified risk characteristics for long term foster care at time of placement (i.e., parental substance abuse, homelessness/inadequate housing, single parent households, or parental incarceration), a goal of adoption or guardianship, and an available caregiver.	175	525	Are there changes in the following over time, and are changes different for children whose caregivers receive the SAFE-FC intervention compared to those who receive business as usual? Caregiver parenting stress Caregiver mental health Caregiver attitudes about child-rearing Caregiver readiness to accept change Caregiver resiliency Caregiver housing stability. Does the relationship between a caregiver and their caseworker change over time and how does that relationship relate to permanency for the child? Are caregivers’ histories of abuse and trauma associated with children’s length of time in foster care?
Illinois DCFS Evaluation:
Youth Interview Battery	Abbreviated Dysregulation Inventory (ADI): Designed to assess three areas: (a) Emotional/Affective Dysregulation, (b) Behavioral Dysregulation, (c) Cognitive Dysregulation Youth Social Support Instrument (YSS): Designed to assess perceived social support received from familial adults, non-parental, and a peer Youth Emotional Permanency Instrument (YEP): Designed to examine the quality of the youth’s relationships with adults Parent Contact with Youth Instrument (PCY): Designed to assess the youth’s contact with his or her mother and father Trauma Symptom Checklist for Children (TSCC): Designed to assess distress and related symptoms after an acute or chronic trauma including Anxiety, Depression, Anger, Posttraumatic Stress, and Dissociation	Completed twice: baseline and then 6 months after	Foster youth ages 11-16 who are placed in traditional, relative, and specialized foster homes throughout the state of Illinois who, upon reaching the 2-year anniversary of entering foster care, are experiencing mental health symptoms and/or have had at least one placement change	864	461	Do foster youth ages 11-16 placed in traditional, relative, and specialized foster homes throughout the state who, upon reaching the 2-year anniversary of entering care, are experiencing mental health symptoms and/or have had at least one placement change, experience increased permanency rates within 3 years of entry if they receive TARGET services compared with similar youth who received services as usual?
Biological Parent Interview Battery	Abbreviated Dysregulation Inventory (ADI): Designed to assess three areas: (a) Emotional/Affective Dysregulation, (b) Behavioral Dysregulation, (c) Cognitive Dysregulation Social Provisions Scale (SPS): Designed to examine how respondent’s social relationships provide social support. Includes six constructs: (a) Attachment, (b) Social Integration, (c) Reassurance of Worth, (d) Reliable Alliance, (e) Guidance, and (f) Opportunity for Nurturance		Biological parents of the target foster youth, who fit the requirement that they be a viable resource for reunification	518	104
Foster Parent Interview Battery	Abbreviated Dysregulation Inventory (ADI)-Parent Version: Designed to assess three areas: (a) Emotional/Affective Dysregulation, (b) Behavioral Dysregulation, (c) Cognitive Dysregulation. The parent version, which is identical to the original instrument, instructs the parent to complete the instrument using the child as a reference. Parenting Practices Chicago Survey-Parent Version (PPCS): Designed to assess three constructs: Positive Parenting, Discipline Effectiveness, Discipline Avoidance, and Monitoring Social Provisions Scale (SPS): Designed to examine how respondent’s social relationships provide social support. Includes six constructs: (a) Attachment, (b) Social Integration, (c) Reassurance of Worth, (d) Reliable Alliance, (e) Guidance, and (f) Opportunity for Nurturance.		Foster parents of the target youth	864	461

1