PACT Appendix J - Calculations for Burden Requested - 2-11-13

Parents and Children Together (PACT) Evaluation

APPENDIX J

CALCULATIONS FOR BURDEN REQUESTED

Fuller explanations of calculations for reporting burden for the data collection instruments included in this ICR are presented in this Appendix.

For cost calculations in all of the following tables, we estimate the average hourly wage for staff at the grantee organizations is the average hourly wage of “social and community service managers” taken from the U.S. Bureau of Labor Statistics, National Compensation Survey, 2010 ($27.86). The average hourly wage of program applicants is estimated from the average hourly earnings ($4.92) of study participants in the Building Strong Families Study (Wood et al. 2010). These average hourly earnings are lower than minimum wage because many study participants were not working. We expect that to also be the case for PACT study participants.

Previously Approved Burden: (1) Selecting Study Grantees and (2) RF Introductory Script, (3) RF Baseline Survey, and (6) RF Study MIS

Table 1 summarizes burden and costs for previously approved instruments. The burden and costs for all these instruments are annualized over three years, meaning the total number of respondents over the three year period has been divided by three to determine the annual number of respondents.

Table 1. PREVIOUSLY APPROVED AND ONGOING – Estimates of Burden and Costs for the PACT Evaluation

Activity/Respondent	Annual Number of Respondentsa	Number of Responses per Respondent	Average Burden per Response (hours)	Total Annual Burden Hoursa	Average Hourly Wage	Total Annualized Cost
SITE SELECTION
(1) Selecting study grantees1 Discussions/ program and partner organization staff	50	1	1	50	$29.34	$1,467
IMPACT
Responsible Fatherhood Grantee Impact Evaluation
(2) Introductory script2
a) Program staff	30	70.2	0.167	351	$27.86	$9,779
b) Program applicants	2,105	1	0.167	351	$4.92	$1,726
(3) Baseline survey2
Study participants	2,000b	1	0.5	1,000	$4.92	$4,920
Responsible Fatherhood Grantee Implementation Evaluation
(6) RF study MIS2
Program staff	30	2,533	0.03	2,533	$27.86	$70,569
Total				4,285		$88,461

^a Burden estimates are annualized over three years.

^b Note that the 2,000 “study participants” are part of the 2,105 “program applicants,” as the study participants will all begin as program applicants. (Five percent of program applicants are not expected to agree to participate in the study and complete the baseline survey, thus there are 5 percent more program applicants than study participants.) The 2,000 study participants do not represent 2,000 individuals in addition to the 2,105 program applicants.

¹ Approved April 20, 2012.

² Approved October 31, 2012.

Burden for Data Collection Efforts Covered by this ICR

Impact Evaluation

Table 2 summarizes burden and costs for newly requested instruments associated with the impact study (which, as stated in A1, contains experimental impact evaluations complemented with implementation and qualitative strategies). The burden and costs for all these instruments are annualized over three years, meaning the total number of respondents over the three year period has been divided by three to determine the annual number of respondents.

Table 2. CURRENT REQUEST – Estimates of Additional Burden and Costs for the Impact Portion of the PACT Evaluation

Instrument Respondent	Annual Number of Respondentsa	Number of Responses per Respondent	Average Burden per Response (hours)	Total Annual Burden Hoursa	Average Hourly Wage	Total Annualized Cost
IMPACT
Healthy Marriage Grantee Impact Evaluation
(4) Introductory script
Program staff	30	70.2	0.167	351	$27.86	$9,779
Program applicants	4,210	1	0.167	702	$4.92	$3,454
(5) Baseline survey
Study participants	4,000 b	1	0.5	2,000	$4.92	$9,840
Healthy Marriage Grantee Implementation Evaluation – MIS
(7) HM Study MIS
Program staff	30	3,400	0.033	3,400	$27.86	$94,724
Responsible Fatherhood and Healthy Marriage Grantee Implementation Evaluation – Additional Implementation Data Collection Instruments
(8) Semi-structured interview topic guide
Program staff	250	2	1.03	517	$27.86		$14,404
(9) On-line survey
Program staff	250	2	0.5	250	$27.86		$6,965
(10) Telephone interviews (with staff at referral organizations)
Program staff at referral organizations	50	1	0.5	25	$27.86		$697
(11) On-line Working Alliance Inventory
1) Program staff	50	20	0.167	167	$27.86		$4,653
2) Program Participants	1,000	1	0.167	167	$4.92		$822
(12) Focus group guide
Program participants	600	1	1.5	900	$4.92		$4,428
(13) Telephone interviews
Program participants (program dropouts)	150	1	0.25	38	$4.92		$187
Responsible Fatherhood Grantee Qualitative Evaluation
(14) Guide for in-person, in-depth interviews
Study participants	32	3	2	192	$4.92		$945
(15) Check-in call guide
Study participants	32	4	0.167	21	$4.92		$103
Total				8,730			$151,001

^a All burden estimates are annualized over three years.

^b Note that the 4,000 “study participants” are part of the 4,210 “program applicants,” as the study participants will all begin as program applicants. (Five percent of program applicants are not expected to agree to participate in the study, thus there are 5% more program applicants than study participants.) The 4,000 study participants do not represent 4,000 individuals in addition to the 4,210 program applicants.

Experimental Impact (Healthy Marriage Grantee Evaluation). The first part of Table A.7 summarizes the proposed total estimated reporting burden for the use of the HM (4) introductory script and (5) baseline survey. The estimates assume that five HM grantees will participate in the impact study and that there will be 6,316 couples who apply to the program (12,632 applicants) across the five grantees. Further description follows:

(4) Introductory script (for HM program staff to discuss with program applicants). We expect 90 program staff to provide information about the HM program and the PACT study to the 6,316 couples. Annualizing over three years, results in 30 program staff/year and 2,105 couples/year. Hence each program staff member meets with about 70.2 couples to explain the program services and random assignment. These meetings are expected to last approximately 10 minutes. Thus, the total annualized burden for program staff is 351 hours/year (30 program staff/year x 70.2 program applicants x 10 minutes = 21,060 minutes/year; 21,060 minutes/year ÷ 60 minutes = 351 hours/year). Because these meetings will involve one staff member and the two applicants who make up the couple, the total burden on program applicants is twice the burden on grantee staff. Annualized, there are 4,210 applicants/year (12,632 over three years), each of whom will likewise spend 10 minutes per meeting for an annual burden of 702 hours/year (4,210 applicants/year x 10 minutes = 42,100 minutes/year; 42,100 minutes/year ÷ 60 minutes = 702 hours/year).

(5) Baseline survey (for HM study participants). We expect there will be an annualized 4,210 applicants/year (12,632 applicants over three years) during the study intake period. It is assumed that 95 percent of applicants to the program will consent to participate in the study and complete the baseline interview. Thus, annualized, 4,000 (95 percent of 4,210) applicants/year are expected to complete the baseline survey. We expect each survey to last 30 minutes, for a total of 2,000 burden hours/year (4,000 applicants/year x 30 minutes = 120,000 minutes/year; 120,000 minutes/year ÷ 60 minutes = 2,000 hours/year).

Implementation– MIS (Healthy Marriage Grantee Evaluation). The next part of Table A.7 describes the burden associated with the (7) HM study Management Information System (MIS). This burden is based on the number of computer entries grantee staff will make as they enroll and track participation by participants.

We calculate that there will be 306,000 MIS entries over three years:

• We anticipate 12,000 study participants (members of 6,000 couples) will be randomly assigned (with 6,000 participants – members of 3,000 couples – assigned to the treatment group and 6,000 assigned to the control group): each of these applicants will have one MIS entry, for a total of 12,000 MIS entries.

• During program participation, information will be entered about 49 times¹ for each of the 6,000 program group participants (members of 3,000 couples), for a total of 294,000 MIS entries.

We expect there will be 6 program staff in up to 15 grantees (5 as part of the impact study, 10 as part of an implementation/qualitative only study) entering information into the study MIS, for a total of 90 program staff. This means that each staff person will need to make 3,400 MIS entries (306,000 MIS entries ÷ 90 staff = 3,400 MIS entries/staff).

We estimate that each entry will take about 2 minutes.²

We have annualized the program staff (90 staff ÷ 3 years = 30 staff/year), for a total of 30 annualized respondents. Thus, there will be a total annualized burden of 3,400 hours/year (30 program staff/year x 3,400 MIS entries/staff x 2 minutes = 204,000 minutes/year; 204,000 minutes/year ÷ 60 minutes = 3,400 hours/year).

Implementation Evaluation – Additional Implementation Data Collection Instruments (Responsible Fatherhood and Healthy Marriage Grantee Evaluation). The next part of Table A.7 describes the burden associated with additional implementation data collection instruments in RF and HM sites. Up to 30 sites are anticipated: though the mix may change, at present, burden is calculated for …

• RF sites: 5 impact, and 10 implementation/qualitative only sites; and

• HM sites: 5 impact, and 10 implementation/qualitative only sites.

(Although this section of item A12 refers to the impact evaluation and not the implementation/qualitative only evaluation, the same implementation instrumentation will be used in both evaluations, and thus they are discussed together here.)

(8) Semi-structured interview topic guide (for program staff). We plan to visit up to 30 sites. In each site, we expect to interview up to 25 staff. Hence, in total, annualized over three years, there will be 250 respondents/year (30 sites x 25 staff ÷ 3 years = 250 staff/year). Grantee visits to conduct discussions with staff will occur twice over the three-year study period (once in the first year and once in the second). We expect each interview to last an average of 62 minutes. Hence, the annualized burden estimate for the staff interviews is 517 hours (250 staff/year x 2 visits x 62 minutes = 62,000 minutes/year; 620,000 minutes/year ÷ 60 minutes = 517 hours/year).

(9) On-line survey (for program staff). The staff survey will be administered to up to 25 program staff in each of up to 30 grantees, for a total annualized number of 250 respondents/year (30 sites x 25 staff ÷ 3 years = 250 staff/year). The survey will be administered twice. Each survey will last approximately 30 minutes, for a total annualized burden estimate of 250 hours (250 staff/year x 2 survey administrations x 30 minutes = 15,000 minutes/year; 15,000 minutes/year ÷ 60 minutes = 250 hours/year).

(10) Telephone interview guide (for program staff at referral organizations). Telephone interviews will occur with up to 5 referral organizations for each of up to 30 grantees for a total annualized number of 50 respondents (5 referral organizations x 30 grantees ÷ 3 years = 50 referral organizations/year). Each telephone interview will last approximately 30 minutes, for a total annualized burden estimate across the three study years of 25 hours (50 referral organizations/year x 30 minutes = 1,500 minutes/year; 1,500 minutes/year ÷ 60 minutes = 25 hours/year).

(11) On-line Working Alliance Inventory (for program staff and participants). We expect up to 5 case manager staff in each of up to 30 grantees will complete the inventory for a total annualized number of 50 staff (5 case manager staff x 30 grantees ÷ 3 years = 50 staff/year). We expect up to 100 participants will complete the inventory in each of up to 30 grantees for a total annualized number of 1,000 participant respondents (100 participants x 30 grantees ÷ 3 years = 1,000 participants/year). Each case manager staff will be expected to complete an inventory once for each of up to 20 participants. Each participant will complete the instrument once. It will require about 10 minutes per respondent to complete, for a total annualized burden of 167 hours for case manager staff (50 case manager staff/year x 20 participants x 10 minutes = 10,000 minutes/year; 10,000 minutes/year ÷ 60 minutes = 167 hours/year) and 167 hours for participants (1,000 participants/year x 1 case manager staff x 10 minutes = 10,000 minutes/year; 10,000 minutes/year ÷ 60 minutes = 167 hours/year).

(12) Focus group discussion guide (for program participants). In each of up to 30 sites, two focus groups will be conducted in each of the two rounds of site visits. We will recruit up to 15 participants for each focus group (no participant will attend multiple focus groups). Hence, we expect that there will be up to 1,800 participants in focus groups (30 sites x 15 participants x 2 focus groups x 2 rounds of site visits = 1,800 participants) or, annualized, 600 respondents/year (1,800 participants ÷ 3 years = 600 participants/year). We expect each focus group to last about 90 minutes. The annualized burden estimate for the focus groups is 900 hours (600 participants/year x 90 minutes = 54,000 minutes/year; 54,000 minutes/year ÷ 60 minutes = 900 hours/year).

(13) Telephone interview guide (for program dropouts). The telephone interviews with program dropouts are planned to occur in the second year, with 15 program dropouts in each of up to 30 grantees – for an annualized total of 150 program dropouts/year (15 program dropouts x 30 grantees ÷ 3 years = 150 program dropouts/year). We expect each telephone call to last 15 minutes, for an annualized burden estimate of 38 hours/year (150 program dropouts/year x 15 minutes = 2,250 minutes/year; 2,250 minutes/year ÷ 60 minutes = 38 hours/year).

Qualitative (Responsible Fatherhood Grantee Evaluation). The next part of Table A.7 describes the burden associated with qualitative data collection instruments in RF sites.

(14) In-person, in-depth interview guide (for program dropouts). We expect to interview approximately 95 fathers across the RF grantees for a total annualized number of 32 participants (95 participant fathers ÷ 3 years = 32 participants/year). Each father will be interviewed three times. Interviews with the respondents will each last approximately 2 hours. Hence, the total annualized burden will be 192 hours (32 participants/year x 3 interviews x 2 hours = 192 hours).

(15) Telephone check-in guide (for program participants). Following on the first and second (14) in-person, in-depth interviews, we expect to hold check-in calls with the same approximate 95 fathers across the RF grantees for a total annualized number of 32 participants (95 participant fathers ÷ 3 years = 32 participants/year). Twice between the first and second in-person interviews, and then twice again between the second and third interviews, a 10-minute check-in call will be made. Hence, the total annualized burden will be 21 hours/year (32 participants/year x 4 calls x 10 minutes = 1,280 minutes/year; 1.280 minutes/year ÷60 minutes = 21 hours).

Implementation/Qualitative Only Evaluation: Hispanic RF Sub-study

Table 3 summarizes burden and costs for newly requested instruments associated with the Hispanic RF implementation/qualitative only sub-study. The burden and costs for all these instruments are annualized over three years, meaning the total number of respondents over the three year period has been divided by three to determine the annual number of respondents.

Table 3. CURRENT REQUEST – Estimates of Burden and Costs for the Hispanic RF Implementation/Qualitative Only Sub-study of the PACT Evaluation

Instrument Respondent	Annual Number of Respondentsa	Number of Responses per Respondent	Average Burden per Response (hours)	Total Annual Burden Hoursa	Average Hourly Wage	Total Annualized Cost
IMPLEMENTATION/QUALITATIVE ONLY
Responsible Fatherhood Grantee Implementation Evaluation – Grantees with a Focus on Hispanic Populations
(16) Semi-structured interview topic guide
Program staff	42	1	1.5	63	$27.86		$1,755
(17) Focus group guide
Program participants	20	1	1.5	30	$4.92		$248
(18) Questionnaires
Program participants in focus groups	20	1	0.333	7	$4.92		$34
Total				100			$2,037

^a All burden estimates are annualized over three years.

(16) Semi-structured interview topic guide (for program staff). (This instrument differs from the (8) semi-structured interview topic guide in that it focuses heavily on program features related to Hispanic culture.) Individuals or small groups of staff will participate in each discussion, with a total of up to 25 staff in each of the 5 grantees for a total annualized number of 42 staff/year (25 staff x 5 grantees ÷ 3 years = 42 staff/year). We expect each discussion to last an average of 90 minutes. The annualized burden estimate for the semi-structured interviews is 63 (42 staff/year x 90 minutes = 3,780 minutes/year; 3,780 minutes/year ÷ 60 minutes = 63 hours/year).

(17) Focus group discussion guide (for program participants). One focus group with up to 12 participants will be conducted in each of the 5 sites, for an annualized total of 20 participants/year (12 participants x 5 sites ÷ 3 years = 20 participants/year). Each focus group will last up to 90 minutes. The total annualized burden for the focus groups is 30 hours/year (20 participants/year x 90 minutes = 1,800 minutes/year; 1,800 minutes/year ÷ 60 minutes = 30 hours).

(18) Questionnaires (for program participants). All the focus group participants will be asked to complete a questionnaire (as just discussed, there will be an annualized number of 20 participants/year). Questionnaires are estimated to take 20 minutes to complete. Hence, the total annualized burden for these questionnaires is 6.7 hours (20 participants/year x 20 minutes = 400 minutes/year; 400 minutes/year ÷ 60 minutes = 6.7 hours).

Combined Total Burden

Table 4 summarizes the total estimated reporting burden and costs for the previously approved and currently requested burden. If the current request is approved, 13,115 hours and $241,149 would be approved for the PACT study.

Table 4. Estimate of Burden and Cost for the PACT Evaluation – TOTAL Burden Request

Data Collection	Total Annual Burden Hoursa	Total Annualized Cost
Previously Approved and Ongoing (from Table A.6)	4,285	$88,461
Current Request – Impact Evaluation (from Table A.7)	8,730	$151,001
Current Request – Hispanic RF Implementation/Qualitative Only Sub-study (from Table A.8)	100	$2,037
Total	13,115	$241,149

^a Burden estimates are annualized over three years.

1 We estimate that there will be 24 MIS entries to record group sessions, an additional 24 entries to record case management services, and one MIS entry for program exit.

2 Based on the Building Strong Families Study and the speed at which data can be entered, we expect each entry to take about 2 minutes.