Supporting Statement
A For the Paperwork Reduction Act of 1995: Approval for the Baseline Data Collection, Implementation Study Site Visits, and Staff Surveys for the Job Search Assistance (JSA) Strategies Evaluation
OMB No. 0970-0440
November 4, 2014
Submitted by:
Office of
Planning,
Research & Evaluation
Administration for Children & Families
U.S. Department of
Health
and Human Services
Federal Project Officer
Erica Zielewski
Table of Contents
A.1 Necessity for the Data Collection 1
A.2 Purpose of the Data Collection 4
A.3 Improved Information Technology to Reduce Burden 6
A.4 Efforts to Identify Duplication 6
A.5 Involvement of Small Organizations 6
A.6 Consequences of Less Frequent Data Collection 6
A.8 Federal Register Notice and Efforts to Consult Outside the Agency 7
A.12 Estimation of Information Collection Burden 10
A.13 Cost Burden to Respondents or Record Keepers 11
A.14 Estimate of Cost to the Federal Government 11
A.16 Publication Plans and Project Schedule 11
A.17 Reasons not to Display OMB Expiration Date 12
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 12
Attachments
Baseline Information Form
Master Protocol and Topics by Respondent
JSA Staff Survey
JSA Staff Survey Screen Shots
JSA Supervisor Survey
JSA Supervisor Survey Screen Shots
Informed Consent Form
OMB 60-Day Notice
This document provides supporting statements for the collection of information for the Job Search Assistance (JSA) Strategies Evaluation (hereafter, JSA Evaluation), funded by the Office of Planning, Research and Evaluation (OPRE) in the Administration for Children and Families (ACF). The goal of the JSA evaluation is to determine which JSA strategies are most effective in transitioning Temporary Assistance for Needy Families (TANF) applicants and recipients to work. The impact study will randomly assign individuals to contrasting JSA approaches and then compare their employment and earnings to determine their relative effectiveness in moving TANF recipients to work and increasing their earnings. The implementation study will document services received under each JSA approach, and provide operational lessons. The evaluation is being conducted by Abt Associates and its partner, Mathematica Policy Research.
The study received OMB approval in October 2013 for JSA Evaluation data collection instruments used as part of the field assessment and site selection process (OMB No. 0970-0440). Instruments approved in that submission included the Discussion Guide for Researchers and Policy Experts, the Discussion Guide for State and Local TANF Administrators, and the Discussion Guide for Program Staff.
Although the full evaluation involves four additional data collection instruments, this submission seeks OMB approval for three data collection instruments to be used as part of the JSA Evaluation:
Baseline data collection. This is for the collection of baseline data from TANF recipients at the time of enrollment in the study.
Implementation study site visits. This activity involves conducting site visits for the purpose of documenting the program context, program organization and staffing, the JSA service components, and other relevant aspects of the TANF program. During the visits, site teams will interview key administrators and line staff using a semi-structured interview guide.
JSA staff survey. This on-line survey, administered to TANF management and line staff involved in JSA activities, will be used as part of the implementation study to systematically document program operations and the type of JSA services provided across the study sites.
The fourth data collection will be a follow-up survey of study sample members, which ACF intends to elect in FY15.1 Once ACF has elected the option, we will submit a separate clearance request.
JSA programs are short-term, relatively low-intensity and low-cost programs to help job seekers find jobs. Some JSA programs focus on helping job seekers find jobs more quickly than they would on their own; some focus on helping job seekers find better jobs; and others focus on both. In conjunction with their income support role, state and local TANF programs typically provide some type of job search assistance services.
At a general level, there is evidence that JSA strategies are effective in increasing employment, but the impacts are modest. However, among the variety of ways in which JSA can be implemented, there is very little evidence regarding which strategies are more effective than others. For example, is providing JSA in a group setting with instruction on job search techniques more or less effective than when individuals search for jobs on their own? Are longer time commitments to search for a job more effective than shorter? Does adding one-on-one guidance or job development services that make direct connections with employers to self-directed and/or group JSA activities improve how fast participants find jobs? Given that JSA is an important TANF work activity in all virtually all states, ACF sponsored this evaluation to determine the relative effectiveness of various JSA approaches in a large-scale randomized trial across multiple sites.
The evaluation will address the following principal research questions:
What are the differential impacts of TANF JSA program approaches and components on short-term participant employment and earnings outcomes?
How do specific JSA program design features affect primary impacts?
In addition, the evaluation will explore the following secondary research questions:
What is the impact of particular JSA program approaches and components on the following: (a) public benefits received; (b) job retention; (c) job quality (including work-related benefits, consistency and predictability of hours); (d) family economic well-being; and (e) non-economic outcomes (including motivation to search for a job, perception of the job search process, and personal and family well-being)?
How do impacts vary by local economic conditions, participant characteristics, and over-arching TANF program philosophies?
The evaluation began with a field assessment to identify the contrasts of JSA program features that are of most interest to TANF policymakers and practitioners to test as part of the evaluation, and identify sites for the evaluation.2 As of the date of submission, August 11, 2014, this activity is winding down.
Once the tests are identified and the sites selected, the JSA evaluation will use an experimental design to determine the effectiveness of contrasting approaches of JSA. For the evaluation, TANF recipients will be randomly assigned to one of at least two JSA approaches to measure the “value-added” of more intensive approaches. The evaluation will not include a true “no services” control group. The evaluation consists of two key components:
The impact study will estimate the effects of contrasting JSA services on TANF recipients’ employment, earnings, and benefit receipt. The study will derive primary outcomes from existing data, including the date of new hire and quarterly earnings from the National Directory of New Hires (NDNH), and administrative data on receipt of TANF and Supplemental Nutrition Assistance Program (SNAP) benefits. ACF has the option to elect a follow-up survey for a subsample of study members to examine employment-related and other issues not available through administrative data including wages, job benefits, job retention, and family well-being. Should ACF choose to fund the survey, we will submit a separate OMB package for its approval. The impact study will also link variations in programmatic content and implementation factors to variation in program impacts, to determine what program features appear to most important in driving impacts.
The implementation study will document program operations and service delivery. The implementation study will document features and dimensions of the approaches under evaluation (e.g., type and mode of services provided, frequency of contacts between TANF recipients and program staff, sequencing of JSA among other TANF work activities) and program context.
While at the time of this submission we are still identifying the tests of interest, we assume that the evaluation will include approximately 10 sites, all implementing similar contrasting approaches to providing JSA services, with a total sample size of 25,000. A site is likely to be a locality with more than one TANF office. At this point, we have not identified the sites to be included in the evaluation.
There are no legal or administrative requirements that necessitate the collection.
Addressing the research questions adequately requires collecting detailed data from multiple sources. We are requesting approval for three data collection activities:
Baseline data collection (Attachment A). The baseline data covered by this clearance will collect basic identification, demographic, education, employment and income, and contact information for study participants. The form includes standard items used in prior ACF evaluations. It will enable the research team to: (1) describe the characteristics of study participants at the time they are randomly assigned to one of the experimental groups; (2) ensure that random assignment was conducted properly; (3) create subgroups for the analysis; (4) provide contact information to locate individuals for potential follow-up surveys; and (5) improve the precision of the impact estimates.
Implementation study site visits (Attachment B). The primary purpose of these visits will be to document differences in services and practice within and across contrasting JSA approaches. This activity involves conducting site visits for the purpose of documenting the program context, program organization and staffing, the JSA service components (e.g., assessment, use of self-directed job search, group job search, one-on-one assistance, job development), sequencing and flow of activities, and other relevant aspects of the TANF program (e.g., sanction policy, economic and community context). A second purpose for the site visits will be site monitoring to assess whether distinctions between contrasting approaches are being maintained.
JSA staff survey (Attachments C, D, E and F). The on-line survey of TANF supervisors and line staff involved in the provision of JSA services will be used to collect data on JSA services and other aspects of the TANF program systematically across the study sites. This survey data will be used for two major purposes: (1) to advance the documentation of each JSA approach under study and (2) to derive many of the measures that will used in the impact analysis to link program characteristics and implementation factors to program effects.
These data are not available through any current source, as described further in section A.4. Approval for a follow-up survey will be requested separately if ACF funds the survey.
The JSA evaluation includes three data collection instruments relevant to this OMB package – the baseline information form, instruments used for the implementation study site visits, and the JSA staff survey. Each of these activities is described below.
All persons who apply for TANF, are determined eligible, and are assigned to JSA services will be told about the study (including random assignment) and asked to sign an informed consent form confirming that they have been informed about and understand the study and agree to participate. If an individual does not consent, they will have access to the standard set of JSA services and will not be included in the study. That said, our experience with other similar studies suggests that most people will agree to participate since there is minimal risk to doing so. The JSA informed consent form is shown in Attachment G.
Data from the baseline information form will be entered into a secured web-based system, known as the Participant Tracking System (PTS) that will compile data on study sample members and conduct random assignment. The discussion below addresses those baseline elements to be collected:
Identifying information. This includes complete name, address, telephone number, email, birth date, gender, and social security number (SSN)—enough information to ensure that each individual is randomly assigned only once. This information is also necessary for tracking and locating sample members should we conduct the follow-up survey and for ensuring that we can obtain and accurately match administrative records on sample members. We will have strong data security protections to ensure this data remains confidential, as required by our Institutional Review Board (IRB).
Demographic characteristics and education and employment history. Baseline data in these areas are required to ensure that the random assignment process was conducted properly (by confirming that the research groups have similar characteristics at baseline) and to monitor random assignment. We will also use the information to describe the study sample and to document differences in the populations served across the study sites, as well as to understand factors that could affect their ability to find and keep jobs. This information will allow us to conduct subsequent analyses of subgroups.
Locating information. Accurate locating information is crucial to achieving high survey response rates, which will be important when ACF elects the follow-up survey. The baseline information form will capture each applicant’s landline and cellular telephone numbers and email address. Because reaching individuals via landlines is difficult in an era of heavy cell phone use and use of social media, the form will also capture Facebook name/permission to contact the individual on Facebook. We plan to use Facebook to send private messages to study participants if they are unreachable via email or phone. Abt is also collecting Facebook name for the IRB-approved Green Jobs and Health Care Impact Evaluation that is being conducted for the U.S. Department of Labor (OMB No. 1205-0481NOA). The baseline information form also asks permission to send a text message the individual’s cell phone, which can greatly assist in tracking and administering the follow-up survey. Finally, we will also collect alternative contact information for up to three relatives or friends who might know how to contact the study participant.
Implementation Study Site Visits
A rigorous random assignment evaluation requires clear and specific documentation of the services provided under the contrasting JSA approaches. This qualitative information will enable the evaluation team to describe the services and operations for each approach, interpret the impact analysis results, and identify lessons learned for the purpose of program replication. As part of the implementation study, at each site in the evaluation, we will conduct semi-structured interviews with state and local TANF administrators and staff involved with the job search process including TANF eligibility workers, case managers, job developers, data specialists, job search workshop facilitators, and other appropriate staff.
Site visits will be conducted to two TANF offices in each site, or 20 local TANF offices across the 10 sites. An average of 15 staff per TANF office will be interviewed during these site visits, for 300 total interviews. See Attachment B for the interview study guide and information on which topics will be covered for each type of respondent.
A team of two experienced evaluators will visit each site approximately four months after the start of random assignment. Prior to the site visits, site visitors will be trained to ensure a common understanding of the key objectives and the study protocols. The team will conduct semi-structured interviews with individual program staff and administrators in a private office or room on-site following established procedures for maintaining strict individual privacy. Notes from the interview will be handwritten or entered into a password-protected laptop computer. After each visit, the field notes will be stripped of any identifying information to guard against any violations of privacy provisions. Notes will be stored in a secure computer or file cabinet at Abt Associates or Mathematic Policy Research that can only be accessed by the evaluation team.
As part of the implementation study, we will administer a staff survey to all TANF supervisory and line staff involved in JSA activities to collect systematic data on the services provided. Line staff and their supervisors who are involved in providing JSA services and monitoring and reporting on their participation will be included. The staff survey will collect data on staff and organizational background characteristics, type of job search assistance provided (including content and frequency of services), staff responsibilities, staff attitudes and perspectives on program services, and barriers faced by TANF recipients. This data will be collected via a web survey. Supervisors and line staff will be asked questions in the same domains, although the questions are worded slightly differently to reflect their different level of responsibilities. Thus, there are two versions of the JSA staff survey, one for supervisors and one for line staff. See Attachments C and D for the line staff survey instrument and staff survey screen shots and Attachments E and F for the supervisor survey instrument and supervisor survey screen shots.
Each site will identify a study liaison during the site agreement process who will work closely with a liaison from the research team throughout the course of the study. The site liaison will work with the research team liaison to develop a list of staff that could potentially, based on their roles and functions, be asked to respond to the on-line survey. We expect that we will be able to conduct the survey as a census of all staff involved in the delivery and management of JSA services in each site. Our current assumption for the survey is that it will be administered to 600 line staff and 60 managers across the 10 sites
The primary beneficiaries of this planned data collection effort will be ACF, other federal agencies, TANF program operators, state and local policymakers, and other policy makers and researchers. ACF will use the information to determine what job search assistance strategies are most effective in moving TANF recipients to work and increasing their earnings. This will be important information in guiding the operation of state and local TANF programs. Secondary beneficiaries of this data collection will be those in the public policy and social science research community who are interested in further understanding which JSA services most help TANF recipients find employment, get better jobs and/or hold jobs longer, and reduce TANF receipt.
The Participant Tracking System (PTS) is a web-based system for gathering background information on study participants, as well as for executing random assignment for the evaluation. Line staff will enter only minimal identifying information into the PTS needed to conduct random assignment. The remainder of the baseline data will be entered into the PTS by the research team. The drop-down menus and response categories further minimize data entry burden, and the research team will train staff on how to use the system.
The JSA staff survey will be hosted on the Internet via a live secure web-link. This approach is particularly well-suited to the needs of these surveys in that respondents can easily stop and start if they are interrupted and review and/or modify responses in previous sections. To reduce burden, the staff survey will employ the following: (1) secure log-ins and passwords so that respondents can save and complete the survey in multiple sessions; (2) drop-down response categories so that respondents can quickly select from a list; (3) dynamic questions and automated skip patterns so that respondents are only shown those questions that apply to them (including those based on answers provided previously in the survey); and (4) logical rules for responses so that respondents’ answers are restricted to those intended by the question.
The information collection will not duplicate information that is already available. Strategies to identify and avoid duplication are discussed in the following two subsections. The first covers the baseline data collection and the second covers the staff survey and site visit data collection efforts.
The information collected on the baseline information form is not available consistently across states. The research team’s prior experience evaluating similar populations suggests that types and quality of data collected vary significantly across states, making it difficult to document the characteristics of the sample, in terms of education and employment history, across all study sites. Additionally, many states do not collect alternative contact information for TANF recipients, which will be important for the JSA Evaluation when ACF funds the participant follow-up participant survey.
The data to be collected from the staff survey and during the site visits are not available from any other source. There is no other data source providing detailed information on program staff backgrounds; nature and content of JSA services, including staff roles and responsibilities; or attitudes about the TANF program.
The data collection does not involve small businesses or other small entities.
All of the data to be collected associated with this specific burden request are one-time in nature. The data collection effort described in this document are designed to provide unique information to answer questions of interest to policymakers.
Without collecting baseline information on study participants, the study could not implement random assignment correctly or ensure that it had been conducted appropriately. The lack of baseline information would limit the ability to describe the TANF recipients involved in the study and compromise the analysis of impacts of the services on subgroups, hence limiting the ability to determine the groups for which JSA approaches are most effective. Without baseline data, impact estimates would be less precise (so that small impacts would be less likely to be detected), and adjustments for nonresponse to the follow-up surveys would have to be based on less-detailed administrative data. Further, if detailed contact information for study participants is not collected on the baseline information form, it would be more difficult to locate participants to administer the participant follow-up survey. This would likely lead to a higher nonresponse rate and thus pose a greater risk to the quality of survey data and, in turn, the impact estimates.
The information collected through the JSA staff survey and implementation study site visits will enable the team to describe the program design and operations in each site, interpret the impact analysis results, provide data for analysis to link program characteristics to program impacts, and identify lessons learned for purposes of program replication. The consequences of not collecting this information would be a lack of in-depth information about the nature of the JSA strategies developed and employed at participating study sites, and the types of services that produced any observed impacts. The staff survey and site visits will both provide an opportunity to document the JSA services being delivered.
There are no special circumstances for the proposed data collection.
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995)), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on June 6, 2014, Volume 79, Number 112, page 33559, and provided a 60-day period for public comment. A copy of this notice is included as Attachment H. During the notice and comment period, the government received one request for materials and no comments.
Experts in their respective fields from ACF, Abt, and Mathematica Policy Research listed below were consulted in developing the design, the data collection plan, and the materials for which clearance is requested (the baseline information form, the site visit protocol, and the staff survey).
Ms. Erica Zielewski
Mr. Mark Fucello
Ms. Naomi Goldstein
Ms. Nicole Constance
Ms. Karin Martinson
Dr. Stephen Bell
Dr. Laura Peck
Mr. Jacob Klerman
Mr. Howard Rolston
Ms. Karen Gardiner
Ms. Michelle Derr
Ms. Alicia Meckstroth
There will be no payments for completing any of the three data collections: the baseline information form for participants, interviews of agency staff conducted during site visits, and the JSA staff survey. We will describe the tokens of appreciation associated with the fourth data collection activity – the follow-up survey of study sample members – in a separate submission, should ACF elect to undertake the survey.
Abt and Mathematica are very cognizant of federal, state, and DOL data security requirements. All Abt and Mathematica study staff will comply with relevant policies related to secure data collection, data storage and access, and data dissemination and analysis.
Every effort will be made to maintain the privacy of respondents to the extent permitted by law. All respondents included in the study will be told that information they provide will be used only for the purpose of this research. Individuals will not be cited by name (or other identifying information) as sources of information in prepared reports. All research staff working with personally identifiable information (PII) will be trained to protect private information and will sign a pledge stating that they will keep all information gathered private to the extent permissible by law. All papers that contain participant names or other identifying information will be kept in locked areas and any computer documents containing identifying information will be protected with a password.
The implementation study site visit interviews and JSA staff survey are purely voluntary. Respondents will be told that all of their responses will be kept private, their names will not appear in any written reports, and that responses to the questions are voluntary.
The research team will take the following specific measures to protect respondents’ privacy:
Using rigorous security measures for baseline data. Abt and Mathematica have established safeguards that provide for the confidentiality of data and the protection of the privacy of the sampled individuals on all of its studies. At baseline, all information on individuals will be entered into the PTS. The web application will reside on a Microsoft Windows server running an IIS web server that will be physically located at MaximumASP, a monitored access-controlled secure data center. The web server has been hardened using a best-practices security hardening checklist (NIST). Administrator access to the database server will be restricted to an authorized Abt Associates server administrator and the system developer. Accounts on the web server will be protected with passwords that are at least eight characters long, that contain at least one special character and number, and that will not contain dictionary words. These requirements are enforced upon account creation. Passwords will expire every 90 days, and users will have to create new passwords that fulfill the requirement of the password policy. All sensitive data will be encrypted and protected by Secure Socket Layer (SSL). Logging or output files will not contain confidential data and will be limited to generic system and error data. Furthermore, data extracts for use by the project team will be stored on the server in an encrypted form and provided to project team members via the File Transfer Protocol over SSL (FTPS) site.
Using web-based surveys. Administering the on-line survey (for program staff) via web eliminates security risks related to shipping hard-copy forms containing PII to the evaluator.
Restricting access to the study network folder. Any data collected that contains PII for the JSA evaluation will be housed on secure servers. Access to the study network will be restricted by assigning a password to each relevant staff member.
In addition to these study-specific procedures, the evaluator has extensive corporate administrative and security systems to prevent the unauthorized release of personal records, including state-of-the-art hardware and software for encryption that meets federal standards and other methods of data protection (e.g., requirements for regular password updating), as well as physical security that includes limited key card access and locked data storage areas.
The baseline information form will collect background information on participants who have consented to participate in this evaluation (see section A.2.1).Contact information for study participants and alternative contacts (e.g., address, telephone numbers) is needed to identify and contact participants. It is important to collect this information in order to keep in touch with study participants between random assignment and the follow-up survey date so that contact information is current. The baseline information form also collects information on characteristics of participants, such as age (date of birth), sex, race/ethnicity, marital status, education level, and employment history—data used to ensure that random assignment was conducted correctly, to create subgroups for the analysis, and to enhance the impact estimates. This type of information is collected as part of enrollment in most programs and is therefore not considered sensitive. Date of birth, along with name, will also be used to avoid duplication of random assignment. Since it is unlikely that two individuals will share a first and last name as well as date of birth, the PTS will use these fields to check whether anyone with the same name and date of birth has already been entered into the system. This will prevent individuals from being randomly assigned more than once.
The baseline information form will also collect Social Security Numbers (SSN). SSNs will be used so that the researchers can match to critical administrative data—NDNH records and data from public assistance systems. Records from NDNH will be used to measure the primary outcomes of interest to the evaluation: the impact of the programs on time to employment and earnings from work. Data from state public assistance systems will be used to measure the secondary outcomes of impact of the programs on receipt of TANF and SNAP benefits. The only way to accurately access an individual’s data is through their SSN; other identifiers such as name and date of birth are not unique enough to ensure that the correct data will be obtained. In particular, NDNH only allows matches by SSN. Thus, without collecting SSNs, the evaluation could not examine employment and earnings through administrative records, and would have to rely on more expensive methods (i.e. surveys).
No questions of a sensitive nature will be asked of agency staff in either semi-structured interviews during the onsite visits or the JSA staff survey.
The time burden for administering the baseline information form is estimated to be 12 minutes for each study participant. This estimate is based on the number of items on the form, as well as previous experience on numerous other studies conducted by Abt Associates using a similar items and forms. Other studies using similar items and forms upon which this time estimate are based include the Innovative Strategies for Increasing Self-Sufficiency Evaluation and the Health Professions Opportunity Grants Impact Evaluation both conducted for ACF and the Green Jobs and Health Care Impact Evaluation conducted for the U.S. Department of Labor.
Our current assumption for the JSA survey is that it will be administered to the universe of JSA staff in each site, estimated to be 600 line staff and 60 managers across the 10 sites (or 100 line staff and six managers in each site). The average estimated time of the staff survey is 30 minutes based on results from a pre-test of the survey (discussed in more detail in Section B.4).
During site visits, state and local TANF administrators and other staff will participate in semi-structured interviews. We plan to conduct interviews with 30 respondents in each of the 10 sites, including key administrators and line staff, for a total of 300 interviews across all sites. The time per response is estimated at 60 minutes (1 hour).
Exhibit A-1 presents the reporting burden on study respondents completing the instruments included in this data collection request and their total cost.
We calculated the average hourly wage for each respondent group based on information from the Bureau of Labor Statistics3 or the federal minimum wage. We calculated the average hourly rate4 for each respondent group using the following categories:
Study participant: the minimum hourly wage ($7.25) plus a 40 percent adjustment to account for benefits, or $10.15 per hour.
Community and Social Service Occupations (SOC 21-0000): wage rate of $21.50 plus a 40 percent adjustment for benefits, or $30.01.
Social and Community Service Manager Occupations (SOC 11-9151): wage rate of $31.61, plus a 40 percent adjustment for benefits, or $44.25.
When members of a respondent group come from multiple job categories, we took an average across the relevant categories, as noted.
Exhibit A-1: Annual Information Collection Activities and Cost
Instrument |
Total Number of Respondents |
Number of Responses Per Respondent |
Average Burden Hours Per Response |
Total Burden Hours |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Baseline Information Form |
25,000 |
1 |
0.2 |
5,000 |
5,000 |
$10.15 |
$50,750.00 |
Staff Survey |
660 |
1 |
0.5 |
330 |
330 |
$37.136 |
$12,252.90 |
Semi-structured Interview |
300 |
1 |
1.00 |
300 |
300 |
$37.135 |
$11,139.00 |
Total Burden Hours: |
5,630 |
5,630 |
|
$74,141.90 |
|||
This data collection effort involves no recordkeeping or reporting costs for respondents other than those described in Exhibit A-1 above.
The total cost for these data collection activities will be $1,517,138. This includes the cost of initial information collection from the field, developing and pretesting data collection instruments and tools, administering the surveys and interviews, and analyzing implementation study data. Data collection activities will occur over a four-year period. Annual costs to the Federal government will be $379,285 for the proposed data collection.
This evaluation involves new data collection that increases the public reporting burden under this OMB number. Section A.12 documents in the increase in burden figures.
Exhibit A-2 presents an overview of the project schedule for information collection. It also identifies publications associated with each major data collection activity. In addition to the reports listed in the table, we will also produce a final report in September 2017 that will document the methodology and results of the implementation and impact studies.
Exhibit A-2: Overview of Project Data Collection Schedule
Data Collection Activity |
Timing |
Associated Publications |
Baseline information form |
March 2015 – January 2016 |
Evaluation Implementation Memorandum (February 2016) |
Site Visits and semi-structured interviews with TANF program staff |
July 2015 – January 2016 |
Evaluation Implementation Memorandum (February 2016); Memorandum Summarizing Study (April 2016) |
JSA staff survey |
June 2015 – January 2016 |
Evaluation Implementation Memorandum (February 2016); Memorandum Summarizing Study (April 2016) |
All instruments created for the JSA Evaluation will display the OMB approval number and the expiration date for OMB approval.
No exceptions are necessary for this information collection.
Bell, S.H., & L.R. Peck (2013). Using Symmetric Predication of Endogenous Subgroups for
Causal Inferences about Program Effects under Robust Assumptions: Part Two of a Method
Note in Three Parts. American Journal of Evaluation, 34(2). DOI: 10.1177/1098214013489338
Bloom, Howard S., Carolyn J. Hill & James A. Riccio. (2003). “Linking Program
Implementation and Effectiveness: Lessons from o Pooled Sample of Welfare-to-Work
Experiments.” Journal of Policy Analysis and Management, 22(4), 551-575. DOI:
10.t0O2/pam.tO154
Harvill, E., S.H. Bell, & L.R. Peck (2012). On Overfitting in Analysis of Symmetrically-
Predicted Endogenous Subgroups from Randomized Experiment Samples. Presented at the
Annual Fall Research Conference of the Association for Public Policy Analysis and
Management, Baltimore, MD.
Peck, L.R. (2003). Subgroup Analysis in Social Experiments: Measuring Program Impacts Based
on Post Treatment Choice. American Journal of Evaluation, 24(2), 157–187. DOI:
10.1016/S1098-2140(03)00031-6
Peck, L.R. (2013). On Analysis of Symmetrically-Predicted Endogenous Subgroups: Part One of
a Method Note in Three Parts. American Journal of Evaluation, 34(2), 225–236. DOI:
10.1177/1098214013481666
1 We proposed a follow-up survey of study participants to OPRE as an optional task under this evaluation. ACF intends to fund the survey in FY15. Once OPRE elects this option, we will submit the survey to OMB for approval.
2 This field assessment includes semi-structured interviews with state and local TANF administrators, program staff who provide JSA services, and researchers and policy experts. Discussion guides for these information-gathering interviews were approved under OMB clearance number 0970-0440.
3 http://www.bls.gov/oes/current/oes_nat.htm
4 Assuming 2080 FTE hours worked.
5 Source: Bureau of Labor Statistics, National Compensation Survey, 2011: Combined average hourly wage of Community and Social Service Occupations and Social and Community Service Manager Occupations
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-27 |