NSIDS Pilot Test Memo

Introduction

The Bureau of Justice Statistics (BJS) is planning to conduct a census of all indigent defense systems (i.e., systems providing criminal legal defense to the poor) within the United States through a project titled the National Survey of Indigent Defense Systems (NSIDS). The project will examine the methods in which states with centralized indigent defense systems and counties in states with localized indigent defense systems employ public defenders, assigned counsel, and contract attorneys to provide criminal representation to the indigent. Specifically, the NSIDS will examine the fiscal resources and expenditures allotted to indigent defense offices; the case-types and caseloads handled by indigent defense systems; the procedures for handling situations involving conflicts of interest and case overload; the staffing and compensation of indigent defense attorneys; the procedures for indigence determination; the professional development opportunities provided to defense attorneys; the adherence to formal standards and guidelines; and the utilization of oversight boards/commissions among the different types of indigent defense.

The NSIDS will be the fifth study sponsored by BJS to examine indigent defense systems in the U.S. Earlier studies focused on collecting indigent defense information from a national sample of counties and the entirety of states with centralized indigent defense systems.¹ BJS’ last indigent defense study took place in 2007 and involved a census of all state and county-based public defender offices. Each of these studies has attempted to broadened the population of indigent defense systems covered (e.g., expanding the survey to include all public defender offices) and enhance the content of subject areas examined (e.g., drawing upon the American Bar Association’s ten principles as a means for assessing the public defender data). BJS’ upcoming NSIDS will constitute a substantial leap forward from earlier studies because it involves a census of all forms of indigent defense delivery methods including public defenders, assigned counsel, and contract attorneys in operation throughout the U.S. As a result of this census, BJS will be able to provide information about the different indigent defense delivery methods employed by all 50 states and the District of Colombia.

BJS initiated this census in order to address how criminal defense is administered throughout the U.S. Indigent defense delivery systems typically lag far behind the other components of the criminal justice system when it comes to the ability to form policy based on objective information. With scarce resources and ever-growing caseloads, most right to counsel systems dedicate almost every funding dollar to direct client services rather than producing data to inform higher level decision-making. In addition to a lack of information on core program characteristics, there is growing evidence that many indigent defense systems are being overwhelmed by budget cutbacks as a result of the economic downturn. The lack of a statistical series capable of examining the totality of the nation’s indigent defense systems and the growing evidence that these systems are under increasing budgetary and other pressures, has produced a situation where there is a clear need for detailed national information on the structures, resources, staffing, budgets, and policies of indigent systems. These issues are crucial to examine because they ultimately affect how justice is administered to those who cannot afford legal counsel and have major impacts on the concept of equality of justice in America’s court systems.

The need to collect more detailed and comprehensive information about the nation’s indigent defense systems has also been acknowledged by a number of federal officials including Attorney General Eric Holder. In addition, the importance of indigent defense has been emphasized by the U.S. Department of Justice which initiated an Access to Justice (ATJ) initiative in order to improve justice delivery systems for people who are unable to afford lawyers. ATJ has been at the forefront in advocating the need for indigent defense systems to receive more resources, opportunities for best practices training, and augmented research funds and initiatives. Other non-profit organizations including the American Bar Association, the National Association of Criminal Defense Lawyers, and the National Legal Aid and Defender Association have stressed the need to collect national level indigent defense data.

Request for Developmental Work

The Bureau of Justice Statistics (BJS) plans to conduct developmental research for the NSIDS project under the generic clearance (OMB # 1121-0339). On behalf of BJS, project staff from the NORC at the University of Chicago will pilot test the NSIDS survey instrument in 39 indigent defense systems and engage in follow-up interviews that will debrief at least 20 survey respondents about their experiences completing the survey instrument.

The nature of indigent defense systems in the U.S. varies dramatically across and within states. In some states, the system is a state function, with common service delivery models in each county. In other states, the delivery methods varies across the counties and employs various combinations of government and non-government public defenders, assigned counsels, and contract attorneys. As part of our current project design efforts, we have developed six prototype indigent defense systems [further described in this document] that need to be covered in the pilot test. Each of these service delivery models presents different challenges to the NSIDS project as a data collection protocol that works well in one environment may not work well in another. While BJS has attempted to design an instrument that should function well in each of the six service delivery systems, this assumption must be tested. This request for developmental research under BJS’ generic clearance seeks permission to field test the draft NSIDS instrument in a sufficient number of jurisdictions in each of the six modes of service delivery to assess the instrument’s ability to function well within each.

The pilot tests of the NSIDS survey instrument, with follow-up interviews will provide an estimate of the burden hours for jurisdictions in each of the six delivery modes. In general, the (conceptually) six pilot tests will provide data on the time required to (1) identify the points of contact (POCs) with responsibility for completing the survey instrument, (2) select respondents to complete the survey instrument, and 3) determine the amount of time required to complete the survey instrument. The follow-up interviews will provide qualitative assessments of how well the instrument’s items function in jurisdictions with different delivery modes.

This approach will allow BJS to utilize the pilot test for the purpose of calculating burden estimates and assessing the functionality of the survey instrument. In particular, it will allow BJS to pilot test the project in enough jurisdictions where indigent defense information should be readily accessible (those with centralized state-based systems), while also conducting the pilot in an appropriate number of county-based sites where the information being sought could be difficult obtain. The pretest will also serve to inform BJS about the capacity of respondents to complete each question on the survey instrument, the work required to gather all necessary information, the sections that are unclear or involve inordinate amounts of time to complete, and the questions where problems of nonresponse or missing information might occur. Moreover, the pretest will allow the data collection agent for this project to test the functionality of the online survey tool including protocols for identifying missing or inconsistent responses. Lastly, the follow-up interviews will further enhance the pilot by providing respondents with a forum for reviewing and evaluating each question for its content, clarity, and accuracy.

In some research, the goals of the pilot study described above could be achieved by contacting nine respondents. However, an appropriate pilot test for the NSIDS will require collecting more sites. NSIDS is designed to survey every jurisdiction in the nation (i.e., state and county) that provides indigent defense services. The normal pre-test protocol for such a survey instrument is insufficient in the NSIDS effort because of the inherent differences across the population of respondents. By limiting the pre-test to nine respondents, the pilot would only be able to send the survey to one or two jurisdictions with their unique service delivery configuration, which is not a sufficient test of the instrument for that class of respondent.

A successful NSIDS pilot test must take into account the varied configurations by which states and localities administer their indigent defense systems. In general, indigent defense is either administered in states with centralized systems or operates in states where its administration occurs at the local or county level. At a secondary level, both state and county-based systems differ in terms of how they deliver their primary and conflict case representation to indigent defendants. For example, some states and counties rely solely on public defenders, while others employ a mixture of public defenders, assigned counsel, and contract attorneys to represent indigent defendants. In order to adequately pre-test the instrument within each of the typologically different forms of indigent defense service delivery, BJS has determined that the NSIDS pilot should contact 39 jurisdictions, including 4 states where indigent defense is centrally organized and 35 counties in states where indigent defense is administered at the county-level.

Initially, we considered conducting the pre-test in 4 state-based and 5 county-based indigent defense systems. Conducting the pre-test in only 4 state-based systems was sufficient for the pilot because of BJS’ prior experiences collecting indigent defense data from states with centralized indigent defense offices that employ primarily public defenders. Had all county-based systems employed uniform methods of delivering their indigent defense services, the 5 county-based sites might have been sufficient for the pre-test; however, given the divergent structure of indigent defense in the county-based systems, it was necessary to conduct the pretest in enough jurisdictions to produce reliable and useful information for each type of indigent defense delivery system. Among the 4 county-based configurations, those counties using only public defenders are similar to the state-based systems and hence did not require more than 5 sites for the pretest. In the remaining county-based systems, the methods for delivering indigent defense services differed sufficiently enough from the state-based systems, and from each other, that we decided to conduct the pretest in 10 sites per category. Ten sites were chosen because it adheres closely to the OMB mandate that 9 sites constitute an optional number of pretest sites for each county-based indigent defense category, while including an additional site to account for the possibility of non-response. The breakdown of the types of delivery systems, their approximate number in the U.S., and the number to be contacted in the pilot are displayed in the following table:

Type of Indigent Defense Configuration	Number of indigent defense systems	Number included in pilot
State-based indigent defense systems	29 state-based indigent defense systems located in 28 states and the District of Columbia	4 state-based systems
State based indigent defense office (typically public defender) responsible for both primary and conflict defense services	19 indigent defense systems; 1 respondent per indigent defense system	1 indigent defense system
State based indigent defense office (typically public defender) responsible for primary defense services only; separate office oversees conflict services involving assigned counsel and/or contract attorneys on statewide basis	10 indigent defense systems; 2-3 respondents per indigent defense system (1 chief executive overseeing primary system and 1-2 administrators overseeing assigned counsel/contract attorneys	3 indigent defense systems
County-based indigent defense systems	1,533 county-based indigent defense systems located in 22 states/A	35 county-based systems
County based public defender office responsible for both primary and conflict defense services	1 respondent per indigent defense system	5 indigent defense systems
County based public defender office responsible for primary defense services only; separate office oversees conflict services involving assigned counsel and/or contract attorneys	2-3 respondents per indigent defense system (depending on the number of delivery methods utilized)	10 indigent defense systems
Non-governmental public defender office (501-c3) responsible for primary defense services; separate office oversees conflict services involving assigned counsel and/or contract attorneys	2-3 respondents per indigent defense system (depending on the number of delivery methods utilized)	10 indigent defense systems
Contract attorneys responsible for both primary and conflict defense services	1 respondent per indigent defense system – respondent will most likely be local chief trial court justice or county administrator	10 indigent defense systems
Total	29 state-based indigent defense systems + 1,533 county-based systems = 1,562 total indigent defense systems to be surveyed for census	39 total indigent defense systems included in pilot
A/Please note that the number of county-based indigent defense systems for each service delivery method is not listed because it is unknown at this point.

The pilot will be implemented through several steps. First, it will be necessary to identify the points of contact (POC) with responsibility for completing the pilot test survey questionnaire and follow-up interviews. Identifying the POCs in the 4 state-based systems should be relatively straightforward as that person will most likely be the chief public defender, executive, or commission administrator or someone tasked by that official to complete the survey instrument for the entire state. Ascertaining the POCs for the 35 county-based indigent defense systems may be problematic because these counties will rely on a mixture of public defenders, assigned counsel, and contract attorneys to administer their indigent defense systems. Moreover, the administration of indigent defense in some jurisdictions might be a function of county government or trial court oversight rather than public defender attorneys. BJS will rely on NORC and its partners with expertise in the administration of indigent defense including the National Legal Aid and Defender Association and the National Association of Counties to identify the POCs.

Once the POCs are identified, an invitation letter (see Attachment A) will be mailed to each indigent defense system that has been selected for the pilot. The letter will explain the purpose of the pilot and request their participation. It will provide the URL (or link) to the web survey and the indigent defense system’s unique PIN and Password for logging on to survey. It will also contain the toll-free number and e-mail address for the project so that respondents may contact BJS or its data collection agent (NORC) with questions or for assistance. The survey instrument for the pilot will be provided online and web-based data collection will be the method of completing this pilot (see Attachment B).

Approximately one week after the invitation letter mailing, NORC will send a thank-you/reminder postcard to all pilot participants. This postcard will thank those respondents who completed the survey and encourage those who haven’t to complete the online questionnaire (see Attachment C). NORC data collection specialists will begin telephone prompting non-respondents approximately one month after the thank-you/reminder correspondence. This will serve to verify receipt of materials, answer questions, determine and attempt to resolve potential problems with timely submission, and prompt for questionnaire completion (see Attachment D). Toward the end of the pilot data collection period, NORC will send a “Last Chance Postcard” that indicates the deadline for the pilot survey and pushes for their completion (see Attachment E).

In addition to pilot-testing the survey instrument in 39 jurisdictions, 20 of the pilot participants will be selected for a telephone debriefing to discuss their experiences completing the survey. Participants will be selected based on the pattern of responses received to the pilot survey and their answers to the “Survey Feedback” section of the pilot instrument. These indigent defense systems will be contacted for a follow-up interview upon receipt of their survey responses and be asked about their experience with the web survey, the availability of the requested information, and any recommendations for improving the questionnaire (i.e. question wording, response choices, etc.). A protocol of the debriefing questions is provided (see Attachment F).

All information gleaned from the pretest will be integrated into the full information clearance package that will be submitted to OMB by August 2013. BJS plans to fully implement the NSIDS by early 2014.

As the survey collects information about organizations and not individuals, NORC’s Institutional Review Board determined that the activities associated with data collection for this project are not considered human subjects research.

The estimated maximum time to conduct the pilot test for the NSIDS project is 156.5 hours. The burden hour estimates are divided across three pilot tasks including identification of points of contact (19.5 hours), pilot testing the survey instrument in 39 jurisdictions (117 hours), and conducting follow-up interviews for 20 jurisdictions (20 hours). The burden hour estimates are based on prior experience collecting similar nationwide data on public defender offices in 2007 (see OMB Control # 1121-0095). Further details about the burden hour estimates are provided in the following table.

Pilot tasks	Average burden per jurisdiction	Total estimated burden hours
Identifying points of contact to complete survey instrument	.5 hours	39 indigent defense systems * .5 hours = 19.5 hours
Pilot test survey instrument	3 hours	39 indigent defense systems * 3 hours = 117 hours
Conduct follow-up interview	1 hour	20 indigent defense systems selected for interview * 1 hour = 20 hours
Total Respondent Burden = 156.5 hours

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below.

Ronald Malega, Ph.D.
Statistician

Bureau of Justice Statistics
U.S. Department of Justice
810 7th Street NW, Room 2326
Washington, DC 20531
Office Phone: 202-353-0487
E-Mail: Ronald.Malega@usdoj.gov

Attachment A: Intial contact letter for pilot

Attachment B: Survey instrument for pilot

Attachment C: Thank you postcard for pilot

Attachment D: Telephone follow-up protocol

Attachment E: Last chance follow-up postcard

Attachment F: Protocol for debriefing respondents

1 BJS conducted studies of indigent defense systems in 1982, 1987, 1999, and 2007 (see OMB control number # 1121-0095 for more information).

4