Form 495 DOJ 495 DOJ 495 DOJ JAG Questionnaire Final 3-30-12

CFI Group Draft 3-27-12

Department of Justice

Bureau of Justice Assistance

Justice Assistance Grant Program

Customer Satisfaction Survey

Introduction

The Department of Justice, Bureau of Justice Assistance (BJA) is committed to providing you, our customers, with services that truly meet your needs. Gathering your feedback helps to ensure that we are delivering on our commitment to you. To this end, we have commissioned CFI Group, an independent third-party research group, to conduct a survey that asks about your current satisfaction with the Edward Byrne Memorial Justice Assistance Grant (JAG) program performance measures and provides an opportunity to provide suggestions for improvement.

CFI Group will treat all information you provide as confidential. All information you provide will be combined with information from other respondents for research and reporting purposes. Your individual responses will not be released. This brief survey will take approximately 15 minutes of your time.

This survey is authorized by the U.S. Office of Management and Budget Control No. 1090-0007.

Screening Question (ALL)

Demo1. Which of the following best describes your agency?

1. I am a primary JAG grantee of BJA’s with sub-recipient(s)

2. I am a primary JAG grantee of BJA’s with no sub-recipient(s)

3. I am a primary JAG grantee of BJA’s and a sub-recipient of another agency’s JAG funds

4. I am not a primary JAG grantee of BJA’s (terminate)

(If Demo1=3, ask Demo2)

Demo2. As a primary JAG grantee of BJA’s, do you also have sub-recipients?

1. Yes

2. No

Demographics

Demo3. (Ask if Demo1=1 or Demo2=1) How many sub-recipients do you have?

1. 1-5

2. 6-10

3. 11-25

4. 25 or more

Demo4. Do you or someone from your organization report in the BJA Performance Measurement Tool (PMT)? www.bjaperformancetools.org

1. Yes

2. No

Demo5. (Ask if Demo4=1 and (Demo1=1 or Demo2=1)) Are you also reporting in the PMT on behalf of your sub-recipients?

1. Yes

2. No

Demo6. (Ask if Demo5=1) What are your concerns with allowing your sub-recipients to report directly into the PMT? (check all that apply)

1. Timeliness

2. Data quality

3. Sufficient time for review of sub-recipient data

4. Compliance with reporting

5. Completeness of reporting

6. Other (please specify)

7. I have no concerns (programmer: no other selections with this choice)

Grant Monitoring

Thinking about the support and service that you received from BJA Staff, on a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent” please rate the BJA Staff on (If an item does not apply to you please select “N/A.”) (randomize):

PS1. Timely notification of on-site visit or Enhanced Programmatic Desk Review (EPDR)

PS2. Detailed instructions provided with notification for on-site visit or EPDR

PS3. Clarity of expectations for on-site visit or EPDR

PS4. Courtesy and professionalism of program staff during on-site visit or EPDR

PS5. Assistance received during on-site visit or EPDR

PS6. Overall usefulness of the on-site visit or EPDR

PS7. Ability of the program staff to answer your questions about grant policies, procedures, regulations and legislation during the on-site visit or EPDR

PS8. Ability of the program staff to answer questions about specific programs during on-site visit and EPDR

PS9. Ability of program staff to direct you to useful information that addresses your concerns during on-site visit or EPDR

Performance Measures

(Ask if Demo3=1) Now, thinking about the program performance measures, please rate the following on a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent” (if an item does not apply to you please select “N/A”):

PM1. Number of measures is manageable (not excessive)

PM3. Ease of collecting information for reporting on measures

(Ask if Demo3=1) Thinking specifically about managing and reporting program performance data, using a scale of 1 to 10, where 1 is “Strongly Disagree” and 10 is “Strongly Agree,” please indicate your level of agreement or disagreement with the following statements:

PM4. I am able to ensure reporting data is accurate

PM5. Enough time is provided for reporting

PM6. The frequency of reporting is appropriate

PM7. The measures apply to my organization

ACSI Benchmark Questions

1. First, please consider all your experiences to date with the BJA Justice Assistance Grant program’s post award service and performance measures. Using a 10-point scale on which “1” means “Very dissatisfied” and “10” means “Very satisfied,” how satisfied are you with the JAG program’s post award service and performance measures?

2. To what extent has the BJA Justice Assistance program’s post award service and performance measures fallen short of your expectations or exceeded your expectations? Please use a 10-point scale on which “1” now means “Falls short of your expectations” and “10” means “Exceeds your expectations.”

3. Forget about the BJA Justice Assistance Grant program’s service and performance measures for a moment. Now, imagine the ideal program’s post award service and performance measures to support and report your agency’s grant successes. How well do you think the BJA Justice Assistance Grant program’s post award service and performance measures compares with that ideal? Please use a 10-point scale on which “1” means “Not very close to the ideal” and “10” means “Very close to the ideal.”

Outcome Questions

OUT1. How confident are you that BJA does a good job administering the Justice Assistance Grant program performance measures. Use a 10-point scale on which “1” means “Not at all confident” and “10” means “Very confident.”

OUT2. How willing would you be to say positive things about BJA and the Justice Assistance program post award service and performance measures to colleagues or people from other organizations? Please use a 10-point scale on which “1” means “Not at all willing” and “10” means “Very willing.”

Additional Reporting Questions

For the next several questions, we would like your input on issues related to reporting on performance measures.

AR1. (Ask of all grantees) How do you ensure accurate reporting? (Please select all that apply) (randomize all but other)

1. Completing data collection prior to PMT reporting deadline

2. Spot check

3. Verify results on site visits

4. Make phone calls to validate data

5. Other (Specify ________________________)

AR2. Which of the resources below would be the most helpful to assist with your general reporting needs? (Please select only one item) (randomize all but other)

1. Frequently asked questions

2. Templates for individual level data collection for entry into the PMT in aggregate

3. Web forms

4. More reminders

5. Chat forums

6. Other (Specify)

AR3. If you could choose only one, what one resource do you need the most to assist with accurate reporting? (Please select only one item) (randomize all but other)

1. More technical assistance with data gathering

2. Clearer definitions of the questions and what should be reported

3. Additional technical assistance around data entry and/or the usage of the PMT

4. More time for data entry

5. Other (Specify ________________________)

AR4. What is your desired reporting frequency for performance measures?

1. Monthly

2. Quarterly

3. Semi-annually

4. Annually

AR5. How would you like BJA to share the data from the JAG performance measures? (Please select all that apply) (randomize all but other)

1. Publish an annual report

2. Display data on the BJA website

3. Distribute a newsletter which would include data reports

4. Display a map on the BJA website with the states’ usage of data

5. Email JAG results to grantees on a regular basis

6. Other (Specify)

AR6. How is the data you’ve reported into the PMT used? (Please select all that apply) (randomize punches 1-4, keep 5 and 6 in order at end)

1. It is reported to others outside my agency

2. It is used to make budgetary decisions

3. It is used to evaluate performance of the grant

4. It is reported internally

5. Other (Specify)

6. It is not used (Programmer: only punch that can be selected)

AR7. Do you collect performance data beyond BJA’s JAG performance measures?

1. Yes

2. No

AR8. Outside of what is reported to the BJA, what other activities are you interested in reporting on? (open)

AR9. Is there anything you would like to suggest to BJA that would make reporting easier?

Performance Measure Revision Questions

Need new link

PMR1. Have you reviewed BJA’s revisions to the JAG performance measures? http://www.ojp.usdoj.gov/BJA/grant/jag_measures.html

1. Yes (Skip to introduction text above PMR3)

2. No (continue with instructions below and ask PMR2)

Please take a few minutes to review BJA’s revisions to the JAG performance measures. The new measures were developed to provide grantees with the opportunity to better describe how JAG funding is being used and to describe program accomplishments. Your feedback on the revisions is very important to us.

Please right click on the link below to open a new browser window to review the revised JAG performance measures. Please take care not to replace the survey window with the new link as it may cause technical problems with returning to the survey.

When done with your review, please return to this survey browser window and select “Done” below to continue with the survey.

http://www.ojp.usdoj.gov/BJA/grant/jag_measures.html

(Programmer: new screen)

PMR2. Were you able to navigate the link and review BJA’s revisions to the JAG performance measures?

1. Yes (continue with programming note below)

2. No (skip to ADD1)

(Programmer: present introduction text on the next screen below:)

In the next brief section will ask a few questions to get your thoughts on BJA’s revisions to the JAG performance measures.

Using a scale of 1 to 10, where 1 is “Strongly Disagree” and 10 is “Strongly Agree,” please indicate your level of agreement or disagreement with the following statements. Reporting on the revised JAG measures will…:

PMR3. Make my reporting easier

PMR4. Decrease my workload

PMR5. Make it easier to collect accurate data

PMR6.Use a scale where 1 is “Little or No Benefit” and 10 is “Significant Benefit.” How much would your organization benefit by reporting on the new JAG measures?

PMR7. (Ask if Demo1=1 or Demo2=1) Will the revised measures change how your sub-recipients report?­­­­­­­­

1. Yes

2. No

3. Don’t know/not sure

PMR8. (Ask if Demo1=1 or Demo2=1) For your sub-recipients, how much time will you need to collect and report the data for the new performance measures?

1. Less than 2 weeks

2. 14 to 30 days

3. 31 to 45 days

4. More than 45 days

PMR9. How much lead time do you think you will need to prepare for changes to the measures?

1. Less than 2 months

2. 2 months to 3 months

3. More than 3 months to 6 months

4. More than 6 months

PMR10. (If AR7=yes) Earlier in the survey you mentioned that you collect performance data beyond BJA’s JAG performance measures. Will you continue to collect additional data after BJA changes the performance measures?

1. Yes

2. No

Training and Communications

Please consider a few questions about your training and communications needs

TR1. How will the new reporting requirements affect your training needs? (Select all that apply)

1. Frequency of training will increase

2. Different types of training (train-the-trainer sessions, webinars) will be needed

3. My agency will need to train additional people

4. Training materials will change

5. No change in training needs is anticipated

TR2. How would you like to receive training on the new reporting requirements from BJA? (Select all that apply)

1. Train-the-trainer sessions

2. In-person meetings

3. Webinars

4. Pre-recorded training to be watched at your convenience

5. Other specify:_____

TR3. Considering all agencies experience staff turnover, what resources would you like to see from BJA to assist you with continuity of performance measure and PMT knowledge? (open)

COMQ1. What type of information on the new reporting requirements do you want from BJA? (Select all that apply)

1. Catalog of what BJA is collecting and why

2. National level data/aggregate data

3. State level data

4. Other (Please specify___________________________)

COMQ2. How would you prefer BJA to communicate with you in regards to sharing information and answering questions about the new reporting requirements? (Select all that apply)

1. Email

2. Telephone/Conference Call

3. Website

4. Chat forum

5. Other (Please specify___________________________)

Additional Questions

ADD1. Would you be interested in serving on an advisory group to BJA on the performance measures?

1. Yes

2. No

ADD2. (Ask if ADD1=1) Thank you for your interest. As part of the advisory board to the BJA on performance measures, we will need your input on the feasibility for collecting the information, the burden, the clarity of the definitions, etc. If still interested, please provide your email address below, or send an email to JAPerformanceMeasurement@usdoj.gov

Your email address:_________________

ADD3. Would you be interested in serving on a technical advisory group to BJA on the usage of the PMT?

1. Yes

2. No

ADD4. (Ask if ADD3=1) Thank you for your interest. As part of the technical advisory board to the BJA, we will need your input on the functionality of the PMT, proposed improvements, and your help testing mockups. If still interested, please provide your email address below, or send an email to JAPerformanceMeasurement@usdoj.gov

Your email address:_________________

ADD5. Please let us know if you have any comments about BJA’s monitoring program to help us improve your support and service, and/or any efforts that you would like to recognize. (open)

ADD6. Do you have any additional comments you would like to share about BJA’s Justice Assistance Grant program? All comments are welcomed – overall or specific to any part of the program.