CFI 183 CNS Learn and Serve questionnaire

2008 Customer Satisfaction Questionnaire-LSA

Introduction

1. Hello, we are conducting research about the degree to which Learn and Serve America grantees are satisfied with the services provided by Learn and Serve America and the Corporation for National and Community Service (the Corporation). My name is ____________________ calling from ________ on behalf of Learn and Serve America. May I please speak with __________?
   1 Yes (Continue to INTRO2)
   2 Person not available (Schedule a callback)
   3 No such person (See below)
   99 Refusal/Hung Up (“Thank you and have a nice day!”)

[If “no such person”] The purpose of this research is to help improve services to your organization and others like it. Would it be possible for me to speak with your organization’s director? [capture name, continue with interview from INTRO1a if possible]

(Programmer instructions: Read when the person named in INTRO1 comes to the phone)

INTRO1a. Hello, we are conducting research about the degree to which Learn and Serve America grantees are satisfied with the services provided by Learn and Serve America and the Corporation for National and Community Service (the Corporation). My name is ____________________ calling from ________ on behalf of Learn and Serve America. The purpose of this research is to help improve the services provided to your organization and others like it. This survey is being conducted both by phone and on the Internet. Do you recall receiving an invitation to take the survey on-line?

1. Yes [skip to INTRO1a1]

1. No [skip to INTRO1a1.B]

INTRO1a1 Would you like to take the survey with me right now over the phone? The survey is relatively short.

1. Will take online [thank , inform them that they may periodically receive reminder emails or phone calls, and then terminate]

2. IF RESPONDED NO TO INTRO1A. Would like to take on-line [capture email address for invitation and thank]

3. Phone (skip to INTRO2)

4. Do not plan to participate [thank and terminate]

2. When responding, please answer on behalf of the collective experiences of your organization covering the last 12 months. If the survey asks about experiences that you are not that familiar with, please feel free to ask a colleague for assistance. We want and need to hear from you. In doing so, it is important that the survey is completed in its entirety.

Responses to this data collection will be used only for purposes of this research.  The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific organization or individual.  We will not provide information that identifies individuals to anyone outside the study team, except as required by law. This interview is authorized by Office of Management and Budget Control No. 1505-0191. Is this a good time?

1 Yes (Continue)
2 No “Can we schedule a time that is more convenient for you?”

Web Introduction

You recently received an electronic message from Learn and Serve America, inviting you to complete the 2008 Learn and Serve America Customer Satisfaction Survey. You can access the survey using the link listed below. Learn and Serve America and the Corporation for National and Community Service have utilized the survey since 2004 to better understand your experiences as a resource recipient of Learn and Serve America, particularly about your organization’s level of satisfaction with the program’s dynamics and operations.

Although participation is voluntary, your responses will help Learn and Serve America and the Corporation improve upon its services to your organization and others like it. The survey should take you approximately 10 minutes to complete.

When responding to the survey, please answer on behalf of the collective experiences of your organization covering the last 12 months. Please consider only your organization’s experience as a Learn and Serve America Grantee, and not with any other programs with which your organization may be affiliated. If the survey asks about an experience that you are not that familiar with--please feel free to ask a colleague for assistance. We are very pleased about this opportunity to collaborate with you on such an important matter.

Survey Instructions

Should you encounter any technical difficulties while taking the survey, please send an email to survey@cfigroup.com and you will be contacted with assistance. When moving through the survey please use the navigation buttons (below the survey text and questions) rather than your browser's "navigation bar". After entering your responses on each page, please click on the "NEXT" button at the bottom of the page until you reach the end of the questionnaire. If you have not experienced a particular area included in the survey, please enter your response in the last column under "DK/NA" for "Don't Know" or "Not Applicable." In order for your responses to be counted, please press the "Finish" button on the final page upon completion of the survey. If you would like to speak to someone directly at the Corporation for National and Community Service about this survey, please contact….

The survey is authorized by Office and Management and Budget Control No. ____. Every effort will be made to maintain the privacy and confidentiality of respondents.  Please know that your responses to this data collection will be used only for purposes of this research.  The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific organization or individual.  We will not provide information that identifies individuals or their grants and locations to anyone outside the study team, except as required by law. The confidentiality procedures adopted for this study during data collection, data processing, and analysis will consist of the items below:

•     All respondents will be assured that participation in the study is voluntary and the information they provide is confidential and will be used only for the purpose of conducting this research. 

•    To ensure data security, all individuals hired by the Corporation’s contractor, are held to strict standards and are required to sign an oath of confidentiality as a condition of employment.

•     Hard-copy data collection forms will be delivered to a locked area for receipt and processing.  The CFI Group maintains restricted access to all data preparation areas (receipt, coding, and data entry).  All data files on multi-user systems will be under the control of a database manager, with access limited to project staff on a “need-to-know” basis only.

•     Individuals identifying information will be maintained separately from completed data collection forms and from computerized data files used for analysis.  No respondent identifiers will be contained in public use files made available from the study, and no data will be released in a form that identifies individual corps staff, service providers, service participants, or other survey respondents.

Thank you in advance for your participation.

DEMOGRAPHIC QUESTIONS (Do not read)

Please consider only your organization’s experience as a Learn and Serve America grantee, and not with any other programs with which your organization may be affiliated.

DEM1 How long has your organization managed a Learn and Serve America Grant?

• Less than 6 months

• 6 months to 1 year

• One – less than two years

• Two – less than five years

• Five – ten years

• More than ten years

APPLICATION PROCESSES (Do not read)

Please consider only your organization’s experience as a Learn and Serve America grantee, and not with any other programs with which your organization may be affiliated.

We will begin by asking you about the process that organizations go through to renew their Learn and Serve America grants. For this series of questions, please think only of the application process itself. We will ask technology and eGrants related questions later in the survey. Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent,” please rate the application processes on the following items.

APP1 The ease of obtaining the application instructions and accessing the relevant forms

APP2 The timeliness of posting information about the application/continuation process

APP3 The clarity of the application instructions and proposal submission guidelines

APP4 The amount of time it takes to complete the application

APPLICATION REVIEW, APPROVAL, and AWARD PROCESS (Do not read)

Now please think about the multi-step process for reviewing grant applications and making awards. Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent,” please rate the process on the following items.

ARAP1 How well you were informed about the status of submitted applications

ARAP2 The reasonableness of grant renewal requirements

ARAP3 The fairness of the grant selection and review process

ARAP4 The timeliness of being notified of renewal decisions

ARAP5 The ease of the negotiation and revision process

ARAP6 The timeliness of receiving your grant award

LEARN AND SERVE AMERICA STAFF (Do not read)

Now please think about the Learn and Serve America staff with whom you have interacted as a Learn and Serve America grantee.

STF1 Does your organization interact with staff from the Learn and Serve America Program Office?

• Yes

• No (Skip to STF10)

STF1a. On average, how often do you interact with a Learn and Serve America staff member?

 

• Daily

• Weekly

• Monthly

• Annually

• Less than once a year

• Other

 

STF2 [If stf1=’Yes’] On average, how long does it take for the Learn and Serve America staff to respond to your phone calls and/or acknowledge receipt of your e-mails?

• Within 1 business day

• Within 3 business days

• Within 1 week

• Within 2 weeks

• More than 2 weeks

• N/A

Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent,” how would you rate the items below? If your organization is not familiar with a particular experience then please respond ‘N/A’.

STF4 The availability of Learn and Serve America staff

STF5 The Learn and Serve America staff’s knowledge of program requirements

STF6 The courteousness of Learn and Serve America staff

STF7 The timeliness of Learn and Serve America staff’s response to inquiries and concerns

STF8 Learn and Serve America staff’s sharing of information about effective practices

STF9 The overall effectiveness of Learn and Serve America staff in helping you meet your needs/requirements as a Learn and Serve America grantee.

STF9.1 The overall service delivered by Learn and Serve America staff to my organization

STF10 In the last 12 months, has your project been visited by a Learn and Serve America staff member?

• Yes

• No (Skip to GM1)

Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent,” how would you rate the items below?

STF11 The clarity of purpose for the site visit

STF12 The helpfulness of the feedback provided as a result of the site visit

STF13 The timeliness of feedback provided as a result of the site visit

GRANTS MANAGEMENT Staff (Do not read)

Now please think about your organization’s interactions with a Grants Officer as a Learn and Serve America grantee. (INTERVIEWER: THE GRANTS OFFICER COULD BE FROM THE OFFICE OF GRANTS MANAGEMENT IN WASHINGTON, DC OR THE FIELD FINANCIAL MANAGEMENT CENTER, LOCATED IN PHILADELPHIA, PA FORMERLY KNOWN AS THE SERVICE CENTER) .

GM1. Do you or someone in your organization interact with the Corporation’s Grants Officers from the Office of Grants Management (Washington, DC) or with those in the Field Financial Management Center (Philadelphia, PA, formerly called Service Centers)?

 

• Office of Grants Management, Headquarters

• Field Financial Management Center

• Both

• Neither (skip to PMR1)

GM2. On average, how often do you interact with your Grants Officer?

 

• Daily

• Weekly

• Monthly

• Annually

• Less than once a year

• Other

 

GM3. On average, how long does it take your Grants Officer to acknowledge receipt of your email or telephone question (by providing an answer or informing you about a timeline for which you will receive a response to your question)?

 

• Within 1 business day

• Within 3 business days

• Within 1 week

• Within 2 weeks

• More than 2 weeks

• N/A

GM4. Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent,” how would you rate the items below? If your organization is not familiar with a particular experience then please respond ‘N/A’.

 

• Your Grants Officer’s knowledge of Learn and Serve America fiscal and administrative requirements

• The availability of your Grants Officer

• The courteousness of your Grants Officer

• The timeliness of your Grants Officer’s resolution to your questions and concerns

• The overall explanation received from your Grants Officer when requesting financial information from you or bringing a financial problem to your attention

• Your Grants Officer’s overall expertise regarding the Corporation’s grant management policies and government-wide federal grant requirements  

• The overall effectiveness of your Grants Officer in helping you meet your fiscal requirements as a Learn and Serve America grantee

GM5. Using a scale of 1 to 10, where 1 is “Does Not Meet My Needs” and 10 is “Fully Meets My Needs,” how would your rate the items below?

• The provisions and/or terms and conditions provided with a grant award

• The Corporation’s policy documents and policy FAQs

• The Corporation’s regulations

• Overall service delivered by my Grants Officers to my organization

• The availability of your Grants Officer (including availability during conferences and training events, and for on-site monitoring)

GM6. The items below should be completed by the person responsible for submitting your Financial Status Report (FSR).  If this person is not available to complete these questions please consult with him or her. Using a scale from 1 to 10, where 1 is “Does Not Meet My Needs” and 10 is “Fully Meets My Needs,” how would you rate the items below? 

 

• How well you were informed about your Financial Status Report due date(s)

• Your Grants Officer’s responses to questions regarding the FSR

Did your organization receive any feedback regarding your Financial Status Report?

• Yes

• No (Skip to PMR1)

GM6.1 Using a scale from 1 to 10, where 1 is “Does Not Meet My Needs” and 10 is “Fully Meets My Needs,” how would you rate the helpfulness of feedback regarding your Financial Status Report

• If your organization did not receive feedback regarding your Financial Status report then respond ‘N/A’.

REVIEW PROCESSES (Do not read)

Now, please think about the Corporation process for monitoring and providing feedback to grantees.

The items below should also be completed by the person responsible for submitting your Financial Status Report (FSR)

Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent”, how would you rate the items below? For this series of questions please think only of the Financial Status Report itself. We will ask technology and eGrants related questions later in the survey.

PMR1 The ease of knowing who to contact with questions or concerns about the Financial Status Report

PMR2 The clarity of instructions regarding the Financial Status Report

PMR3 The ease of preparing the Financial Status Report

PMR4 The amount of time it takes to complete the Financial Status Report

PMR4.1 Using a scale from 1 to 10, where 1 is “Does Not Meet My Needs” and 10 is “Fully Meets My Needs,” how would you rate the items below? 

• The timeliness of feedback provided as a result of information shared in the Financial Status Report

• The helpfulness of feedback provided as a result of information shared in the Financial Status Report

PMR5 Has your organization completed and submitted an Annual Progress Report?

• Yes

• No (Skip to PMR11)

Using a scale from 1 to 10 scale where 1 is “Poor” and 10 is “Excellent,” how would you rate the items below? For this series of questions please think only of the Progress Report itself. We will ask technology and eGrants related questions later in the survey.

PMR6 The ease of knowing who to contact with questions or concerns about the Progress Report

PMR7 The clarity of instructions regarding the Progress Report

PMR8 The ease of preparing the Progress Report

PMR9 The amount of time it takes to complete the Progress Report

PMR10 Did your organization receive any feedback regarding your submitted Progress Report?

• Yes

• No (Skip to PMR11)

PMR10.1 Using a scale from 1 to 10, where 1 is “Does Not Meet My Needs” and 10 is “Fully Meets My Needs,” how would you rate the items below? 

• The timeliness of feedback provided as a result of information shared in the Progress Report

• The helpfulness of feedback provided as a result of information shared in the Progress Report

PMR11 Has your organization completed or submitted the Program and Performance Reporting Forms, also known as LASSIE?

• Yes

• No (Skip to WEB1)

PMR12a Have your sub-grantee (s) completed or submitted the Program and Performance Reporting Forms, or known as LASSIE?

• Yes

• No

Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent,” how would you rate the items below:

PMR13 The ease of knowing who to contact with questions or concerns about LASSIE

PMR14 The clarity of instructions regarding LASSIE

PMR15 The ease of preparing LASSIE

PMR16 The amount of time it takes to complete LASSIE

PMR17 The ease of submission of LASSIE

PMR18 The effectiveness of using LASSIE to manage sub-grantee information

PMR19 Ease of knowing who to contact with questions/concerns about the Corporation’s process for monitoring and providing feedback to grantees .

USABILITY OF THE NATIONAL SERVICE WEBSITES

Now please think about your organization’s experiences with the official national service websites run by the Corporation for National and Community Service. If you are not familiar with a specific website than simply reply, “Never Accessed”

WEB1. Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent”, please rate the

Usability (User-friendliness)

• Learn and Serve America Website: http://www.LearnandServe.gov/ _____

Or

Never accessed the Learn and Serve America Website

• Corporation Website: www.nationalservice.gov _____

Or

Never accessed the Corporation Website

Usefulness of information,

• Learn and Serve America Website: http://www.LearnandServe.gov/ _____

Or

Never accessed the Learn and Serve America Website

• Corporation Website: www.nationalservice.gov _____

Or

Never accessed the Corporation Website

COMMUNICATION OF SYSTEM OUTAGES

These next series of questions asks you to rate the extent to which the Corporation has informed your organization about updates regarding the technology systems, including shutdowns/upgrades, and unplanned outages. During this section of the survey, we are asking that you only consider the timeliness and helpfulness of the Corporation’s announcements that the eGrants system was unavailable. We will ask additional technology and eGrants questions about the usability of the systems later in the survey.

SO1 Have you received announcements that the Corporation’s management and reporting systems were temporarily unavailable (i.e. postings on the Operating Status page, http://www.nationalservice.gov/status , and/or messages sent through the CNCS list serves)?

• Yes

• No (Skip to TECH1)

• Not Applicable (Skip to TECH1)

Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent”, how would you rate the items below?

SO2 The Corporation’s timeliness in posting messages on the Operating Status Page, http://www.nationalservice.gov/status, about problems and issues that may arise as a result from computer maintenance (including shut downs/upgrades).

- Never Accessed the Operating Status Page

TECHNOLOGY (eGRANTS) ( Do not read)

Please think about your use of eGrants, the online system Learn and Serve America uses for grant applications, management and reporting. Using a scale from 1 to 10 where 1 is “Poor” and 10 is “Excellent” [note: survey will include “not applicable” option for all items], please rate the:

TECH1 Clarity of instructions for using eGrants

TECH2 Effectiveness of the error messages system and computer-automated guidance during data entry

TECH3 Ease of submitting applications through eGrants such as applications or progress reports.

Not Applicable (Skip to TECH5)

Using a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent”, how would you rate the items below?

TECH4 The speed of loading online reports through eGrants (such as the Progress Report and Financial Status Report)

TECH5 Overall effectiveness of eGrants in capturing information required for the application

TECH6 [If PMR5=’Yes’] Ease of submitting Progress Reports through eGrants

TECH7 [If PMR5=’Yes’] Overall effectiveness of eGrants in capturing information required for the Progress Reports

TECH8 [If PMR1=’Yes’] Ease of submitting Financial Status Report through eGrants

TECH9 [If PMR1=’Yes’] Overall effectiveness of eGrants in capturing information required for the Financial Status Report

TECH10 The ease of reaching the eGrants Help Desk or submitting an eGrants Help Desk request

TECH11 The timeliness of response from the eGrants Help Desk

TECH12 The quality of service received from the eGrants Help Desk

TECHNICAL ASSISTANCE RESOURCES (Do not read)

As a Learn and Serve America grantee please now consider your program’s experience with the National Service-Learning Clearinghouse, http://servicelearning.org/, the comprehensive online collection for service-learning resources.

TTA1 Overall, how often does staff from your organization access the National Service-Learning Clearinghouse? (http://www.servicelearning.org/)

• Weekly

• 2-3 Time per Month

• Once per Month

• Once every 6 Months

• Less than once every 6 months

• None (Skip to TTA11)

TTA2. Now please indicate your awareness of the items below resources, your frequency of usage, and the level of usefulness, on a scale from 1 to 10, where 1 is “Poor” and 10 is “Excellent”.

Awareness

Have accessed this resource

Not aware help was available

Aware but I did not access

Frequency

Weekly

2-3 times a month

Once per month

Once every 6 months

Never

Usefulness

Very Useful

Useful

Somewhat Useful

Moderately Useful

Not Useful At All

1. Effective Practices Collection

2. Service-Learning Curriculum

3. Service-Learning Research

TTA3 Using a scale from 1 to 10, (where 1 is “Poor” and 10 is “Excellent”), please rate the Service-Learning Clearinghouse based on the items below:

TTA4 The relevance of the topics covered.

TTA5 The ease of navigation

TTA6 The usability (user friendliness) of the Clearinghouse

TTA7 The overall site design

TTA8 The usefulness of information (such as printed publications, grant listings, online courses and event calendars).

TTA9 The relevance of search results.

TTA10 Your ability to accomplish what you want on the site.

TTA11.These next set of questions are about the Corporation’s Resource Center (http://www.nationalserviceresources.org/), the online site for tools, training and information for volunteer and service programs.

Below is a list of the features on the new Resource Center (the Resource Center was recently re-designed and launched in 2008). Please indicate your awareness of each of these services, your frequency of usage, and the level of usefulness, on a scale from 1 to 10, where 1 is "Poor" and 10 is "Excellent". If you are not familiar with a specific resource or service than reply ‘Not aware help was available”.

Awareness:

Have accessed this resource

Not aware help was available

Aware but I did not access

Frequency:

Weekly

2-3 times a month

Once per month

Once every 6 months

Never

Usefulness

Very Useful

Useful

Somewhat Useful

Moderately Useful

Not Useful At All

1. Monthly Feature

2. What’s New

3. Consult A Pro

4. Grab and Go

5. Resources for Specific Groups and Audiences

6. The Effective Practices Collection

7.Online Courses

8.The Lending Library

9. E-Newsletters

10. Sample Forms and Downloadable documents (survey directions-only ask about ‘awareness’ and ‘usefulness’)

11. Email Discussion Lists

12. Calendar of Events

13. Directory of Training and Technical Assistance Providers (survey directions-only ask about ‘awareness’ and ‘usefulness’)

In the items below areas, please also rate the Resource Center, using a scale from 1 to 10, (where 1 is “Poor” and 10 is “Excellent” :

TTA12 The relevance of the topics covered

TTA13 The ease of navigation

TTA14 The usability (user friendliness) of the Resource Center

TTA15 The overall site design

TTA16 The usefulness of information (such as printed publications, grant listings, online courses and event calendars).

TTA17 The relevance and accuracy of search results based on your queries

TTA18. Below is a list of topic areas where the Corporation provides some support to grantees in the form of tools, training, or information. Resources are mostly made available through remote access such as the Service Learning Clearinghouse and/or the Resource Center. However, some topics have been covered at the annual Learn and Serve grantee meeting.

Please indicate how important the topic is to your project, whether you are aware that CNCS provides resources in this topic area, and your rating of the quality of offerings in the topic area for grantees given the need (was it sufficient to meet the need) using a scale from 1 to 10, where 1 is "Poor" and 10 is "Excellent".

Importance/Awareness:

Have accessed this resource

Not aware help was available

Aware but I did not access

This topic is important for grantees

Frequency:

Weekly

2-3 times a month

Once per month

Once every 6 months

1. Performance Measurement and Evaluation

2. Financial and Grants Management

3. Resource and Fund Development

4. Disability Inclusion

5. Using eGrants

6. Engaging Baby Boomers

7. Volunteer Recruitment and Management

8. Disaster Preparedness and Response

ACSI Benchmark Questions (Do not read)

BMK1 Again, thinking of your experiences as a Learn and Serve America grantee, and using a 10-point scale on which “1” means “Very Dissatisfied” and “10” means “Very Satisfied,” how satisfied are you with the programs and services provided by a Learn and Serve America and the Corporation?

BMK2 Using a 10-point scale on which “1” now means “Falls Short of your Expectations” and “10” means “Exceeds your Expectations," to what extent have the programs and services provided by a Learn and Serve America and the Corporation fallen short of or exceeded your expectations?

BMK3 Forget for a moment your experience with Learn and Serve America and Corporation. Now, imagine what an ideal institution allocating grants would be like. How well do you think AmeriCorps State and National and the Corporation compares with that ideal institution you just imagined? Please use a 10-point scale on which “1 ”means“ Not at all Close to the Ideal and 10” means “Very Close to the Ideal.”

Closing (do not read)

CLOSE1 We just have one last question for you today. Do you have any specific suggestions for a Learn and Serve America and the Corporation about how the program can strengthen their customer service to you and your organization?” [capture verbatim]

Thank you very much again for your time. Have a nice day.

p 9 of 11