OMB Package
IMLS Museums for America Evaluation Study
Contract 2009-057
.
March 19, 2010
OMB Package
IMLS Museums for America Evaluation Study
Contract 2009-057
Submitted to:
Carlos Manjarrez
Office
of Policy, Planning, Research & Communication
Institute of
Museum and Library Services
1800 M Street NW, 9th
Floor
Washington, DC 20036-5802
Phone: 202-653-4647
Fax:
202-653-4611
email: cmanjarrez@imls.gov
Submitted by:
RMC Research Corporation
1000 Market Street, Building 2
Portsmouth, NH 03801
Phone: 603.422.8888
Fax: 603.436.9166
email: aapley@rmcres.com
March 19, 2010
Contents
Justification A & B separate documents
Supporting Statement for Paper Work Reduction Act Submissions
Overview
This document has been prepared to support the clearance of procedures and data collection instruments to be used in the Institute of Museum and Library Services’ (IMLS) study of its Museums for American grant program. This study falls under the IMLS contract 2009-057.
Part A, “Justification,” and Part B, “Information Collection Using Statistical Methods,” respond to instructions in SF93. Appendices contain copies of the instruments and descriptions of procedures for which clearance is sought.
Purpose
The Institute of Museum and Library Services (IMLS) is the primary source for federal support for the nation’s libraries and museums (federal law 20 U.S.C. Section 9171). Museums for America (MFA) is the Institute’s largest grant program for museums and is designed to support projects and ongoing activities that build museums’ capacities to serve their communities. The program serves all museum types (aquariums, arboretums, botanical gardens, art museums, youth and children’s museums, general museums, historic houses and sites, history museums, nature centers, natural historical and anthropology museums, planetariums, science and technology centers, specialized museums, and zoological parks), with the exception of for-profit and federally funded institutions. The Museums for America grant program supports high-priority activities that advance an institution’s mission and strategic goals. From 2004 to 2007 IMLS solicited applications and awarded grants in three categories: Serving as Centers of Community Engagement, Sustaining Cultural Heritage, and Supporting Lifelong Learning. More recently awards are being made in refinements of the earlier categories: Engaging Communities (Education, Exhibitions, and Interpretation); Building Institutional Capacity (Management, Policy, and Training); and Collections Stewardship.
In response to its legislative authority to conduct analyses on the impact and effectiveness of museum and library services (20 U.S.C. Chapter 72, Section 9108), IMLS proposes to assess the effectiveness and impact of its Museums for America grant-making program. Data from the study will be used to inform IMLS of 1) applicant and grantee profiles, 2) the applicants’ view of the application process, 3) funded project and implementation activities, and 4) organizational and community impacts from funded projects that have completed. Evaluation findings will contribute to enhancing the MFA application process and the grant-making itself. In addition, the study results will highlight the overall accomplishments and impacts the grant program has on the museum field.
Methodology
RMC Research Corporation has designed a two-part data collection effort. The first is an online survey. The survey will solicit information from all applicant institutions (approximately 1,800 institutions) that applied for Museums for America grants from 2004 to 2009. Questions will include perceptions of outreach, grant requirements and assistance, application outcomes for an institution, and the role of IMLS funding. Additional survey questions will solicit data on the project implementation process from all grantees during this period (approximately 850 institutions). These items will include questions about partnerships formed and grant activities. A further set of questions on the impacts of grantee activities will be addressed to the 2004 – 2007 grantees with completed projects (approximately 350 institutions). If a grantee’s project was completed three or more years ago, the grantee will be asked if there were any long-term impacts on the institution or community from the funded project (approximately 250 institutions). See Appendix A for the online survey outline, flow chart, and survey design.
Six case studies will be conducted at selected MFA grantee sites. These case studies will provide a nuanced view of the impacts of Museums for America grantee activities through the close study of six museums with completed MFA projects. Qualitative data gathering at these institutions will allow for targeted and specific inquiries into the factors associated with impacts both within the institution and in the surrounding community. These data will provide an opportunity to investigate the connections among institutional and community characteristics, program implementation choices, and impact outcomes.
The case studies will be based on interviews and focus groups conducted with museum administrators, program staff, and participants as appropriate to each program, accompanied by extensive review of grant applications, annual reports and other documentation, media coverage, and first-hand observations. Two-person teams of RMC staff will conduct each site visit, anticipated to take one-and-a-half days to complete, with extensive pre-visit planning contact, and follow-up telephone interviews as needed.
Case study site visit selection will occur in close consultation with the IMLS. It is anticipated that two studies will be conducted in each of the three grant categories (Serving as Centers of Community Engagement, Sustaining Cultural Heritage, and Supporting Lifelong Learning). In seeking maximum variability among case studies, selection criteria may also include institution size, location (geographic and urban or rural), audience(s) served, and exemplary successes. RMC intends to conduct three (one from each grant category) of the six site visits as pilot cases prior to OMB approval. See Appendix B for the case study outline and protocols.
Because of staff turnover and the potential for new staff’s unfamiliarity with the MFA application process and in order to secure adequate representation to examine the application process by institution size and type, the entire universe of about 1,800 MFA applying institutions will be solicited for the online survey, rather than a representative sample. A survey of all applicants will allow the Institute to garner valuable information about the application process (including its potential value to non-funded museums in clarifying their mission or strengthening their organizations). All 850 institutions awarded grants from 2004 to 2009 will be asked additional questions concerning project implementation, and a subset of 350 grantees with grants completed will be asked questions about the impacts of their grant-related activities. These latter questions will focus on the institutions’ perceived impact of the grants on their operations and their communities.
Data Collection
Online Survey
With the approval of the survey by OMB and the completion of the online forms and database, RMC will begin the implementation of the online survey. Respondents will complete the surveys online using a code and password. The sequence will be as follows:
IMLS will send out an announcement via letter of the study including the importance of participation to all Museums for America applicants.
RMC will send invitations to Museums for American applicants via postcards addressed to the current director; invitations will direct recipients to the url of the online survey form and contain the code and password to access the survey.
Email reminders will be sent to all non-responding applicants.
To ensure a high response rate of completed project grantees within the desired analytic characteristics (e.g., institutional types and sizes), follow-up phone calls will be made to non-responding grantees.
Completed surveys will be automatically saved and logged into RMC’s secure database.
Any outlying or questionable survey data will be verified via email or phone to ensure data integrity.
Case Studies
RMC Research Corporation has developed a menu of case study protocols in order to choose the most appropriate depending on the type of project a grantee undertook. Case study data will be analyzed as they are collected in order to identify the range of responses for each indicator.
Measures to ensure reliability of the qualitative data include: a) consistent note-taking, b) exposure to multiple and different situations using triangulated methods, and c) comparing emerging themes with findings from related studies. Validity will be addressed as follows:
Face validity (i.e., whether observations fit a plausible frame of reference);
Internal validity through pre-testing instruments, rigorous note-taking and methods, peer debriefing, and member checks or participant verification;
External validity through “thick description” and comprehensive description of our methods so that others can determine if our findings compare with theirs.
Pilot Testing
In order to minimize any technical problems and to clarify minor wording or language issues that could cause confusion, RMC will pilot-test the online survey with four applicants and four grantees. The pilot testing will take place after the online survey has been approved by the OMB. In order to avoid duplication of effort, data collected through pilot testing will be used as survey data.
Three pilot case studies are proposed prior to conducting another three site visits after OMB approval.
Timeline
The following chart details the evaluation timeline by task.
Task Description |
Jan '10 |
Feb |
Mar |
Apr |
May |
Jun |
Jul |
Aug |
Sep |
Oct |
Nov |
Dec |
Jan '11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Develop Evaluation Instruments and OMB package |
|
|
|
|
|
|
|
|
|
|
|
|
|
Develop survey and case study instruments |
X |
X |
|
|
|
|
|
|
|
|
|
|
|
Draft OMB documents |
X |
|
|
|
|
|
|
|
|
|
|
|
|
Submit OMB draft package to IMLS |
X |
|
|
|
|
|
|
|
|
|
|
|
|
Revise and submit final package to IMLS |
|
X |
|
|
|
|
|
|
|
|
|
|
|
OMB Submission and Review |
|
|
X |
X |
X |
X |
X |
|
|
|
|
|
|
Develop Online Survey & Database |
|
|
|
|
|
|
|
|
|
|
|
|
|
Obtain grantee data and contact information from IMLS |
X |
X |
|
|
|
|
|
|
|
|
|
|
|
Create data base of grantees and grants and contact info |
|
|
X |
|
|
|
|
|
|
|
|
|
|
Develop online survey forms |
|
|
X |
X |
X |
|
|
|
|
|
|
|
|
Pilot respondents to test online version |
|
|
|
|
|
X |
|
|
|
|
|
|
|
Finalize forms and database |
|
|
|
|
|
|
|
X |
|
|
|
|
|
Collect Data |
|
|
|
|
|
|
|
|
|
|
|
|
|
Conduct Pilot Case Study Visits (n=3) |
|
|
X |
X |
X |
|
|
|
|
|
|
|
|
Conduct Case Study Visits |
|
|
|
|
|
|
|
|
X |
X |
|
|
|
Prepare and contact grantees for online survey announcement |
|
|
|
|
|
|
|
X |
|
|
|
|
|
Collect online survey data from applicants and grantees |
|
|
|
|
|
|
|
X |
X |
|
|
|
|
Send reminders to complete online survey |
|
|
|
|
|
|
|
|
X |
|
|
|
|
Collect & verify data in database |
|
|
|
|
|
|
|
|
X |
X |
|
|
|
Analyze Data |
|
|
|
|
|
|
|
|
|
|
|
|
|
Analyze existing administrative applicant and grantee data |
|
|
X |
|
|
|
X |
|
|
|
|
|
|
Analyze survey data for preliminary findings |
|
|
|
|
|
|
|
|
|
X |
|
|
|
Review preliminary findings with IMLS |
|
|
|
|
|
|
|
|
|
X |
|
|
|
Conduct final analyses |
|
|
|
|
|
|
|
|
|
|
X |
|
|
Preparation of Final Deliverables |
|
|
|
|
|
|
|
|
|
|
|
|
|
Write draft report |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
Write data file documentation |
|
|
|
|
|
|
|
|
|
|
X |
|
|
Submit draft report and documentation to IMLS for review |
|
|
|
|
|
|
|
|
|
|
|
X |
|
Revise report and documentation |
|
|
|
|
|
|
|
|
|
|
|
X |
|
Design and produce copies of exec summary |
|
|
|
|
|
|
|
|
|
|
|
X |
X |
Submit final report, exec summary, and data files to IMLS |
|
|
|
|
|
|
|
|
|
|
|
|
X |
Send exec summary to responding grantee institutions |
|
|
|
|
|
|
|
|
|
|
|
|
X |
IMLS Museums for America Evaluation Tasks and Timeline
Appendices
Appendix A
Online Survey Outline
Online Survey Flow Chart
Online Survey Questions
Appendix B
Case Study Outline
Case Study Protocols
Appendix A
Online Survey Outline
Online Survey Flow Chart
Online Survey Questions
MFA Online Survey Outline
March 18, 2010
NOTE: Large institutions serving as a parent to multiple organizational units that have applied to the MFA program will be identified. Each organizational unit will be considered a separate, unique entity. Each organizational unit will be asked to participate in the survey and respond to its MFA experiences. The population of MFA applicants from stand-alone museums and separate organizational units are referred to as museum below.
PART A: ALL APPLICANTS from 2004-2009(~1800 museums)
I. Introduction and Instructions
II. Museum Background
All Pre-Populated Records of Application History \ Knowledge of each application
Institutional Type, governance, maturity
Experience with other grant programs
III. Overall Application Process
Application Process Experience
Application Process Impacts
PART B: OVERALL UNFUNDED MFA APPLICATION REACTIONS (~ 1400 museums)
IV. Overall Unfunded Project Experience
Selection and Award Process
Reapplication History
PART C: FUNDED GRANT PROJECT ACTIVITIES and STATUS (~850 museums with open or closed projects)
V. Individual Grant Project Descriptions
Pre-Populated Record of Funded Grant\ Knowledge of the project
Grant Project Background
Partnerships
Activities
Exhibitions
Collections
Technology and Online Resources
Organizational Development
Groups Served by the Project
Status of Activities
Open Projects
Closed Projects
PART D: IMPACTS and REFLECTIONS (~350 museums with 2004-2007 closed grants)
NOTE: This section on individual funded project impacts will provide data for identifying patterns of impacts across analytic characteristics such as grant categories, the size of grants, and institution types. In addition, project impacts will be examined by the different types of activities the institution engaged in during the project. The project impact data are NOT intended to trace any direct causal influence from each project activity.
VI. Individual Grant Project Impacts
Project Impacts
Organizational
Community
Impacts due to Partnerships
Sustainability and Reflection Questions
PART E: LONG TERM IMPACTS (< 250 museums with projects completed 3 or more years ago)
VII. Individual Grant Project Long-Term Impacts
Organizational, Community, and Unexpected Impacts
MFA Evaluation
Online Survey Flow Chart
Each institution/organizational unit will be asked to complete at least two parts of the online survey. The number of parts will depend on the status of their applications (any unfunded, any funded), the timing of project awards (<2008, >=2008), and how long ago the project was completed (3 or more years ago). The chart below presents the sections of the survey a respondent will fill out according to the examples provided.
|
Examples of: Institution/Organizational Unit MFA application and funding completion status |
|||
Target Respondents Online Survey Parts
|
1 or more MFA applications; none were funded |
3 MFA applications; 2 unfunded; 1 funded and currently in process |
3 MFA applications; 1 unfunded; 1 funded 2006 completed < 3yrs ; 1 funded 2008 completed 2010 |
2 MFA 2004-2006 applications; 2 funded; 2 completed 3+ yrs. ago |
Part A All Applicants Background Other Grant Programs Overall Application Process Overall Application Impacts |
X |
X |
X |
X |
Part B Any Unfunded Applicants Overall Unfunded Application Experience |
X |
X |
X |
|
Part C Funded MFA Grantees Earliest Individual Project Background Status |
|
X Project #1 |
X Project #1 |
|
Part D Funded 2004-7 Closed Earliest Individual Project Impacts and Reflections |
|
|
X Project #1 |
X Project #1 |
Part E Funded 2004-7 Closed 3+yrs Earliest Individual Project Long-term impacts |
|
|
|
X Project #1 |
|
|
|
Return to: |
Return to:
|
Part C Funded MFA Grantees Next Earliest Individual Project Background Status |
|
|
X Project #2 |
X Project #2 |
Part D Funded 2004-7 Closed Next Earliest Individual Project Impacts and Reflections |
|
|
|
X Project #2 |
Part E Funded 2004-7 Closed 3+yrs Next Earliest Individual Project Long-term impacts |
|
|
|
X Project #2 |
Online Survey Questions
Museums for America Evaluation
March 18, 2010
PART A: MUSESUM BACKGROUND and OVERALL MFA APPLICATION PROCESS
COMPLETED BY ALL ELIGIBLE MFA APPLICANTS from 2004-2009
[Introduction to come: including discussion of survey purpose, and confidentiality]
IMLS will assist in wording of confidentiality
(INFO) = additional definition of terms in pop-up form
Pre-Populated Record <Fields> To Appear On The Online Survey
1.1 Are you knowledgeable about each of the following MFA applications?
<Institution/Museum Name> <OrgUnit>
All MFA applications submitted to IMLS by first submit date to last
Knowledgeable about? |
Fiscal Year |
Project Title |
Grant Category |
Current Status |
Contact Name |
Contact Title |
|
No |
Yes |
|
|
|
|
|
|
No |
Yes |
|
|
|
|
|
|
No |
Yes |
|
|
|
|
|
|
No |
Yes |
|
|
|
|
|
|
Museum Background
2.1 Which of the following most closely describes your museum? (select one)
|
Aquarium |
|
Historic House/Site |
|
Science/Technology Museum |
|
Arboretum/Botanical garden |
|
History Museum |
|
Specialized Museum (INFO) |
|
Art Museum |
|
Natural History/Anthropology Museum |
|
Zoo |
|
Children’s /Youth Museum |
|
Nature center |
|
|
|
General Museum (INFO) |
|
Planetarium |
|
Other; please specify:_____________________ |
(INFO) A museum with collections representing two or more disciplines equally (e.g., art and history)
(INFO) A museum with collections limited to one narrowly defined discipline (e.g., textiles, stamps, maritime, ethnic group, tribal)
Which of the following most closely describes your museum’s governance? (select one)
|
College, university or other academic entity |
|
Federal |
|
Local (county or municipal) |
|
Non-profit, non-governmental organization or foundation |
|
State |
|
|
|
Native American Tribe/Native Hawaiian Organization |
|
|
|
|
2.2.a. If your museum is governed by a college, university, or other academic entity, which of the following most closely describes your governance? (select one)
|
Community college |
|
Private four-year college or university |
|
Historically Black college or university |
|
State four-year college or university |
|
|
|
Other; please specify: ___________________ |
What year was the museum first open to the public? ______
How many people have held the museum’s leadership role (e.g., director, curator, president) over the past seven years (since 2004)? ______
How many current staff members does the museum have?
___ full-time employees
___ part-time employees
___ unpaid staff (volunteers, docents, Board members)
Other Grant Programs
3.1 How many times has your museum applied for any IMLS grant in the last 12 months? ______ _ Don’t Know
3.2. Has your museum applied for any OTHER IMLS grants listed below?
3.2. If yes, were you awarded a grant?
3.3. If yes, was it before or after your MFA application?
Other IMLS Grant Programs |
3.2. Did your institution apply? |
3.3. If yes, were you awarded a grant? |
3.4. If funded, was it before or after MFA application? |
||||
|
No |
Yes |
Don’t Know |
No |
Yes |
Before |
After |
Skip to b |
|
Skip to b |
Skip to b |
|
|
|
|
b. Connecting
to Collections: |
Skip to c |
|
Skip to c |
Skip to c |
|
|
|
Skip to d |
|
Skip to d |
Skip to d |
|
|
|
|
Skip to e |
|
Skip to e |
Skip to e |
|
|
|
|
Skip to f |
|
Skip to f |
Skip to f |
|
|
|
|
Skip to g |
|
Skip to g |
Skip to g |
|
|
|
|
Skip to h |
|
Skip to h |
Skip to h |
|
|
|
|
h. Museum Grants for African American History & Culture |
Skip to i |
|
Skip to i |
Skip to i |
|
|
|
i. Native American/Native Hawaiian Museum Services, |
Skip to j |
|
Skip to j |
Skip to j |
|
|
|
j. American Heritage Preservation Grants |
Skip to 3.4 |
|
Skip to 3.4 |
Skip to 3.4 |
|
|
|
3.4 Has your museum applied for any federal agency grants other than to IMLS?
|
No |
|
Yes |
|
Don’t Know |
4.Application Process Experience
In order to improve the MFA application process, IMLS is interested in your experience(s) with the process. Please take into account all MFA applications your museum has submitted to IMLS.
4.1. The IMLS Museums for America grant program requires museums to commit to a 1:1 cost sharing for the proposed project. Between 2004 and 2009, did your institution ever CONSIDER applying for a MFA grant but DID NOT APPLY due to the cost sharing requirement?
|
No |
|
Yes |
|
Don’t Know |
4.2. For the MFA application(s) the museum did submit to IMLS, did the cost-sharing requirement have an impact on any of the following with respect to your proposed project(s):
|
No |
Yes |
Don’t Know |
Amount of time to raise the cost sharing funds |
|
|
|
Type of MFA grant applied for |
|
|
|
Duration of the proposed project |
|
|
|
Scope of the proposed project |
|
|
|
Staffing costs of the proposed project |
|
|
|
Non-labor costs of the proposed project |
|
|
|
4.3 Which of the following description(s) apply to the person(s) who wrote your MFA application(s)?
(select all that apply)
|
Dedicated staffed grant writer (paid or unpaid) |
|
Administrative Leadership (e.g., Executive Director, Head Curator, President) |
|
Hired grant writer(s) as consultant |
|
Other staff member(s) (e.g., program coordinator) |
|
Other: please specify________________________ |
|
Don’t Know |
4.4 Were you aware of the IMLS outreach activities listed below?
4.5 If yes, did you participate in any of the activities?
4.6 If yes, how helpful were the activities in completing your MFA application?
|
4.4. Aware |
4.5. If aware, did you participate |
4.6. If participated, how helpful was the activity? |
||||
|
No |
Yes |
No |
Yes |
Not at all helpful |
Somewhat helpful |
Very helpful |
a. Information session(s) at conferences/meetings |
Skip to b |
|
Skip to b |
|
|
|
|
b. Information from national, regional, or state associations regarding MFA application |
Skip to c |
|
Skip to c |
|
|
|
|
c. Audio conference calls prior to application deadlines |
Skip to d |
|
Skip to d |
|
|
|
|
Id. ndividual counseling through phone calls, emails, or in-person visits |
Skip to 4.7 |
|
Skip to 4.7 |
|
|
|
|
4.7. Were you aware of the following MFA resources posted on the IMLS website?
4.8. If yes, did you utilize any of the resources?
4.9. If yes, how helpful were the resources in completing your application?
|
4.7 Aware |
4.8 If aware, did you utilize? |
4.9 If yes, how helpful was the activity? |
||||
|
No |
Yes |
No |
Yes |
Not at all helpful |
Somewhat helpful |
Very helpful |
a. Sample MFA applications and tips on IMLS website |
Skip to b |
|
Skip to b |
|
|
|
|
b. Outcome-based evaluation resources |
Skip to 4.10 |
|
Skip to 4.10 |
|
|
|
|
To what extent do you disagree or agree with the following statements?
Topic |
Statement |
Strongly Disagree |
Disagree |
Agree |
Strongly Agree |
Don’t Know or Not Applicable |
Access to MFA grant information |
Information about the MFA grant opportunities was easy to obtain. |
|
|
|
|
|
Clarity of MFA application |
The guidelines and requirements were clear. |
|
|
|
|
|
|
Instructions on budsget (e.g.;direct, indirect costs, cost-sharing) were clear. |
|
|
|
|
|
|
The evaluation requirements were clear. |
|
|
|
|
|
|
Instructions on submitting an application were clear. |
|
|
|
|
|
|
The MFA grant-review process was clear. |
|
|
|
|
|
Grants.gov |
Downloading the Grants.gov application from the website was easy. |
|
|
|
|
|
|
Submitting the online applications through the Grants.gov was easy. |
|
|
|
|
|
|
IMLS staff was helpful in answering questions about using the Grants.gov website. |
|
|
|
|
|
|
Submitting an application by hand and mail was easier than using grants.gov |
|
|
|
|
|
4.11 Including yourself, has anyone on your staff served as a:
|
No |
Yes |
Don’t Know |
Field reviewer for the MFA grant program |
|
|
|
Grant review panelist for the MFA grant program |
|
|
|
Application Process Impacts
5.1. IMLS is interested in hearing if the application process itself, regardless of funding, had any effects on your museum. To what extent did the application process prompt your museum to :
|
Not at All Effected |
Somewhat Effected |
EffectedA Lot |
Don’t Know |
Create a mission/strategic plan for the first time |
|
|
|
|
Revise your mission/strategic plan |
|
|
|
|
Increase awareness of different programming |
|
|
|
|
Increase awareness of your resources/strengths |
|
|
|
|
Reframe your project in order to fit into the categories in IMLS guidelines |
|
|
|
|
Adopt new ways of integrating outcomes-based planning in the project development and evaluation processes |
|
|
|
|
Explore new external partnerships |
|
|
|
|
Collaborate among departments/staff |
|
|
|
|
Explore new technologies |
|
|
|
|
Improve your institution's ability to apply for other (non IMLS) funding |
|
|
|
|
Other; please specify: |
|
|
|
|
5.2. Is the MFA application process, more difficult (more complex, requires more time), the same or less difficult (less complex, requires less time) than other types of grant applications?
|
More difficult |
Same |
Less difficult |
Don’t Know |
Other Federal Grants |
|
|
|
|
Other Government Grants (State, Local, County, etc) |
|
|
|
|
Private Grants |
|
|
|
|
Were you aware that IMLS DOES NOT track reapplications, therefore each application, whether a reapplication or not, is treated individually and on its own merits?
|
No |
|
Yes |
How likely is it that your museum will apply for another Museums for America grant in the future?
|
Not at all likely |
|
Somewhat Likely |
|
Very Likely |
|
Don’t Know |
5.4.a If not at all likely, why not? (Check all that apply)
|
Already have an application or project in process |
|
Our Mission or strategic is no longer current |
|
Unavailable resources to plan and write an application |
|
Unavailable or inefficient resources to carry out a project if awarded |
|
Finding funds/resources for cost sharing is too difficult |
|
Funding is not needed at this time |
|
Don't see ourselves as competitive/ discouraged by previous failed applications |
|
The application process requires hardware and/or software not available to us |
|
Other; please specify: ____________________________________________ |
What recommendations do you have for improving the MFA application process?
Structured at the institution\organizational unit level (n=1400)
Unfunded project questions ask about overall experience (not app specific)
6. Application Experience
To what extent do you disagree or agree with the following statements:
|
Strongly Disagree |
Disagree |
Agree |
Strongly Agree |
Don’t Know or Not Applicable |
Our application(s) was/were given full and fair consideration. |
|
|
|
|
|
The reviewers’ comments were useful for redesigning the project. |
|
|
|
|
|
IMLS staff was helpful in providing feedback. |
|
|
|
|
|
Did your museum do any of the following with any of the unfunded MFA applications:
Did you: |
No |
Yes |
Don’t Know |
redesign an unfunded project and resubmit the application to the IMLS-MFA program? |
|
|
|
If YES, did you receive IMLS-MFA funding for the redesigned project? |
|
|
|
resubmit a MFA unfunded project to a different IMLS grant program? |
|
|
|
If YES, did you receive IMLS funding from a non-MFA grant program? |
|
|
|
submit a MFA unfunded project to a non-IMLS funder? |
|
|
|
If YES, did you receive funding from the non-IMLS funder? |
|
|
|
6.2.a. If your museum did not receive any outside funding for unfunded MFA projects, were you able to execute any version of a MFA unfunded project?
|
No |
|
Yes |
|
Don’t Know |
Thank you for taking the time to reply to this survey.
Please hit the SUBMIT button if you are done.
Pre-populated fields for an individual funded project.
SYSTEM NOTE: Table below will be presented at the top of each new survey page for Section C, D, E
540 museums had 1 funded grant; 160 had 2;
42 had 3-5 THOSE WILL BE ASKED TO ANSWER QUESTIONS ON ONLY 2 awards – respondent will chose which ones based on knowledge
Knowledgeable about? |
Grant#/Total Grants |
Fiscal Year |
Project Title |
Grant Category |
Current Status |
Contact Name |
Contact Title |
|
No |
Yes |
|
|
|
|
|
|
|
V. Individual Grant Project Descriptions
7. Grant Project Background
(INFO) Partner organizations are defined as any outside organization which expended cash or in-kind resources on the project. In-kind resources include s any payment made in the form of goods and services, rather than cash.
|
Other Museums; please identify museum type _(DROP DOWN MENU)_________________________ |
|
Library |
Government organizations |
|
|
State government |
|
Local or County government |
|
City Government |
|
Other; please describe ___________________________________ |
Non-Government Organizations/Community Organizations |
|
|
Community Health facility (hospital, mental health facility, health clinic, etc) |
|
Family Services Organization (day care, YMCA, family services center, etc) |
|
Youth Organization (Boys and Girls Club, 4H Club, Afterschool Program) |
|
Senior Services |
|
Arts and Culture Organization |
|
Legal Services Organization (family court, legal aid organization, etc) |
|
Local Media |
|
Civic Organization (Kiwanis, Chamber of Commerce, etc) |
|
Local Business |
|
Other Non-Profit Organization; please describe_________________________________ |
|
Other; please describe |
Education |
|
|
Elementary School |
|
Secondary/High School |
|
4 Year College or University |
|
Community College |
|
Other; please describe |
|
Policy and Research Organization; please identify ______________ |
|
Online/ Technology Partner; please identify ______________________ |
|
Other; please describe _____________________________________ |
|
NO partner organizations were\are involved |
Programming Activities– If this MFA project included programming activities, please indicate for each activity whether the museum developed a new activity, OR enhanced an existing activity (e.g., expanding for different age groups, upgrading materials) OR continued an existing activity.
|
Select one - Radio button for selection |
||
|
Develop new programs or materials |
Enhance existing programs or materials |
Continue existing programs or materials |
|
|
|
|
Concerts |
|
|
|
Film festivals |
|
|
|
Live Performances |
|
|
|
Broadcasts |
|
|
|
Demonstrations and Workshops |
|
|
|
Lectures |
|
|
|
Community discussion groups |
|
|
|
Education programs at our institution |
|
|
|
In-school programs |
|
|
|
Afterschool programs |
|
|
|
Curriculum guides |
|
|
|
Interpretive programs or materials |
|
|
|
Classes or institutes |
|
|
|
Training sessions |
|
|
|
Conferences |
|
|
|
Internships, mentoring or apprenticeships opportunities |
|
|
|
Other; please describe
|
|
|
|
|
No Programming Activities |
B. Exhibitions (select all that apply)
|
Upgrade/expanded current exhibit, including making current exhibit accessible (e.g., multi-languages, Braille) |
|
Develop concept for new exhibit |
|
Research new exhibit |
|
Plan new exhibit |
|
Fabricate a new exhibit |
|
Create traveling exhibition |
|
Other, please describe |
|
No Exhibitions |
C. Collections (check all that apply)
|
Digitize collections |
|
Convert non-digital content to digital content |
|
Repurpose digital content (INFO) |
|
Create new digital content |
|
|
Less than 10% of collections |
|
|
---|---|---|---|---|
|
|
Between 10-25% of collections |
|
|
|
|
Between 25%-50% of collections |
|
|
|
|
Between 50% and 75% of collections |
|
|
|
|
Over 75% but less than 100% of collections |
|
|
|
|
100% of our collections |
|
|
|
|
Not Sure |
|
|
|
Inventory collections |
|||
|
Move collections |
|||
|
Create public collections finding guides |
|||
|
Create new collections management guidelines/procedures |
|||
|
Implement new collections management system |
|||
|
Other; please describe |
|
No Collection Activities |
D. Technology and Online Resources (select all that apply)
|
Update or create new website |
|
Create online access to collections records or information |
|
Purchase technology equipment (e.g., computers, digital cameras, scanner, voice recorder) |
|
Consolidate multiple databases |
|
Develop searchable online database |
|
Upgrade, purchase or install new software (e.g., new collections management software) |
|
Create audio tour |
|
Create online exhibition |
|
Develop high-tech interactive exhibition |
|
Other; please describe |
|
No Technology Activities |
E. Organizational Development, including Staffing Resources (select all that apply)
|
Provide staff, volunteer and/or docent training, including in use of new technology and online resources |
|
Hire full-time staff (e.g., program coordinator, director, curator, educator) |
|
Hire part-time staff (e.g., program coordinator, director, curator, educator) |
|
Hire a consultant (e.g., interpretive, planning, education, exhibition, web) |
|
Contract for services (fabrication, design, security, etc) |
|
Train interpreters, volunteers or docents |
|
Create or expand interpreter, docent or volunteer program |
|
Support a research and evaluation program, including conducting surveys |
|
Develop key management plans (e.g., comprehensive interpretive plan, emergency plans, collections management plans) |
|
Other; please specify |
|
No Organizational Development |
|
Paid staff; please describe |
|
Unpaid staff (docents, interns, volunteers); please describe |
|
Adult learners |
|
Seniors |
|
Parents |
|
Youth |
|
Teachers |
|
Students |
|
Administrators |
|
Professionals |
|
Residents of particular neighborhood or community/geographic area within a community |
|
Policymakers |
|
Persons with disabilities |
|
Low income citizens |
|
Specific racial/ethnic communities |
|
Families |
|
Not group-specific |
|
Other; please specify |
|
Preschool |
|
Grades K-5 |
|
Grades 6-8 |
|
Grades 9-12 |
|
Post high school/Young adults (18-25) |
|
Adults (26-64) |
|
Seniors (65 and older) |
|
Multi-age/Family |
|
Other, please specify |
8. Status of Activities
Status of activities –the list of activities that grantees check off in the previous section will be populated to this section.
OPEN GRANTS ONLY
8.1 What is the completion status of the project’s activities?
Example:
|
Completion Status |
|||
<populated with specified activities> |
Will not do as planned |
Still in progress |
Completed |
Don’t Know |
Afterschool programs |
|
|
|
|
Curriculum guides |
|
|
|
|
Train interpreters or docents |
|
|
|
|
Establish a docent program and recruit docents |
|
|
|
|
Implement a research and evaluation program, including conducting surveys |
|
|
|
|
8.2. Were any other activities conducted or planned?
|
Yes |
|
No |
If yes, please list activities and current status of each below
|
Completion Status |
||
Other activities: |
Will not do as planned |
Still in progress |
Completed |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CLOSED GRANTS ONLY
8.1 What is the completion status of the project’s activities?
Example:
|
|
|||
<populated with specified activities> |
Did not do as planned |
Completed |
Don’t Know |
|
Afterschool programs |
|
|
|
|
Curriculum guides |
|
|
|
|
Train interpreters or docents |
|
|
|
|
Establish a docent program and recruit docents |
|
|
|
|
Implement a research and evaluation program, including conducting surveys |
|
|
|
8.2. Were any other activities conducted or planned?
|
Yes |
|
No |
If yes, please list activities and current status of each below (e.g., Completed, Still In Process
|
|
||
Other activities: |
Did not do as planned |
Completed |
Don’t Know |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
OTHER COMMENTS:
As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.
If interested please fill out the following information:
Name:
Title:
Organization:
Address: City State Zip
Thank you for taking the time to reply to this survey.
Please hit the SUBMIT button if you are done.
SYSTEM NOTE/NOT DISPLAYED:
If this project is completed AND funded under the 3 earlier grant categories, THEN respondent will continue on to Section D: Impacts and Reflections.
If this project is in progress OR recently completed AND the museum has more MFA granted projects THEN the respondent returns to Section C and is asked questions about the next funded project.
VI. INDIVIDUAL Grant Project Impacts
9. Project Impacts
Please review the list of impacts this MFA project may have had on the organization or community.
Select each impact resulting from the MFA project.
ORGANIZATIONAL IMPACTS (select all that apply)
Programming |
|
|
Grant-funded programming continued beyond grant |
|
Sustained grant-funded programming |
|
Did not sustain grant-funded programming |
|
Sought additional funding to sustain grant-funded programming |
|
Developed related programming to expand value of grant-funded program |
|
Expanded our range of possibilities for programs or exhibitions |
|
Other; please describe: |
|
No changes in programming/not applicable to grant |
Organizational Capacity |
|
|
Enhanced staff capacity in program development |
|
Enhanced staff capacity in creating new kinds of exhibitions |
|
Enhanced staff capacity in reaching new or larger audiences |
|
Enhanced staff capacity in forming external partnerships |
|
Enhanced staff capacity in working internally across departments |
|
Enhanced staff capacity in using outcomes based evaluation |
|
Improved staff cohesion and commitment to mission |
|
Greater alignment of staff responsibilities with mission |
|
Greater board involvement |
|
Helped institution fulfill its mission (e.g., meet certain benchmarks) |
|
Increased ability to attract outside funding |
|
Decreased ability to attract outside funding |
|
Other; please describe: |
|
No effect on our organization/not applicable to grant |
COMMUNITY IMPACTS (select all that apply)
Audiences |
|
|
Reached new audiences (e.g., youth, families, minorities) |
|
Increased commitment by existing audiences |
|
Changes in regular audience participation |
|
Increased audience access (e.g., expanded hours, mobile programming) |
|
Other; please describe: |
|
No effect on audiences/not applicable to grant |
Community Profile |
|
|
Changed institutional identity |
|
Strengthened museum’s public image |
|
Raised institution’s prestige in its community |
|
Increased local media coverage of institution’s activities |
|
Increased visibility of institution as a center of community learning |
|
Other; please describe: |
|
No change to our profile/not applicable to grant |
Partnerships |
|
|
Improved skills in building partnerships |
|
Strengthened ongoing partnerships |
|
Developed new partnerships |
|
With schools or Institutes of Higher Education |
|
With youth organizations |
|
With other museums/historical societies |
|
With other community organizations |
|
Developed new types of partnerships |
|
Decreased ability to attract partners |
|
Increased ability to attract more partners |
|
Other; please describe: |
|
No effect on partnerships/not applicable to grant |
IMPACTS DUE TO THE PARTNERSHIP(S) (select all that apply)
|
Brought in new audiences |
|
Brought in new resources not normally available to our organization |
|
Brought in new staff, paid or unpaid |
|
Brought in new volunteers or memberships |
|
Raised community awareness of our organization |
|
Other; please describe: |
|
No effect from partnerships/not applicable to grant |
9.2. Are there any other impacts not covered above? If yes, Please describe
__________________________________________________________________________
9.3. If the grant project enabled new programming, to what extent has your organization been successful in continuing the programs?
|
Not at all successful |
|
Somewhat successful |
|
Very successful |
|
Not applicable |
9.4. If the grant enabled you to bring in new audiences, to what extent has your organization been successful in sustaining these new audiences?
|
Not at all successful |
|
Somewhat successful |
|
Very successful |
|
Not applicable |
9.5. Was this MFA grant from IMLS larger, smaller, or about the same as other grants you may have received?
|
Larger |
|
Smaller |
|
About the same |
|
Not applicable |
9.6. Were the MFA grant funds sufficient to complete the planned project activities?
|
No |
|
Somewhat |
|
Yes |
|
Don’t Know |
OTHER COMMENTS:
As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.
If interested please fill out the following information:
Name:
Title:
Organization:
Address: City State Zip
Thank you for taking the time to reply to this survey.
Please hit the SUBMIT button if you are done.
SYSTEM NOTE/NOT DISPLAYED:
If this project was completed 3 or more years ago THEN respondent will continue on to Section E: Long-term Impacts.
If this project is in progress OR recently completed AND the museum has more MFA granted projects THEN the respondent returns to Section C and is asked questions about the next funded project.
PART E: LONG-TERM IMPACTS
COMPLETED BY grantEEs WHERE PROJECT WAS completed 3 or more years ago
VII. Individual Grant Project Long-Term Impacts
10.1. Please describe any long-term impact(s) that the MFA project has had on your organization.
10.2. Please describe any long-term impact(s) that the MFA project has had on your community or community relationships.
10.3. Please describe any UNEXPECTED long-term impact(s) that the MFA project has had on your organization, community or community relationships.
If there is another MFA project completed under the 3 earlier grant categories THEN the respondent returns to Section C and is asked questions about the next funded project.
OTHER COMMENTS:
As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.
If interested please fill out the following information:
Name:
Title:
Organization:
Address: City State Zip
Thank you for taking the time to reply to this survey.
Please hit the SUBMIT button if you are done.
Appendix B
Case Study Outline
Case Study Protocols
Case Study Outline
Museums for America Evaluation
Case studies will provide a nuanced view of MFA impacts through close study of a select group of completed MFA grants. These case studies will be based on a series of qualitative data collection activities conducted at selected institutions, and will allow for targeted and individual inquiry into the factors producing project impacts, both within the institution and in the community.
The case studies will be based on interviews and focus groups with museum administrators, program staff, and participants as appropriate to each program, as well as extensive review of grant applications, annual reports and other documentation, media coverage, and first-hand observations. Case studies will provide an opportunity for investigating the connections among institutional and community characteristics, program implementation choices, and impacts.
Two-person teams of RMC staff will conduct each site visit. Each visit is expected to take one-and-a-half days to complete, with extensive pre-visit planning contact, and follow-up telephone interviews as needed.
Six case studies will be conducted, and will include a mix of each of the three MFA grant categories (Serving as Centers of Community Engagement, Sustaining Cultural Heritage, and Sustaining Lifelong Learning). In seeking maximum variability among case studies, selection criteria may include:
A range of project activities
Institution size, type and location variety
Project director and staff still employed at the museum
Selection of the case study sites will be made in close collaboration with the IMLS. As part of this process, RMC will review relevant grantee reports. RMC will make the final selection based on recommendation of 10-12 projects/institutions by IMLS program officers.
Site selection will conclude with a discussion between IMLS and RMC about the significant factors leading to the particular selections, i.e. the successes or outstanding features of each of the selected grant experiences.
Project Manager(s)
Museum Leadership
Project Staff/Educators
Partners
Community/Audience Members
Case Study Protocols
Project Manager Interview
Interviewee Name: _______________________ Title: ______________________
Institution: ___________________________ Date:_________ Interviewer: ____
Application Process
Could you share with us the history of this project? How did the idea originate?
How did you decide to apply for the IMLS MFA funding? Did you apply more than once? If you applied more than once, did you get feedback and was it helpful?
How would you characterize the application process compared with other grants you have applied for? How does it compare to other Federal grants you’ve applied for?
Did you find the application process valuable? If yes, what was most valuable about it (e.g. what did you learn about your institution, project development, etc.)?
What was most challenging about completing the application?
How did you approach the cost-sharing requirement?
Project Implementation and Sustainability
To what extent was the project successful in advancing your institution’s mission? How does this project align with your institution’s mission?
What aspects of the project were most successful?
What challenges did your institution encounter in implementing the project?
What was the impact or value to your institution of partnerships developed through the project? Have these partnerships been maintained, and if so, how?
What does sustainability mean to you in the context of this project? (e.g., exhibits or programming continue, new audiences engaged, continuation of new offerings, access to collections or other resources, etc.) How was this project or component parts supported after the grant period?
Community Impacts
How did the target audience and/or your existing audience respond to the program? What trends did you observe over time?
How has this project changed the museum’s reputation in the community?
How has the community’s involvement with your museum continued or grown as a result of the program?
Organizational Impacts
In what areas, if any, has the capacity of your staff and/or institution been improved (e.g., growth in knowledge, skills, efficiency, programming)? To what extent were these capacity changes the result of new staff, professional development, strategic planning, budgeting changes, or new technologies?
To what extent, if any, did the project support the development of new relationships within the institution, such as across departments?
Have there been any outcomes of this improved capacity? If so, what? (e.g. additional new programs or initiatives)
Did the project result in any changes in how individuals in the organization perceive the institution’s mission and/or value to the community?
Planning and Evaluation
To what extent were the outcomes-based planning and evaluation requirements familiar to your or your institution?
Do you view planning and evaluation differently as a result of this project? Please explain your response.
To what extent are you integrating outcomes-based planning and evaluation in subsequent projects?
Summary
Overall, what would you say is the most important impact of having received this grant from IMLS?
Were there any surprises or other impacts you did not expect or foresee?
How would you describe the value of the MFA program for institutions like yours?
Any other comments?
Project Staff
Interviewee Name: _______________________ Title: ______________________
Institution: ___________________________ Date:_________ Interviewer: ____
Describe your involvement with this project. What was your role?
How would you describe the implementation of this project? Did it go smoothly?
What new skills or knowledge did you gain from this project?
How have you been able to use these skills or knowledge in subsequent projects?
Did the grant have an impact in how you do your work? If so, explain.
Are there any other ways in which the grant changed how you do your job, your responsibilities, or improved your ability to do your job?
To what extent were you involved in evaluation aspects of the project?
If you were involved in these, were the IMLS requirements valuable?
What challenges did you face in meeting the evaluation requirements? How might these requirements better fit the needs of the MFA projects?
What did you learn from the experience?
Are there other ways in which this grant affected your institution’s ability to serve the public?
What kind of community feedback have you received about the project?
What other aspects of the project have been sustained since the grant period ended? Please describe. How were these projects supported after the grant period?
Summary
Overall, what would you say is the most important impact of having received this grant from IMLS?
Were there any surprises or other impacts you did not expect or foresee? Please describe.
How would you describe the value of the MFA program for institutions like yours?
Any other comments?
Museum Leadership Interview
Interviewee Name: _______________________ Title: ______________________
Institution: ___________________________ Date:_________ Interviewer: ____
Overall Impacts
Please describe the value of this grant for your institution in terms of your mission (e.g., offerings, audience, efficiency).
How does this grant relate to what you consider your institution’s strength in serving the community?
How does this grant relate to what you consider its greatest challenges?
What aspects of the project were most successful?
What challenges did your institution encounter in implementing the project?
How has this project impacted what your institution does?
[If a conservation grant, probe about conservation practices after grant. What changed?]
[If an outreach grant, probe about outreach practices after grant. What changed?]
Etc.
Community Impacts
Did this project change the museum’s reputation in the community? If so, how?
How has the community’s involvement with your museum continued or grown as a result of the program?
Has the museum’s level of engagement with the community changed over time? If so, how? To what do you attribute that change?
IMLS Grantee
Are you regular applicants to IMLS or other federal funders? Was this the first IMLS grant your institution has received or one of many? First MFA grant?
How important was receipt of this funding for the institution?
How did the size of the grant compare with other grants received?
What kind of prestige is conveyed by receipt of the grant?
Were you able to leverage the funds or the prestige in additional ways?
In what ways did the application process or the implementation of the grant impact how you understand your mission or how you understand how you can achieve it?
How would you describe the value of the MFA program for an institution like yours?
What challenges have you faced in sustaining the project?
What new opportunities for sustaining the project have emerged?
Planning and Evaluation
To what extent were the outcomes-based planning and evaluation requirements familiar to your or your institution?
Do you view planning and evaluation differently as a result of this project? Please explain your response.
To what extent are you integrating the outcomes-based planning and evaluation by IMLS in subsequent projects? Please describe current efforts.
Community Member Interview
Interviewee Name: _______________________ Title: ______________________
Institution: ___________________________ Date:_________ Interviewer: ____
How were you connected to the IMLS MFA grant project (attended exhibit or program, community partner, volunteer or docent at the museum, etc.)?
Have you had any dealings with the museum in the past? If yes, what types of activities or programs did you conduct with the museum?
What did you like most about this project?
What did you learn from your involvement in this project?
What value did this project provide to the community?
Did the project have an impact on your impression of the museum? If yes, please describe.
Did the project have an impact on your attendance at museum events or involvement in museum activities?
How would you characterize the relationship between the museum and the community?
Do you know of other community groups that participate in programming with the museum? If so, what types of activities are they engaged in?
Has the museum’s level of engagement with the community changed over time? If so, how? To what do you attribute that change?
Partner Interview
Interviewee Name: _______________________ Title: ______________________
Institution: ___________________________ Date:_________ Interviewer: ____
What was your institution’s role in the MFA project? To what extent were you involved in developing the project? To what extent were you involved in the grant application process for this project?
What was your interest in participating in this project? In what ways was this project valuable for your institution?
Was this the first time you worked with the museum? If so, did this project represent a change in the types of collaboration you have engaged in previously?
How was the partnership structured?
How has your relationship with the museum changed as a result of this project?
Have you continued to work together as a result of the project? If no, why not? If yes, is this work on the same project or on a new project?
Do you anticipate future partnerships with the museum? If so, what kinds of partnerships you would be interested in pursuing? What aspects of the partnership would you continue? Do differently?
How would you characterize the relationship between the museum and community organizations such as yours?
Do you know of other organizations that participate in programming with the museum? If so, what types of activities are they engaged in?
Has the museum’s level of engagement with the community changed over time? If so, how? To what do you attribute that change?
File Type | application/msword |
Author | JJB |
Last Modified By | KMILLER |
File Modified | 2010-05-12 |
File Created | 2010-04-16 |