IMLS Museums for America Evaluation Study

IMLS Museums for America Evaluation Study.doc

Museums for America Grant Program Evaluation

IMLS Museums for America Evaluation Study

OMB: 3137-0079

Document [doc]
Download: doc | pdf


OMB Package







IMLS Museums for America Evaluation Study



Contract 2009-057


.



March 19, 2010












OMB Package


IMLS Museums for America Evaluation Study



Contract 2009-057




Submitted to:


Carlos Manjarrez

Office of Policy, Planning, Research & Communication
Institute of Museum and Library Services
1800 M Street NW, 9th Floor
Washington, DC 20036-5802
Phone: 202-653-4647
Fax: 202-653-4611
email: cmanjarrez@imls.gov



Submitted by:


RMC Research Corporation

1000 Market Street, Building 2

Portsmouth, NH 03801

Phone: 603.422.8888

Fax: 603.436.9166

email: aapley@rmcres.com





March 19, 2010



Contents

Supporting Statement for Paper Work Reduction Act Submissions

Overview 1

Purpose 1

Methodology 1

Data Collection 2

Pilot Testing 3

Timetable 3


Appendices 5

Appendix A 6

Online Survey Outline 7

Online Survey Flow Chart 9

Online Survey Questions 10

Appendix B 33

Case Study Outline 34

Case Study Protocols 36


Justification A & B separate documents



Supporting Statement for Paper Work Reduction Act Submissions


Overview

This document has been prepared to support the clearance of procedures and data collection instruments to be used in the Institute of Museum and Library Services’ (IMLS) study of its Museums for American grant program. This study falls under the IMLS contract 2009-057.


Part A, “Justification,” and Part B, “Information Collection Using Statistical Methods,” respond to instructions in SF93. Appendices contain copies of the instruments and descriptions of procedures for which clearance is sought.


Purpose

The Institute of Museum and Library Services (IMLS) is the primary source for federal support for the nation’s libraries and museums (federal law 20 U.S.C. Section 9171). Museums for America (MFA) is the Institute’s largest grant program for museums and is designed to support projects and ongoing activities that build museums’ capacities to serve their communities. The program serves all museum types (aquariums, arboretums, botanical gardens, art museums, youth and children’s museums, general museums, historic houses and sites, history museums, nature centers, natural historical and anthropology museums, planetariums, science and technology centers, specialized museums, and zoological parks), with the exception of for-profit and federally funded institutions. The Museums for America grant program supports high-priority activities that advance an institution’s mission and strategic goals. From 2004 to 2007 IMLS solicited applications and awarded grants in three categories: Serving as Centers of Community Engagement, Sustaining Cultural Heritage, and Supporting Lifelong Learning. More recently awards are being made in refinements of the earlier categories: Engaging Communities (Education, Exhibitions, and Interpretation); Building Institutional Capacity (Management, Policy, and Training); and Collections Stewardship. 

In response to its legislative authority to conduct analyses on the impact and effectiveness of museum and library services (20 U.S.C. Chapter 72, Section 9108), IMLS proposes to assess the effectiveness and impact of its Museums for America grant-making program. Data from the study will be used to inform IMLS of 1) applicant and grantee profiles, 2) the applicants’ view of the application process, 3) funded project and implementation activities, and 4) organizational and community impacts from funded projects that have completed. Evaluation findings will contribute to enhancing the MFA application process and the grant-making itself. In addition, the study results will highlight the overall accomplishments and impacts the grant program has on the museum field.

Methodology

RMC Research Corporation has designed a two-part data collection effort. The first is an online survey. The survey will solicit information from all applicant institutions (approximately 1,800 institutions) that applied for Museums for America grants from 2004 to 2009. Questions will include perceptions of outreach, grant requirements and assistance, application outcomes for an institution, and the role of IMLS funding. Additional survey questions will solicit data on the project implementation process from all grantees during this period (approximately 850 institutions). These items will include questions about partnerships formed and grant activities. A further set of questions on the impacts of grantee activities will be addressed to the 2004 – 2007 grantees with completed projects (approximately 350 institutions). If a grantee’s project was completed three or more years ago, the grantee will be asked if there were any long-term impacts on the institution or community from the funded project (approximately 250 institutions). See Appendix A for the online survey outline, flow chart, and survey design.

Six case studies will be conducted at selected MFA grantee sites. These case studies will provide a nuanced view of the impacts of Museums for America grantee activities through the close study of six museums with completed MFA projects. Qualitative data gathering at these institutions will allow for targeted and specific inquiries into the factors associated with impacts both within the institution and in the surrounding community. These data will provide an opportunity to investigate the connections among institutional and community characteristics, program implementation choices, and impact outcomes.

The case studies will be based on interviews and focus groups conducted with museum administrators, program staff, and participants as appropriate to each program, accompanied by extensive review of grant applications, annual reports and other documentation, media coverage, and first-hand observations. Two-person teams of RMC staff will conduct each site visit, anticipated to take one-and-a-half days to complete, with extensive pre-visit planning contact, and follow-up telephone interviews as needed.

Case study site visit selection will occur in close consultation with the IMLS. It is anticipated that two studies will be conducted in each of the three grant categories (Serving as Centers of Community Engagement, Sustaining Cultural Heritage, and Supporting Lifelong Learning). In seeking maximum variability among case studies, selection criteria may also include institution size, location (geographic and urban or rural), audience(s) served, and exemplary successes. RMC intends to conduct three (one from each grant category) of the six site visits as pilot cases prior to OMB approval. See Appendix B for the case study outline and protocols.

Because of staff turnover and the potential for new staff’s unfamiliarity with the MFA application process and in order to secure adequate representation to examine the application process by institution size and type, the entire universe of about 1,800 MFA applying institutions will be solicited for the online survey, rather than a representative sample. A survey of all applicants will allow the Institute to garner valuable information about the application process (including its potential value to non-funded museums in clarifying their mission or strengthening their organizations). All 850 institutions awarded grants from 2004 to 2009 will be asked additional questions concerning project implementation, and a subset of 350 grantees with grants completed will be asked questions about the impacts of their grant-related activities. These latter questions will focus on the institutions’ perceived impact of the grants on their operations and their communities.


Data Collection

Online Survey

With the approval of the survey by OMB and the completion of the online forms and database, RMC will begin the implementation of the online survey. Respondents will complete the surveys online using a code and password. The sequence will be as follows:

  1. IMLS will send out an announcement via letter of the study including the importance of participation to all Museums for America applicants.

  2. RMC will send invitations to Museums for American applicants via postcards addressed to the current director; invitations will direct recipients to the url of the online survey form and contain the code and password to access the survey.

  3. Email reminders will be sent to all non-responding applicants.

  4. To ensure a high response rate of completed project grantees within the desired analytic characteristics (e.g., institutional types and sizes), follow-up phone calls will be made to non-responding grantees.

  5. Completed surveys will be automatically saved and logged into RMC’s secure database.

  6. Any outlying or questionable survey data will be verified via email or phone to ensure data integrity.


Case Studies

RMC Research Corporation has developed a menu of case study protocols in order to choose the most appropriate depending on the type of project a grantee undertook. Case study data will be analyzed as they are collected in order to identify the range of responses for each indicator.

Measures to ensure reliability of the qualitative data include: a) consistent note-taking, b) exposure to multiple and different situations using triangulated methods, and c) comparing emerging themes with findings from related studies. Validity will be addressed as follows:

  • Face validity (i.e., whether observations fit a plausible frame of reference);

  • Internal validity through pre-testing instruments, rigorous note-taking and methods, peer debriefing, and member checks or participant verification;

  • External validity through “thick description” and comprehensive description of our methods so that others can determine if our findings compare with theirs.


Pilot Testing

In order to minimize any technical problems and to clarify minor wording or language issues that could cause confusion, RMC will pilot-test the online survey with four applicants and four grantees. The pilot testing will take place after the online survey has been approved by the OMB. In order to avoid duplication of effort, data collected through pilot testing will be used as survey data.


Three pilot case studies are proposed prior to conducting another three site visits after OMB approval.


Timeline

The following chart details the evaluation timeline by task.


Task Description

Jan '10

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Jan '11

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Develop Evaluation Instruments and OMB package

 

 

 

 

 

 

 

 

 

 

 

 

Develop survey and case study instruments

X

 

 

 

 

 

 

 

 

 

 

 

Draft OMB documents

 X

 

 

 

 

 

 

 

 

 

 

 

 

Submit OMB draft package to IMLS

 X

 

 

 

 

 

 

 

 

 

 

 

 

Revise and submit final package to IMLS


 X

 

 

 

 

 

 

 

 

 

 

 

OMB Submission and Review



X

X

X

X

 X

 

 

 

 

 

 

Develop Online Survey & Database

 

 

 

 

 

 

 

 

 

 

 

 

Obtain grantee data and contact information from IMLS

X

X












Create data base of grantees and grants and contact info



X











Develop online survey forms



 X

 

 

 

 

 

 

 

 

Pilot respondents to test online version

 

 


 

 

 X

 

 

 

 

 

 

 

Finalize forms and database

 

 


 

 

 

 

 X

 

 

 

 

 

Collect Data

 

 

 

 

 

 

 

 

 

 

 

 

 

Conduct Pilot Case Study Visits (n=3)



X

X

 X

 

 

 

 

 

 

 

 

Conduct Case Study Visits









X

X




Prepare and contact grantees for online survey announcement

 

 

 

 

 



 X

 

 

 

 

 

Collect online survey data from applicants and grantees

 

 

 

 

 



 X

 

 

 

Send reminders to complete online survey

 

 

 

 

 



 

 X

 

 

 

 

Collect & verify data in database

 

 

 

 

 



 

 X

 X

 

 

 

Analyze Data

 

 

 

 

 



 

 

 

 

 

Analyze existing administrative applicant and grantee data



X




X







Analyze survey data for preliminary findings

 

 

 

 

 




 

 X

 

 

 

Review preliminary findings with IMLS

 

 

 

 

 

 

 


 

 X

 

 

 

Conduct final analyses

 

 

 

 

 

 

 



 

 X

 

 

Preparation of Final Deliverables

 

 

 

 

 

 

 

 

 

 

 

 

 

Write draft report

 

 

 

 

 

 

 

 

 

X

X

Write data file documentation

 

 

 

 

 

 

 

 

 

 

X

 

 

Submit draft report and documentation to IMLS for review

 

 

 

 

 

 

 

 

 

 

 

X

 

Revise report and documentation

 

 

 

 

 

 

 

 

 

 

 

X

 

Design and produce copies of exec summary

 

 

 

 

 

 

 

 

 

 

 

X

X

Submit final report, exec summary, and data files to IMLS

 

 

 

 

 

 

 

 

 

 

 

 

X

Send exec summary to responding grantee institutions













X

IMLS Museums for America Evaluation Tasks and Timeline






Appendices


Appendix A

Online Survey Outline

Online Survey Flow Chart

Online Survey Questions


Appendix B

Case Study Outline

Case Study Protocols




Appendix A


Online Survey Outline

Online Survey Flow Chart

Online Survey Questions

MFA Online Survey Outline

March 18, 2010


NOTE: Large institutions serving as a parent to multiple organizational units that have applied to the MFA program will be identified. Each organizational unit will be considered a separate, unique entity. Each organizational unit will be asked to participate in the survey and respond to its MFA experiences. The population of MFA applicants from stand-alone museums and separate organizational units are referred to as museum below.


PART A: ALL APPLICANTS from 2004-2009(~1800 museums)

I. Introduction and Instructions

II. Museum Background

  • All Pre-Populated Records of Application History \ Knowledge of each application

  • Institutional Type, governance, maturity

  • Experience with other grant programs

III. Overall Application Process

  • Application Process Experience

  • Application Process Impacts


PART B: OVERALL UNFUNDED MFA APPLICATION REACTIONS (~ 1400 museums)

IV. Overall Unfunded Project Experience

  • Selection and Award Process

  • Reapplication History


PART C: FUNDED GRANT PROJECT ACTIVITIES and STATUS (~850 museums with open or closed projects)

V. Individual Grant Project Descriptions

  • Pre-Populated Record of Funded Grant\ Knowledge of the project

  • Grant Project Background

    • Partnerships

    • Activities

    • Exhibitions

    • Collections

    • Technology and Online Resources

    • Organizational Development

    • Groups Served by the Project

  • Status of Activities

    • Open Projects

    • Closed Projects



PART D: IMPACTS and REFLECTIONS (~350 museums with 2004-2007 closed grants)

NOTE: This section on individual funded project impacts will provide data for identifying patterns of impacts across analytic characteristics such as grant categories, the size of grants, and institution types. In addition, project impacts will be examined by the different types of activities the institution engaged in during the project. The project impact data are NOT intended to trace any direct causal influence from each project activity.


VI. Individual Grant Project Impacts

  • Project Impacts

    • Organizational

    • Community

    • Impacts due to Partnerships

  • Sustainability and Reflection Questions


PART E: LONG TERM IMPACTS (< 250 museums with projects completed 3 or more years ago)

VII. Individual Grant Project Long-Term Impacts

  • Organizational, Community, and Unexpected Impacts

MFA Evaluation

Online Survey Flow Chart

Each institution/organizational unit will be asked to complete at least two parts of the online survey. The number of parts will depend on the status of their applications (any unfunded, any funded), the timing of project awards (<2008, >=2008), and how long ago the project was completed (3 or more years ago). The chart below presents the sections of the survey a respondent will fill out according to the examples provided.


Examples of:

Institution/Organizational Unit MFA application and funding completion status





Target Respondents

Online Survey Parts


1 or more MFA applications; none were funded

3 MFA applications; 2 unfunded;

1 funded and currently in process

3 MFA applications;

1 unfunded;

1 funded 2006 completed

< 3yrs ;

1 funded 2008 completed 2010

2 MFA 2004-2006

applications;

2 funded;

2 completed 3+ yrs. ago

Part A All Applicants

Background

Other Grant Programs

Overall Application Process

Overall Application Impacts

X

X

X

X

Part B Any Unfunded Applicants

Overall Unfunded Application

Experience

X

X

X


Part C Funded MFA Grantees Earliest Individual Project

Background

Status


X

Project #1

X

Project #1


Part D Funded 2004-7 Closed

Earliest Individual Project

Impacts and Reflections



X

Project #1

X

Project #1

Part E Funded 2004-7 Closed 3+yrs

Earliest Individual Project

Long-term impacts




X

Project #1




Return to:

Return to:


Part C Funded MFA Grantees

Next Earliest Individual Project

Background

Status



X

Project #2

X

Project #2

Part D Funded 2004-7 Closed

Next Earliest Individual Project

Impacts and Reflections




X

Project #2

Part E Funded 2004-7 Closed 3+yrs

Next Earliest Individual Project

Long-term impacts




X

Project #2


Online Survey Questions

Museums for America Evaluation

March 18, 2010


PART A: MUSESUM BACKGROUND and OVERALL MFA APPLICATION PROCESS

COMPLETED BY ALL ELIGIBLE MFA APPLICANTS from 2004-2009


Introduction and Instructions

[Introduction to come: including discussion of survey purpose, and confidentiality]

IMLS will assist in wording of confidentiality

(INFO) = additional definition of terms in pop-up form


Museum Background

This section will be structured at the museum (institution\organizational unit) level (n~1800)


  1. Pre-Populated Record <Fields> To Appear On The Online Survey


1.1 Are you knowledgeable about each of the following MFA applications?

 <Institution/Museum Name>  <OrgUnit>


All MFA applications submitted to IMLS by first submit date to last

Knowledgeable about?

Fiscal

Year

Project

Title

Grant Category

Current

Status

Contact

Name

Contact

Title

No

Yes

 

 

 

 

 

 

No

Yes

 

 

 

 

 

 

No

Yes

 

 

 

 

 

 

No

Yes

 

 

 

 

 

 


  1. Museum Background


2.1 Which of the following most closely describes your museum? (select one)



Aquarium


Historic House/Site


Science/Technology Museum


Arboretum/Botanical garden


History Museum


Specialized Museum (INFO)


Art Museum


Natural History/Anthropology Museum


Zoo


Children’s /Youth Museum


Nature center




General Museum (INFO)


Planetarium


Other; please specify:_____________________


(INFO) A museum with collections representing two or more disciplines equally (e.g., art and history)

(INFO) A museum with collections limited to one narrowly defined discipline (e.g., textiles, stamps, maritime, ethnic group, tribal)



    1. Which of the following most closely describes your museum’s governance? (select one)



College, university or other academic entity


Federal


Local (county or municipal)


Non-profit, non-governmental organization or foundation


State




Native American Tribe/Native Hawaiian Organization






2.2.a. If your museum is governed by a college, university, or other academic entity, which of the following most closely describes your governance? (select one)



Community college


Private four-year college or university


Historically Black college or university


State four-year college or university




Other; please specify:

___________________


  

    1. What year was the museum first open to the public? ______


    1. How many people have held the museum’s leadership role (e.g., director, curator, president) over the past seven years (since 2004)? ______


    1. How many current staff members does the museum have?

___ full-time employees

___ part-time employees

___ unpaid staff (volunteers, docents, Board members)

  1. Other Grant Programs

3.1 How many times has your museum applied for any IMLS grant in the last 12 months? ______ _ Don’t Know



3.2. Has your museum applied for any OTHER IMLS grants listed below?

3.2. If yes, were you awarded a grant?

3.3. If yes, was it before or after your MFA application?


 

Other IMLS Grant Programs

3.2. Did your institution apply?

3.3. If yes, were you awarded a grant?

3.4. If funded, was it before or after MFA application?


No

Yes

Don’t

Know

No

Yes

Before

After

a. 21st Century Museum Professionals

Skip to b


Skip to b

Skip to

b




b. Connecting to Collections:
Statewide Implementation Grants

Skip to c


Skip to c

Skip to

c




c. Connecting to Collections:
Statewide Planning Grants

Skip to d


Skip to d

Skip to

d




d. Conservation Assessment Program

Skip to e


Skip to e

Skip to

e




e. Conservation Project Support

Skip to

f


Skip to

f

Skip to

f




f. Museum Assessment Program

Skip to g


Skip to g

Skip to

g




g. National Leadership Grants

Skip to h


Skip to h

Skip to

h




h. Museum Grants for African American History & Culture

Skip to

i


Skip to

i

Skip to

i




i. Native American/Native Hawaiian Museum Services,

Skip to

j


Skip to

j

Skip to

j




j. American Heritage Preservation Grants

Skip to 3.4


Skip to 3.4

Skip to

3.4




3.4 Has your museum applied for any federal agency grants other than to IMLS?


No


Yes


Don’t Know




III. Application Process


4.Application Process Experience 

In order to improve the MFA application process, IMLS is interested in your experience(s) with the process.  Please take into account all MFA applications your museum has submitted to IMLS.      


4.1. The IMLS Museums for America grant program requires museums to commit to a 1:1 cost sharing for the proposed project. Between 2004 and 2009, did your institution ever CONSIDER applying for a MFA grant but DID NOT APPLY due to the cost sharing requirement?


No


Yes


Don’t Know




4.2. For the MFA application(s) the museum did submit to IMLS, did the cost-sharing requirement have an impact on any of the following with respect to your proposed project(s):



No

Yes

Don’t Know

Amount of time to raise the cost sharing funds




Type of MFA grant applied for




Duration of the proposed project




Scope of the proposed project




Staffing costs of the proposed project




Non-labor costs of the proposed project















4.3 Which of the following description(s) apply to the person(s) who wrote your MFA application(s)?

(select all that apply)



Dedicated staffed grant writer (paid or unpaid)


Administrative Leadership (e.g., Executive Director, Head Curator, President)


Hired grant writer(s) as consultant


Other staff member(s) (e.g., program coordinator)


Other: please specify________________________


Don’t Know


4.4 Were you aware of the IMLS outreach activities listed below?

4.5 If yes, did you participate in any of the activities?

4.6 If yes, how helpful were the activities in completing your MFA application?


 

4.4. Aware

4.5. If aware, did you participate

4.6. If participated, how helpful was the activity?


No

Yes

No

Yes

Not at all helpful

Somewhat helpful

Very helpful

a. Information session(s) at conferences/meetings

Skip to b


Skip to b


 

 

 

b. Information from national, regional, or state associations regarding MFA application

Skip to c


Skip to c


 

 

 

c. Audio conference calls prior to application deadlines

Skip to d


Skip to d


 

 

 

Id. ndividual counseling through phone calls, emails, or in-person visits

Skip to 4.7


Skip to 4.7







4.7. Were you aware of the following MFA resources posted on the IMLS website?

4.8. If yes, did you utilize any of the resources?

4.9. If yes, how helpful were the resources in completing your application?


 

4.7 Aware

4.8 If aware, did you utilize?

4.9 If yes, how helpful was the activity?


No

Yes


No

Yes

Not at all helpful

Somewhat helpful

Very helpful

a. Sample MFA applications and tips on IMLS website

Skip to b


Skip to b





b. Outcome-based evaluation resources

Skip to 4.10


Skip to 4.10






    1. To what extent do you disagree or agree with the following statements? 


Topic

Statement

Strongly

Disagree

Disagree

Agree

Strongly

Agree

Don’t

Know or Not

Applicable

Access to MFA grant information

Information about the MFA grant opportunities was easy to obtain.






Clarity of MFA application

The guidelines and requirements were clear.







Instructions on budsget (e.g.;direct, indirect costs, cost-sharing) were clear.







The evaluation requirements were clear.







Instructions on submitting an application were clear.







The MFA grant-review process was clear.






Grants.gov

Downloading the Grants.gov application from the website was easy.







Submitting the online applications through the Grants.gov was easy.







IMLS staff was helpful in answering questions about using the Grants.gov website.







Submitting an application by hand and mail was easier than using grants.gov







4.11 Including yourself, has anyone on your staff served as a:



No

Yes

Don’t Know

Field reviewer for the MFA grant program




Grant review panelist for the MFA grant program





  1. Application Process Impacts


5.1. IMLS is interested in hearing if the application process itself, regardless of funding, had any effects on your museum. To what extent did the application process prompt your museum to : 


 

Not at

All Effected

Somewhat Effected

EffectedA Lot

Don’t

Know

Create a mission/strategic plan for the first time

 

 

 


Revise your mission/strategic plan

 

 

 


Increase awareness of different programming

 

 

 


Increase awareness of your resources/strengths





Reframe your project in order to fit into the categories in IMLS guidelines

 

 

 


Adopt new ways of integrating outcomes-based planning in the project development and evaluation processes





Explore new external partnerships 

 

 

 


Collaborate among departments/staff





Explore new technologies

 

 

 


Improve your institution's ability to apply for other (non IMLS) funding 

 

 

 


Other; please specify:

 

 

 



5.2. Is the MFA application process, more difficult (more complex, requires more time), the same or less difficult (less complex, requires less time) than other types of grant applications?



More difficult

Same

Less difficult

Don’t Know

Other Federal Grants





Other Government Grants (State, Local, County, etc)





Private Grants













    1. Were you aware that IMLS DOES NOT track reapplications, therefore each application, whether a reapplication or not, is treated individually and on its own merits?


No


Yes




    1. How likely is it that your museum will apply for another Museums for America grant in the future?



Not at all likely


Somewhat Likely


Very Likely


Don’t

Know



5.4.a If not at all likely, why not? (Check all that apply)


Already have an application or project in process


Our Mission or strategic is no longer current


Unavailable resources to plan and write an application


Unavailable or inefficient resources to carry out a project if awarded


Finding funds/resources for cost sharing is too difficult


Funding is not needed at this time


Don't see ourselves as competitive/ discouraged by previous failed applications


The application process requires hardware and/or software not available to us


Other; please specify: ____________________________________________



    1. What recommendations do you have for improving the MFA application process?






SYSTEM NOTE : IF MUSEUM HAD ALL APPLICATIONS FUNDED Skip to Part C for individual funded project level.

SYSTEM NOTE: IF MUSEUM HAD ANY APPLICATIONS UNFUNDED Continue with Section B.

PART B: OVERALL UNFUNDED MFA APPLICATION REACTIONS

COMPLETED BY ALL INSTITUTIONS with ANY UNFUNDED MFA APPLICATIONS


IV. Unfunded MFA Project(s) (all institutions with any unfunded MFA applications)

  • Structured at the institution\organizational unit level (n=1400)

  • Unfunded project questions ask about overall experience (not app specific)

6. Application Experience

    1. To what extent do you disagree or agree with the following statements:


 

Strongly

Disagree

Disagree

Agree

Strongly

Agree

Don’t

Know or Not

Applicable

Our application(s) was/were given full and fair consideration.






The reviewers’ comments were useful for redesigning the project.






IMLS staff was helpful in providing feedback.







    1. Did your museum do any of the following with any of the unfunded MFA applications:


Did you:

No

Yes

Don’t

Know

redesign an unfunded project and resubmit the application to the IMLS-MFA program?




If YES, did you receive IMLS-MFA funding for the redesigned project?




resubmit a MFA unfunded project to a different IMLS grant program?




If YES, did you receive IMLS funding from a non-MFA grant program?




submit a MFA unfunded project to a non-IMLS funder?




If YES, did you receive funding from the non-IMLS funder?




6.2.a. If your museum did not receive any outside funding for unfunded MFA projects, were you able to execute any version of a MFA unfunded project?


No


Yes


Don’t

Know


SYSTEM NOTE:IF NO FUNDED GRANTS - EXIT

Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.

SYSTEM NOTE:IF ANY FUNDED GRANTS continue with Part C at the individual funded project level.

PART C: FUNDED GRANT PROJECT ACTIVITIES and STATUS

COMPLETED BY ALL MFA GRANTEES (open and closed projects)

Structured at the museum\project level (n ~ 850)


Pre-populated fields for an individual funded project.

SYSTEM NOTE: Table below will be presented at the top of each new survey page for Section C, D, E

540 museums had 1 funded grant; 160 had 2;

42 had 3-5 THOSE WILL BE ASKED TO ANSWER QUESTIONS ON ONLY 2 awards – respondent will chose which ones based on knowledge


Knowledgeable about?

Grant#/Total Grants

Fiscal

Year

Project

Title

Grant Category

Current

Status

Contact

Name

Contact

Title

No

Yes


 

 

 

 

 

 



V. Individual Grant Project Descriptions


7. Grant Project Background


7.1. Please identify the types of partner organizations (INFO involved in this MFA project.

(INFO) Partner organizations are defined as any outside organization which expended cash or in-kind resources on the project. In-kind resources include s any payment made in the form of goods and services, rather than cash.



Other Museums; please identify museum type _(DROP DOWN MENU)_________________________


Library


Government organizations


State government


Local or County government


City Government


Other; please describe ___________________________________


Non-Government Organizations/Community Organizations


Community Health facility (hospital, mental health facility, health clinic, etc)


Family Services Organization (day care, YMCA, family services center, etc)


Youth Organization (Boys and Girls Club, 4H Club, Afterschool Program)


Senior Services


Arts and Culture Organization


Legal Services Organization (family court, legal aid organization, etc)


Local Media


Civic Organization (Kiwanis, Chamber of Commerce, etc)


Local Business


Other Non-Profit Organization; please describe_________________________________


Other; please describe


Education


Elementary School


Secondary/High School


4 Year College or University


Community College


Other; please describe




Policy and Research Organization; please identify ______________






Online/ Technology Partner; please identify ______________________



Other; please describe _____________________________________



NO partner organizations were\are involved


7.2. Indicate which of the following types of activities (Programming, Exhibitions, Collections, Technology and Online Resources, or Organizational Development) your museum conducted or is in the process of doing as part of this MFA project.


Programming Activities– If this MFA project included programming activities, please indicate for each activity whether the museum developed a new activity, OR enhanced an existing activity (e.g., expanding for different age groups, upgrading materials) OR continued an existing activity.



Select one - Radio button for selection


Develop new programs or materials

Enhance existing programs or materials

Continue existing programs or materials





Concerts




Film festivals




Live Performances




Broadcasts




Demonstrations and Workshops




Lectures




Community discussion groups




Education programs at our institution




In-school programs




Afterschool programs




Curriculum guides




Interpretive programs or materials




Classes or institutes




Training sessions




Conferences




Internships, mentoring or apprenticeships opportunities




Other; please describe








No Programming Activities




B. Exhibitions (select all that apply)



Upgrade/expanded current exhibit, including making current exhibit accessible (e.g., multi-languages, Braille)


Develop concept for new exhibit


Research new exhibit


Plan new exhibit


Fabricate a new exhibit


Create traveling exhibition


Other, please describe



No Exhibitions

C. Collections (check all that apply)



Digitize collections


Convert non-digital content to digital content


Repurpose digital content (INFO)


Create new digital content

(INFO) Utilized digital content in a different way (e.g., used for education programs)

If the MFA project involved digitizing the museum’s holdings, what portion of the collections did/would the project cover? (select one only)




Less than 10% of collections




Between 10-25% of collections




Between 25%-50% of collections




Between 50% and 75% of collections




Over 75% but less than 100% of collections




100% of our collections




Not Sure



Inventory collections


Move collections


Create public collections finding guides


Create new collections management guidelines/procedures


Implement new collections management system


Other; please describe



No Collection Activities



D. Technology and Online Resources (select all that apply)



Update or create new website


Create online access to collections records or information


Purchase technology equipment (e.g., computers, digital cameras, scanner, voice recorder)


Consolidate multiple databases


Develop searchable online database


Upgrade, purchase or install new software (e.g., new collections management software)


Create audio tour


Create online exhibition


Develop high-tech interactive exhibition


Other; please describe



No Technology Activities



E. Organizational Development, including Staffing Resources (select all that apply)



Provide staff, volunteer and/or docent training, including in use of new technology and online resources


Hire full-time staff (e.g., program coordinator, director, curator, educator)


Hire part-time staff (e.g., program coordinator, director, curator, educator)


Hire a consultant (e.g., interpretive, planning, education, exhibition, web)


Contract for services (fabrication, design, security, etc)


Train interpreters, volunteers or docents


Create or expand interpreter, docent or volunteer program


Support a research and evaluation program, including conducting surveys


Develop key management plans (e.g., comprehensive interpretive plan, emergency plans, collections management plans)


Other; please specify


No Organizational Development




7.3. Which of these groups were\are directly served by this project? (select all that apply)



Paid staff; please describe


Unpaid staff (docents, interns, volunteers); please describe


Adult learners


Seniors


Parents


Youth


Teachers


Students


Administrators


Professionals


Residents of particular neighborhood or community/geographic area within a community


Policymakers


Persons with disabilities


Low income citizens


Specific racial/ethnic communities


Families


Not group-specific


Other; please specify



7.4 Age groups served by your grant project (select all that apply)



Preschool


Grades K-5


Grades 6-8


Grades 9-12


Post high school/Young adults (18-25)


Adults (26-64)


Seniors (65 and older)


Multi-age/Family


Other, please specify


8. Status of Activities

Status of activities –the list of activities that grantees check off in the previous section will be populated to this section.


OPEN GRANTS ONLY

8.1 What is the completion status of the project’s activities?


Example:


Completion Status

<populated with specified activities>

Will not do as planned

Still in progress

Completed

Don’t Know

Afterschool programs





Curriculum guides





Train interpreters or docents





Establish a docent program and recruit docents





Implement a research and evaluation program, including conducting surveys






8.2. Were any other activities conducted or planned?



Yes


No


If yes, please list activities and current status of each below


Completion Status

Other activities:

Will not do as planned

Still in progress

Completed


























CLOSED GRANTS ONLY

8.1 What is the completion status of the project’s activities?


Example:



<populated with specified activities>

Did not do as planned

Completed

Don’t Know

Afterschool programs




Curriculum guides




Train interpreters or docents




Establish a docent program and recruit docents




Implement a research and evaluation program, including conducting surveys





8.2. Were any other activities conducted or planned?



Yes


No


If yes, please list activities and current status of each below (e.g., Completed, Still In Process



Other activities:

Did not do as planned

Completed

Don’t Know


















SYSTEM NOTE/NOT DISPLAYED:

IF this project is in progress OR recently completed AND the museum has no additional MFA granted projects EXIT


OTHER COMMENTS:





As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.

If interested please fill out the following information:

Name:

Title:

Organization:

Address: City State Zip


Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.



SYSTEM NOTE/NOT DISPLAYED:

If this project is completed AND funded under the 3 earlier grant categories, THEN respondent will continue on to Section D: Impacts and Reflections.


If this project is in progress OR recently completed AND the museum has more MFA granted projects THEN the respondent returns to Section C and is asked questions about the next funded project.


PART D: IMPACTS and REFLECTIONS

COMPLETED BY MFA GRANTEES WITH COMPLETED GRANTS UNDER THE EARLIER 3 GRANT CATEGORIES 2004-2007

Structured at the museum\project level (n ~ 350 with ~ 22 with 2 completed projects)



VI. INDIVIDUAL Grant Project Impacts

9. Project Impacts


    1. Please review the list of impacts this MFA project may have had on the organization or community.

Select each impact resulting from the MFA project.


ORGANIZATIONAL IMPACTS (select all that apply)


Programming


Grant-funded programming continued beyond grant


Sustained grant-funded programming


Did not sustain grant-funded programming


Sought additional funding to sustain grant-funded programming


Developed related programming to expand value of grant-funded program


Expanded our range of possibilities for programs or exhibitions


Other; please describe:


No changes in programming/not applicable to grant


Organizational Capacity


Enhanced staff capacity in program development


Enhanced staff capacity in creating new kinds of exhibitions


Enhanced staff capacity in reaching new or larger audiences


Enhanced staff capacity in forming external partnerships


Enhanced staff capacity in working internally across departments


Enhanced staff capacity in using outcomes based evaluation


Improved staff cohesion and commitment to mission


Greater alignment of staff responsibilities with mission


Greater board involvement


Helped institution fulfill its mission (e.g., meet certain benchmarks)


Increased ability to attract outside funding


Decreased ability to attract outside funding


Other; please describe:


No effect on our organization/not applicable to grant






COMMUNITY IMPACTS (select all that apply)


Audiences


Reached new audiences (e.g., youth, families, minorities)


Increased commitment by existing audiences


Changes in regular audience participation


Increased audience access (e.g., expanded hours, mobile programming)


Other; please describe:


No effect on audiences/not applicable to grant


Community Profile


Changed institutional identity


Strengthened museum’s public image


Raised institution’s prestige in its community


Increased local media coverage of institution’s activities


Increased visibility of institution as a center of community learning


Other; please describe:


No change to our profile/not applicable to grant


Partnerships


Improved skills in building partnerships


Strengthened ongoing partnerships


Developed new partnerships


With schools or Institutes of Higher Education


With youth organizations


With other museums/historical societies


With other community organizations


Developed new types of partnerships


Decreased ability to attract partners


Increased ability to attract more partners


Other; please describe:


No effect on partnerships/not applicable to grant


IMPACTS DUE TO THE PARTNERSHIP(S) (select all that apply)



Brought in new audiences


Brought in new resources not normally available to our organization


Brought in new staff, paid or unpaid


Brought in new volunteers or memberships


Raised community awareness of our organization


Other; please describe:


No effect from partnerships/not applicable to grant




9.2. Are there any other impacts not covered above? If yes, Please describe

__________________________________________________________________________


9.3. If the grant project enabled new programming, to what extent has your organization been successful in continuing the programs?



Not at all successful


Somewhat successful


Very successful


Not applicable


9.4. If the grant enabled you to bring in new audiences, to what extent has your organization been successful in sustaining these new audiences?



Not at all successful


Somewhat successful


Very successful


Not applicable


9.5. Was this MFA grant from IMLS larger, smaller, or about the same as other grants you may have received?



Larger


Smaller


About the same


Not applicable


9.6. Were the MFA grant funds sufficient to complete the planned project activities?


No


Somewhat


Yes


Don’t Know


SYSTEM NOTE/NOT DISPLAYED :

IF this project ended less than 3 years ago AND the museum has no additional MFA granted projects EXIT


OTHER COMMENTS:





As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.

If interested please fill out the following information:

Name:

Title:

Organization:

Address: City State Zip


Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.


SYSTEM NOTE/NOT DISPLAYED:

If this project was completed 3 or more years ago THEN respondent will continue on to Section E: Long-term Impacts.


If this project is in progress OR recently completed AND the museum has more MFA granted projects THEN the respondent returns to Section C and is asked questions about the next funded project.


PART E: LONG-TERM IMPACTS

COMPLETED BY grantEEs WHERE PROJECT WAS completed 3 or more years ago

VII. Individual Grant Project Long-Term Impacts


10.1. Please describe any long-term impact(s) that the MFA project has had on your organization.



10.2. Please describe any long-term impact(s) that the MFA project has had on your community or community relationships.



10.3. Please describe any UNEXPECTED long-term impact(s) that the MFA project has had on your organization, community or community relationships.

SYSTEM NOTE/NOT DISPLAYED :

IF this project is the only/last one completed under the 3 earlier grant categories THEN GO TO EXIT


If there is another MFA project completed under the 3 earlier grant categories THEN the respondent returns to Section C and is asked questions about the next funded project.


EXIT

OTHER COMMENTS:





As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.

If interested please fill out the following information:

Name:

Title:

Organization:

Address: City State Zip


Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.





Appendix B


Case Study Outline

Case Study Protocols

Case Study Outline

Museums for America Evaluation

Overview

Case studies will provide a nuanced view of MFA impacts through close study of a select group of completed MFA grants. These case studies will be based on a series of qualitative data collection activities conducted at selected institutions, and will allow for targeted and individual inquiry into the factors producing project impacts, both within the institution and in the community.

The case studies will be based on interviews and focus groups with museum administrators, program staff, and participants as appropriate to each program, as well as extensive review of grant applications, annual reports and other documentation, media coverage, and first-hand observations. Case studies will provide an opportunity for investigating the connections among institutional and community characteristics, program implementation choices, and impacts.

Two-person teams of RMC staff will conduct each site visit. Each visit is expected to take one-and-a-half days to complete, with extensive pre-visit planning contact, and follow-up telephone interviews as needed.

Site Selection

Six case studies will be conducted, and will include a mix of each of the three MFA grant categories (Serving as Centers of Community Engagement, Sustaining Cultural Heritage, and Sustaining Lifelong Learning). In seeking maximum variability among case studies, selection criteria may include:

  • A range of project activities

  • Institution size, type and location variety

  • Project director and staff still employed at the museum

Selection of the case study sites will be made in close collaboration with the IMLS. As part of this process, RMC will review relevant grantee reports. RMC will make the final selection based on recommendation of 10-12 projects/institutions by IMLS program officers.

Site selection will conclude with a discussion between IMLS and RMC about the significant factors leading to the particular selections, i.e. the successes or outstanding features of each of the selected grant experiences.

Visit Preparation

Pre-visit discussions will be conducted with the project director and/or other appropriate institution representation to identify optimal interview and focus group subjects and to develop a schedule for the visit that will use the available time to maximum effect.

Possible on-site interviews (approximately six per site) will be conducted with the institution’s project director, executive director, and management or administrative person, project staff, partners, and/or audience members. These may include both paid and unpaid staff and volunteers. Focus group subjects will be drawn, again as appropriate, from community and audience members, education and public programming staff, volunteer staff and interns, community activists, and representatives of school and other partner organizations. RMC will also review available documentation of program effects, such as attendance patterns and histories.

Protocols

RMC will develop protocols for the following categories of interviewees:

  • Project Manager(s)

  • Museum Leadership

  • Project Staff/Educators

  • Partners

  • Community/Audience Members

The interview and focus group protocols will be tailored to the specifics of the institution and grant project prior to each site visit.

Case Study Protocols


Project Manager Interview


Interviewee Name: _______________________ Title: ______________________


Institution: ___________________________ Date:_________ Interviewer: ____


Application Process

  1. Could you share with us the history of this project? How did the idea originate?

  2. How did you decide to apply for the IMLS MFA funding? Did you apply more than once? If you applied more than once, did you get feedback and was it helpful?

  3. How would you characterize the application process compared with other grants you have applied for? How does it compare to other Federal grants you’ve applied for?

  4. Did you find the application process valuable? If yes, what was most valuable about it (e.g. what did you learn about your institution, project development, etc.)?

  5. What was most challenging about completing the application?

  6. How did you approach the cost-sharing requirement?


Project Implementation and Sustainability

  1. To what extent was the project successful in advancing your institution’s mission? How does this project align with your institution’s mission?

  2. What aspects of the project were most successful?

  3. What challenges did your institution encounter in implementing the project?

  4. What was the impact or value to your institution of partnerships developed through the project? Have these partnerships been maintained, and if so, how?

  5. What does sustainability mean to you in the context of this project? (e.g., exhibits or programming continue, new audiences engaged, continuation of new offerings, access to collections or other resources, etc.) How was this project or component parts supported after the grant period?


Community Impacts

  1. How did the target audience and/or your existing audience respond to the program? What trends did you observe over time?

  2. How has this project changed the museum’s reputation in the community?

  3. How has the community’s involvement with your museum continued or grown as a result of the program?


Organizational Impacts

  1. In what areas, if any, has the capacity of your staff and/or institution been improved (e.g., growth in knowledge, skills, efficiency, programming)? To what extent were these capacity changes the result of new staff, professional development, strategic planning, budgeting changes, or new technologies?

  2. To what extent, if any, did the project support the development of new relationships within the institution, such as across departments?

  3. Have there been any outcomes of this improved capacity? If so, what? (e.g. additional new programs or initiatives)

  4. Did the project result in any changes in how individuals in the organization perceive the institution’s mission and/or value to the community?


Planning and Evaluation

  1. To what extent were the outcomes-based planning and evaluation requirements familiar to your or your institution?

  2. Do you view planning and evaluation differently as a result of this project? Please explain your response.

  3. To what extent are you integrating outcomes-based planning and evaluation in subsequent projects?


Summary

  1. Overall, what would you say is the most important impact of having received this grant from IMLS?

  2. Were there any surprises or other impacts you did not expect or foresee?

  3. How would you describe the value of the MFA program for institutions like yours?

  4. Any other comments?



Project Staff


Interviewee Name: _______________________ Title: ______________________


Institution: ___________________________ Date:_________ Interviewer: ____


  1. Describe your involvement with this project. What was your role?

  2. How would you describe the implementation of this project? Did it go smoothly?

  3. What new skills or knowledge did you gain from this project?

    1. How have you been able to use these skills or knowledge in subsequent projects?

  4. Did the grant have an impact in how you do your work? If so, explain.

  5. Are there any other ways in which the grant changed how you do your job, your responsibilities, or improved your ability to do your job?

  6. To what extent were you involved in evaluation aspects of the project?

    1. If you were involved in these, were the IMLS requirements valuable?

    2. What challenges did you face in meeting the evaluation requirements? How might these requirements better fit the needs of the MFA projects?

    3. What did you learn from the experience?

  7. Are there other ways in which this grant affected your institution’s ability to serve the public?

    1. What kind of community feedback have you received about the project?

  8. What other aspects of the project have been sustained since the grant period ended? Please describe. How were these projects supported after the grant period?


Summary

  1. Overall, what would you say is the most important impact of having received this grant from IMLS?

  2. Were there any surprises or other impacts you did not expect or foresee? Please describe.

  3. How would you describe the value of the MFA program for institutions like yours?

  4. Any other comments?





Museum Leadership Interview


Interviewee Name: _______________________ Title: ______________________


Institution: ___________________________ Date:_________ Interviewer: ____


Overall Impacts

  1. Please describe the value of this grant for your institution in terms of your mission (e.g., offerings, audience, efficiency).

    1. How does this grant relate to what you consider your institution’s strength in serving the community?

    2. How does this grant relate to what you consider its greatest challenges?

  2. What aspects of the project were most successful?

  3. What challenges did your institution encounter in implementing the project?

  4. How has this project impacted what your institution does?

    1. [If a conservation grant, probe about conservation practices after grant. What changed?]

    2. [If an outreach grant, probe about outreach practices after grant. What changed?]

    3. Etc.


Community Impacts

  1. Did this project change the museum’s reputation in the community? If so, how?

  2. How has the community’s involvement with your museum continued or grown as a result of the program?

  3. Has the museum’s level of engagement with the community changed over time? If so, how? To what do you attribute that change?


IMLS Grantee

  1. Are you regular applicants to IMLS or other federal funders? Was this the first IMLS grant your institution has received or one of many? First MFA grant?

  2. How important was receipt of this funding for the institution?

    1. How did the size of the grant compare with other grants received?

    2. What kind of prestige is conveyed by receipt of the grant?

    3. Were you able to leverage the funds or the prestige in additional ways?

  3. In what ways did the application process or the implementation of the grant impact how you understand your mission or how you understand how you can achieve it?

  4. How would you describe the value of the MFA program for an institution like yours?

  5. What challenges have you faced in sustaining the project?

  6. What new opportunities for sustaining the project have emerged?


Planning and Evaluation

  1. To what extent were the outcomes-based planning and evaluation requirements familiar to your or your institution?

  2. Do you view planning and evaluation differently as a result of this project? Please explain your response.

  3. To what extent are you integrating the outcomes-based planning and evaluation by IMLS in subsequent projects? Please describe current efforts.


Community Member Interview


Interviewee Name: _______________________ Title: ______________________


Institution: ___________________________ Date:_________ Interviewer: ____



  1. How were you connected to the IMLS MFA grant project (attended exhibit or program, community partner, volunteer or docent at the museum, etc.)?

  2. Have you had any dealings with the museum in the past? If yes, what types of activities or programs did you conduct with the museum?

  3. What did you like most about this project?

  4. What did you learn from your involvement in this project?

  5. What value did this project provide to the community?

  6. Did the project have an impact on your impression of the museum? If yes, please describe.

  7. Did the project have an impact on your attendance at museum events or involvement in museum activities?

  8. How would you characterize the relationship between the museum and the community?

  9. Do you know of other community groups that participate in programming with the museum? If so, what types of activities are they engaged in?

  10. Has the museum’s level of engagement with the community changed over time? If so, how? To what do you attribute that change?






Partner Interview


Interviewee Name: _______________________ Title: ______________________


Institution: ___________________________ Date:_________ Interviewer: ____



  1. What was your institution’s role in the MFA project? To what extent were you involved in developing the project? To what extent were you involved in the grant application process for this project?

  2. What was your interest in participating in this project? In what ways was this project valuable for your institution?

  3. Was this the first time you worked with the museum? If so, did this project represent a change in the types of collaboration you have engaged in previously?

  4. How was the partnership structured?

  5. How has your relationship with the museum changed as a result of this project?

  6. Have you continued to work together as a result of the project? If no, why not? If yes, is this work on the same project or on a new project?

  7. Do you anticipate future partnerships with the museum? If so, what kinds of partnerships you would be interested in pursuing? What aspects of the partnership would you continue? Do differently?

  8. How would you characterize the relationship between the museum and community organizations such as yours?

  9. Do you know of other organizations that participate in programming with the museum? If so, what types of activities are they engaged in?

  10. Has the museum’s level of engagement with the community changed over time? If so, how? To what do you attribute that change?



File Typeapplication/msword
AuthorJJB
Last Modified ByKMILLER
File Modified2010-05-12
File Created2010-04-16

© 2024 OMB.report | Privacy Policy