3/19/2013 Page -
Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.
The National Endowment for the Arts is the primary Federal agency charged with promoting the arts and arts education for the American people. Established in 1965, the NEA supports artistic talent and creativity in the United States and provides Americans with rich and diverse opportunities to experience art. The Arts Endowment fulfills its legislative mandate primarily through the award of grants and cooperative agreements to nonprofit organizations. In addition, the NEA awards fellowships to creative writers and translators, and lifetime achievement awards as authorized. The Agency also exercises leadership through targeted support of key initiatives, research and evaluation projects, publications and convenings, and domestic and international partnerships.
In October 2010, the National Endowment for the Arts (NEA) released a strategic plan establishing a new set of goals, outcomes, and performance measures for the Agency. The plan defined the agency mission, long-term goals, strategies planned, and the approaches to be used to monitor progress with respect to the Endowment's grant-making activities.
Under the plan, a critical NEA goal is to engage the public with diverse and excellent art. By awarding organizational grants in the category of art-making, and grants to individuals where permitted, the NEA harnesses the ingenuity of American artists and is a leader in creativity and innovation. By funding grants, NEA provides the following services: across a diverse spectrum of artistic disciplines and geographic areas: engage the public with works of artistic excellence; provide Americans of all ages with opportunities for lifelong learning in the arts; and strengthen communities through the arts.
However, the ability to engage people successfully with art is beset by a number of complicating variables, not least of which is: what does success look like? In its Strategic Plan, the NEA developed a number of performance measures to help address this question. One such measure is the percent of adults who report being affected by NEA-funded art exhibits and performances. In order to compute this measure, the NEA intends to conduct audience surveys to learn how art-goers respond to NEA-supported artworks being performed and displayed. Data reflecting how the American people are affected by NEA-funded art exhibits, performances, and screenings will help the NEA assess progress toward the goal of engaging the public with diverse and excellent art.
The ArtBeat survey, a survey of audiences at NEA-funded arts events nationwide, will enable the Agency to gauge the quality, depth, and nature of audience responses to NEA-funded artworks. The intent of the survey is to ascertain the proportion of audience members at NEA-funded events who report being intrinsically affected and who report developing appreciation for people of backgrounds and cultures different from their own. The items measuring the intrinsic impact concept have been tested in a preliminary survey, conducted by WolfBrown in FY12. WolfBrown, who are experts in surveying audiences, assessed a number of questions and identified the ones that best capture intrinsic impact. The survey included in this Information Collection Reqeust builds upon the WolfBrown findings and uses their questions, with some minor modifications.
The survey will be open to all audience members attending an NEA-sponsored event. The survey will ask respondents to report the three most memorable emotions associated with the event, whether they were captivated by the event, whether they lost track of time, and if they developed appreciation for people different from themselves during the event. To help describe the audience members, the survey also will ask about respondents’ experience with different art forms, about the type of ticket they held for the event, and how frequently they attend art events. Additionally, the survey will ask respondents for their gender, level of education, ethnicity, race, and age. The demographic information will allow accounting for potential non-response biases by constructing an appropriate set of weights (see Attachment B for more information) Also, the demographic information will assist in evaluating the extent of the NEA sponsored events with live audiences reaching the underserved populations.
Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.
The information will be analyzed by the NEA Office of Research & Analysis to assess performance on the strategic goal of engaging the public with diverse and excellent art. The data will be aggregated by disciplines and geographical areas to help the NEA better understand how particular types of NEA-sponsored events achieve intrinsic impact and develop inter-cultural appreciation among audience members.
This understanding will serve both internal performance measurement and external reporting purposes. Internally, the information will be used by the NEA discipline and program offices that review applications and manage grants. The deeper understanding that the ArtBeat survey provides of the ways in which NEA-funded events affect audiences will aid these offices in conducting outreach to inform the public about grant opportunities, developing and refining guidelines for grant-making, selecting expert panelists to review applications, and selecting applications most likely to achieve the NEA’s goal of supporting excellent art.
Externally, the information will be used in the NEA's annual performance reporting to provide information on the agency's goal of engaging the public with diverse and excellent art. The information will also be used in reevaluation of and planning for the future iterations of the NEA Strategic Plan.
In addition to these NEA-focused benefits of the survey, the survey will benefit participating grantees. The NEA will make available to each participating grantee, at no cost, the raw data and a summary statistical analysis, including graphics, of audience responses to their events as well as aggregate-level ArtBeat survey data. It is expected that grantees will use the information to better understand their audiences and the impact that their event had on those audiences. This information will inform grantees’ future programming and organizational decisions. Since the NEA is not making participation in the ArtBeat survey a requirement for organizations that receive Agency funding, it is expected that the offer to provide this information will boost participation rates among grantees.
The initial stage of this information collection is a pilot survey. Its purpose is to assess the timeliness of responses, success of survey-promotion materials, and whether some questions result in a higher degree of respondent drop-out then other questions measuring similar concepts. The pilot survey will occur in FY14, while the two subsequent years, FY15 and FY16, will see a full roll-out of the ArtBeat survey. It is expected that the survey instrument will be the same in FY15-FY16 as in the pilot year in FY141. However, if the pilot process reveals the need for significant changes to the survey instrument after the pilot year, the NEA will submit a new Information Collection Request or Non-Substantive Change Request, as appropriate.
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.
The collection of information will be fully web-based, and therefore all responses will be completed through electronic submission.
Each grantee will be provided a unique link to an NEA website dedicated to the survey of their particular audiences. Participating grantees will be encouraged to promote the survey link to their audience members via posters, flyers, program inserts, etc. By going to the website, audience members will be given additional information about the information collection, an OMB control number, and an assurance of anonymity of their responses and of the absolutely voluntary nature of the survey. Once at the website, survey participants will be redirected to a survey-taking platform, complete a short survey, and have an opportunity to view a summary of data collected from audiences attending NEA-funded live events.
The primary reason for collecting survey responses via the web is to reduce burden on the public. Web-based survey participation is less burdensome and costly than alternatives involving pencils, paper, postage, mailing, and data entry. An electronic system allows for easier addition of respondents, easier importing of data (including narrative) into analysis software, and easy respondent access to surveys through smartphones, tablets, or computers in a relaxed, informal manner. As an increasing share of the population gains easy access to the Internet, it is also very unlikely that certain demographics of American audiences will be excluded from participating.
A web-based survey also reduces burden on participating NEA grantees. Each grantee will be provided with a unique link to an NEA survey website and with templates for posters, flyers, and programs that the grantees will distribute among audience members in order to promote the ArtBeat effort. The only costs that grantees will incur are the costs associated with bulk printing of promotion materials. In contrast, grantee staff would have to administer any paper-based survey, effectively excluding small grantees with fewer resources.
Finally, by making the survey web-based, it becomes possible to survey every member of the audience. Because there are no additional costs associated with adding participants, every person attending an NEA-funded event will have a chance to answer the ArtBeat questionnaire. Thus, the electronic nature of ArtBeat provides the most efficient mechanism for the NEA to capture responses from a sample that is as representative as possible.
4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.
The information to be collected is specific to the NEA strategic goal of engaging the public with diverse and excellent art and to NEA-funded events. Only audiences attending performances/screenings/exhibits funded by NEA grants can assist in tracking the progress toward achieving this goal. Additionally, although there are other organizations (primarily theaters) that conduct surveys gauging audience reaction, such surveys may not focus on the outcomes in which the NEA is interested, are not comparable across organizations, and are generally limited to learning about a particular performance.
5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.
The NEA takes very seriously its responsibility to minimize burden on grantees, some of which are small arts organizations, and designed this project with that goal in mind. First, as discussed above, by designing a web-based survey, the Agency has eliminated hundreds of hours of labor that would have been required to administer a paper-based on-site survey.
In addition, the NEA has taken actions to minimize the impact on grantees of participating in this web-based survey. The Agency has created artful templates for posters and flyers to advertise the survey to audiences, removing the burden for creating such advertising materials from the grantees. The agency has also created a toolkit for grantees containing advice on best approaches for communicating with audiences about the survey. And the NEA will provide to grantees the findings from a rigorous analysis of the survey, removing from grantees the need to clean and analyze the data in order to benefit from the information the survey provides.
Finally, the NEA has made participation in the ArtBeat survey voluntary, both in the sense that grantees are not required to participate in the project and in the sense that audience members are not required to respond to the survey.
6. Describe the consequence to federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.
This survey will provide the NEA with its only quantitative measure of the intrinsic impact NEA-funded live events and performances have on the American public. Without this measure, the NEA will have no methods for analyzing and assessing the impact of its program and policy choices on the intrinsic value that NEA-funded live performances provide to the public.
An inability to conduct the collection would also make it impossible to report on one of the performance measures outlined in the NEA Strategic Plan under the goal “Engage the Public with Diverse and Excellent Art”: the percent of adults who report being affected by NEA-funded exhibits and performances.
Conducting the collection less frequently or with fewer grantees would not only impede the Agency’s ability to track impact, but would also deprive grantees of an opportunity to learn more about the impact of their exhibits and performances. Every grantee that is funded to perform or present a live event or exhibition can volunteer to participate in the ArtBeat survey initiative. The NEA plans to make the ArtBeat survey available to all such grantees. The larger is the number of survey respondents, the more beneficial the feedback to grantees will be. The survey is voluntary for both grantees and audience members, which means that high participation and response rate over time will be an indication of the benefit that grantees and audience members obtain from their participation.
7. Explain any special circumstances that would cause an information collection to be conducted in a manner:
requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statue or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.
There are no circumstances which would require the collection of information in the manner discussed above.
8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice, required by 5 CFR 1320.8 (d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.
Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.
Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years--even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.
A notice was published in the Federal Register (document 2013–03937) on February 21, 2013, to solicit comments on the “Proposed Collection of Information; Comment Request” prior to submission of this OMB clearance request. No public comments were received at the NEA in response to this notice.
In FY12, prior to making this information collection request, the NEA contracted with WolfBrown, experts in surveying arts audiences, to conduct a preliminary paper-based pilot study with 26 NEA grantees2. Results of the study included recommendations regarding the methods of data collection, the type and phrasing of questions, and feedback from grantee organizations. The decision to reduce the burden of the information collection by making the ArtBeat survey a web-based project was a result of lessons learned during the first pilot. In addition, the survey instrument was shortened based on information gathered during the WolfBrown pilot, and the toolkit, which is to be sent to grantees that participate in the ArtBeat survey, incorporates feedback and lessons learned during that pilot.
The Arts Endowment will collect additional feedback on the implementation of the web-based survey platform during the first stage of this information collection. During this time, staff will be available to consult with grantees participating in the survey through webinars and calls, grantees will be asked for feedback on the burden and value of the process, and audience members participating in the survey will be invited to submit feedback and questions to the NEA Office of Research & Analysis. The NEA is considering whether to hold a webinar as a service to share survey results with the participating grantees and to answer any questions.
Finally, a second Federal Register Notice (Document No. 2013-11801) was published on May 17, 2013.
9. Explain any decision to provide any payment or gift to respondents, other than reenumeration of contractors or grantees.
Not applicable. The Arts Endowment does not provide any payments or gifts to respondents.
10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.
No uniquely identifying information about grantees will be collected. Therefore, while responses to the survey will not be confidential, the identities of respondents will not be collected. This will be communicated to the respondents on the survey instrument.
11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
No questions of a sensitive nature are included in the information collection.
12. Provide estimates of the hour burden of the collection of information. The statement should:
Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.
If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens.
Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14.
ESTIMATED BURDEN IN HOURS
It is planned that in FY14, in the pilot phase of the information collection, only three grantees from each of nine NEA disciplines will participate in the ArtBeat survey, for a total of 27 grantee participants. The NEA grantees with live audiences primarily belong to the following NEA arts disciplines: Dance, Folk & Traditional Arts, Literature, Media Arts, Museums, Music, Opera, Theater & Musical Theater, and Visual Arts. To make sure that the survey in the pilot phase is not overrepresented by a particular discipline, at this stage, between 9 and 11 randomly chosen grantees will be contacted and invited to participate. Of these grantees the first three agreeing to participate will be included in this information collection. Therefore, the selection of grantees participating in the FY14 pilot phase is actually a stratified sample of NEA grantees, with the NEA disciplines being the strata from which a random subset of the grantees will be selected for an invitation in the ArtBeat survey.
In subsequent years, the Agency expects to make the survey available to all grantees with exhibitions or live performances from these nine disciplines. Based on the assumption that 67% of invited grantees will participate, the NEA estimates that 794 grantees with live audiences will invite their attendees to participate in FY15 and FY16. This is how the number has been computed:
Using previously submitted grantee final reports from awards that were concluded between FY10-FY12, count the total number of grantees per discipline per year. The results are shown in Table 1. The sum of grantees with audiences is 593 per year.
Table 1: Historical Average Number of Grantees per Year, FY10-FY12
At a participation rate of 67%, the number of grantees with audiences participating in any given year is 397. Thus, the total number of grantees expected to participate in the ArtBeat survey across FY15 and FY16 is 794.
These 794 grantees in FY15 and FY16, plus 27 grantees in FY14 (three grantees from each of the nine disciplines), generates a total estimate of 821 grantees participating in the ArtBeat survey.
The NEA estimates that it will take each grantee three hours to prepare to implement the survey, including time to become familiar with survey promotion materials provided by the NEA, time to print and position these promotion materials, and time to participate in a voluntary NEA-led webinar. This time estimate generates the total burden estimate for participating grantees shown in Table 2:
Table 2: Estimated Grantee Burden per Year, FY14-FY16
To compute the expected number of audience members we take the following steps:
Using previously submitted grantee final reports from awards that were concluded between FY10-FY12, estimate the median number of audience members for each NEA discipline. The results are shown in Table 3:
Table 3: Historical Median Audience Size per Eligible Grant, FY10-FY12
Compute the number of grantees expected to participate in the ArtBeat survey between FY14-FY16 by NEA discipline. Begin with Table 1 above (Average Number of Grantees Per Discipline Per Year, FY10-FY12).
For FY14, the pilot test year, count three grantees per discipline.
For FY15-FY16, multiply each the number of expected grantee participants in each discipline by the estimated grantee participation rate (67%). The results of this calculation are shown in Table 4:
Table 4: Estimated Number of Grantee Participants per Year, FY14-FY16
Based on our review of the literature and on the first phase pilot test conducted by WolfBrown in 2012, we assume a 10%3 response rate among audience members.
We also assume each survey takes 5 minutes to complete.
To calculate the total burden for audience members, we multiply the estimated median audience size (Table 3) by the expected number of participating grantees for each discipline (Table 4). We take 10% of this total for each discipline based on our estimated response rate and multiply the result by 5/60 (the time each survey takes to complete) to calculate the total estimated burden hours. The result of these calculations are shown in Table 54.
Table 5: Estimated Audience Burden Hours per Year, FY14-FY16
The total of the estimated audience burden across all disciplines is 41,504 hours. When added to the estimated burden hours on grantees (Table 2), the total estimated burden on the public is 43,967 hours.5
ESTIMATED COST
It is estimated that grantees choosing to participate in the ArtBeat survey will incur only the costs of printing requisite survey promotion materials. Using the uprinting.com rates, we estimate a cost of $22.22 to print out two posters that will contain the invitation to participate in a survey. We assume that grantees will print no more than 2 posters.
We assume grantees will also print an average of 4,000 postcards/flyers to include in their program materials at a cost of approximately $134.17. Survey promotional materials printed out directly on programs would result in no extra costs to grantees. Thus, the estimated cost per grantee is approximately $156.
The total estimated cost for all grantees, broken down by discipline in Table 6, is approximately $127,771.
Table 6: Total Estimated Cost by Discipline, All Years (FY14-FY16)
There are no costs associated with audience members participating in the survey.
13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet.)
The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.
If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.
Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.
Not applicable. There are no annual costs to respondents or record keepers resulting from this collection of information.
14. Provide estimates of annualized cost to the federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies may also aggregate cost estimates from Items 12, 13, and 14 in a single table.
The NEA estimates that 2 NEA personnel will spend two full work-days organizing and hosting a webinar once every FY to serve as a forum for information exchange and to address potential concerns by grantees participating in the ArtBeat survey. Additionally, the Agency expects that two NEA personnel will spend two full work-days responding to grantee questions every quarter in FY14-FY16.
The agency expects to require spend additional time developing the infrastructure to manage the project, including developing a template for reporting to grantees, developing email templates. These efforts will minimize the ongoing cost to the government of implementing the project. The total burden on NEA personnel in FY14-FY16 is estimated to be 410 hours. At a salary of $48.35 per hour (it is assumed that government employees participating in the ArtBeat Survey initiative on the average receive a GS-13 step 5 hourly salary in Washington DC), the total cost to the US government is $19,801.74. There are no additional costs to the government associated with the ArtBeat survey.
15. Explain the reasons for any program changes or adjustments reported on the burden worksheet.
Not applicable.
16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.
Not applicable.
17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.
Displaying the expiration date for OMB approval of the information collection is appropriate. The expiration date will be displayed on all application guidelines (including each form) and reporting requirements.
18. Explain each exception to the certification statement identified in “Certification for Paperwork Reduction Act Submissions.”
Not applicable. There are no exceptions to the certification statement.
1 One exception to this is Question 9 in the survey instrument. As explained in the copy of the survey attached to this Information Collection Request, two versions of this question are being tested in the pilot to determine whether one results in a higher survey drop-out rate.
2 The extensive literature review regarding the intrinsic impact constructs could be accessed at http://www.nea.gov/research/Audience-Impact-Study-Literature-Review.pdf. There, WolfBrown, the authors of the review, provide in-depth descriptions of the current best practices in capturing individual-level/intrinsic impact of afts. Generally, it is linked to the sense of engagement and flow that, rather than reflecting the concepts of happiness or liking/disliking an activity, point toward the feeling of satisfaction from being involved in a particular endeavor.
3 A popular web-based provider of survey platforms, surveygizmo.com, indicates that customer satisfaction surveys, which are similar in purpose and target audiences to the ArtBeat Survey, on the average achieve a 10-15% response rate (http://www.surveygizmo.com/survey-blog/survey-response-rates/).
The already mentioned audience survey study conducted in FY12 surveyed audiences of 26 grantee organizations. The three most successful in-person paper survey delivery methods resulted in the average response rate of approximately 38%. David Nulty (2008, Assessment & Evaluation in Higher Education, The Adequacy of response rates to online and paper surveys: what can be done?) indicates that the internet-based surveys response rates are much lower that the paper-based surveys. So the response rate by the audiences using the web-based approach would be much lower than 38%. Deutskens et al (2004, Marketing Leters, Response Rate and Response Quality of Internet-Based Surveys: An Experimental Study), indicated that a relatively long web survey without incentives resulted in a 12.7% response rate.
4 The total number of respondents expected to take the ArtBeat survey in FY14 is 21,830. The total number of respondents expected to take the survey in FY15 is 240,753. The total number of respondents expected to take the survey in FY16 is 240,753. The total number of respondents expected to take the survey in FY14-FY16 is 503,336.
5 Note that, because of the way the ROCIS PRA system works, agencies are required to provide the annual number of responses for each IC submitted under the ICR. Because the pilot test numbers are only applicable to the pilot test year (FY14), we assume 1/3 responses per year per respondent. And because the full implementation numbers are only applicable to the full implementation years (FY15-FY16), we assume 2/3 responses per year per respondent. This is the only method that allows us to generate numbers in the ROCIS system that are consistent with the method described here for calculating total burden.
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | Supporting Statement |
Author | IMD |
File Modified | 0000-00-00 |
File Created | 2021-01-29 |