Supporting Statement B, Improving Assessment of Student Learning in the Arts

Research Study: Improving the Assessment
of Student Learning in the Arts

PART B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent Universe

The study population for this data collection is the universe of members from 16 specific arts agencies, education agencies, cultural organizations, and research organizations. A census (all members of this defined universe) will be asked to participate in this data collection. This respondent universe consists of 130,244 members from these 16 agencies/organizations. We anticipate a response rate of 40%.

B2. Procedures for Data Collection

Respondents will complete the web-based survey online. Membership agency representatives will send an email to their entire membership, and each member (i.e., potential respondent) will be given a survey access code. The email will indicate participation is voluntary and will include the purpose for the survey, a link for accessing the survey, and contact information should respondents have any questions or want additional information about the data collection.

During the data collection period, WestEd will also establish a dedicated email address to allow respondents to ask questions, report problems, or request assistance. The e-mail address will be featured on all pages of the survey and in any related correspondence to respondents.

B3. Methods to Maximize Response Rate

We anticipate a response rate of 40% for the data collection, and will provide information and support before and during the data collection process to facilitate participation. We will provide advance notice of the survey via email prior to data collection to allow respondents to gather the necessary information. The supporting documents that accompany this advance notice will also explain the purpose and importance of the survey for the study.

During data collection, WestEd will provide support as described in the section above, and also will track the response rate as respondents login and complete the survey. The electronic administration of the survey allows for instant access to the data and also allows for instant identification of nonrespondents. The membership agencies will send targeted reminder emails to nonrespondents two weeks after the initial administration, and a second targeted reminder email one month after initial administration. We will provide the text for the email reminders, and agencies can modify them as needed.

Throughout the data collection period, WestEd will provide regular updates to our staff on the response rate and the status of the data collection. In cases where the response rate is unexpectedly low, or where issues are identified regarding the validity of the initial responses, WestEd will convene a conference call with our staff to make any necessary amendments to the data collection plan.

B4. Tests of Procedures or Methods

The survey instrument went through a series of reviews before being submitted for OMB approval. WestEd worked closely with our staff throughout the design and revision process, and enlisted key leaders in the area of arts assessment to review the survey as well.

WestEd also pilot tested the survey with a group of three to four respondents to assess item comprehension, the effectiveness of the proposed strategies for gaining participation, and the length of time (i.e., 30 minutes) for respondents to answer the survey questions. Such information helped determine the actual burden associated with the survey.

B5. Contractor Information and Use of Consultants

The SLA study is led at WestEd by Treseen McCormick, a Senior Project Director in the Evaluation Research program. McCormick and her team of WestEd staff will be responsible for developing the data collection instruments, collecting and analyzing the survey data, and reporting the results to our staff.

Improving the Assessment of Student Learning in the Arts page 2