B. Collection of Information Employing Statistical Methods
This submission requests the clearance of additional data elements, materials, and procedures for the collection of undergraduate transcripts for the full-scale implementation of the 2004/09 Beginning Postsecondary Students Longitudinal Study (BPS:04/09). The addition of transcripts should make it possible to answer additional policy-relevant questions related to the educational experiences of first-time beginning students.
The respondent universe for the BPS:04/09 full-scale consists of all students who began their postsecondary education for the first time during the 2003–04 academic year at any eligible postsecondary institution in the United States or Puerto Rico. The sample students were first-time beginners (FTBs) interviewed during NPSAS:04. For the transcript collection, all institutions attended by the BPS:04 full-scale cohort (n=18,644) will be contacted to request undergraduate transcripts.
The success of the BPS:04/09 transcript collection is closely tied to the active participation of selected institutions. The consent and cooperation of an institution’s coordinator is essential and helps to encourage the timely completion of the transcript collection. If the BPS coordinators have been involved with the NPSAS institution collection, they will be familiar with NPSAS and BPS and recognize the study’s importance to postsecondary education. Initial contact between the project team and institutional coordinators provides an opportunity to have a senior staff member emphasize the importance of the study and to address any concerns about participation.
Proven Procedures. BPS:04/09 procedures for working with institutions will be developed from those used successfully in the B&B:08/09 field test transcript collection, as well as NPSAS:08 and NELS:88/2000. BPS will use a transcript control system (TCS) similar to the system like that used for the B&B transcript collection to maintain relevant information about the institutions attended by each BPS cohort member. IPEDS contact information obtained for each institution will be loaded into the TCS and used for all mailings. Contact information obtained for each institution will be confirmed with a verification call to each institution to collect the name of the registrar (or other appropriate contact), address information, telephone and facsimile numbers, and e-mail addresses. This verification call will help to ensure that transcript request materials are properly routed, reviewed, and processed.
The descriptive materials sent to institutions will be clear, concise, and informative about the purpose of the study and the nature of subsequent requests. The package of materials sent to the coordinators, provided in appendix D, will contain:
A letter from RTI providing an introduction to the BPS:04/09 study,
an introductory letter from NCES on U.S. Department of Education letterhead,
a letter of endorsement from the American Association of Collegiate Registrars and Admission Officers (AACRAO),
a list of other endorsing agencies,
information regarding how to log on to the study’s secure website and access the list of students for which transcripts are requested as well as a form in which they can request reimbursement of expenses incurred with the request (e.g., transcript processing fees) and
descriptions of and instructions for the various methods of providing transcripts.
Follow-up calls to ensure receipt of packet and answer any questions about the study will occur 2 days after the initial mailing. We also anticipate that telephone prompting will be required to obtain the desired number of transcripts. Despite the relatively routine nature of the transcript request many institutions give relatively low priority to voluntary research requests. Telephone follow-up is necessary to ensure that the request is handled within the schedule constraints. In addition to telephone prompting, institutions will be contacted by e-mail prompts, letters, and postcard prompts.
Experienced staff from RTI’s Call Center Services (CCS) will carry out these contacts and will be assigned a set of institutions that is their responsibility throughout the process. This allows RTI staff members to build upon relationships developed during the NPSAS study, and to maintain rapport with the institution staff and provides a reliable point of contact at RTI. Project staff members will be thoroughly trained in transcript collection and in the purposes and requirements of the study, which helps them establish credibility with the institution staff.
Endorsements. In NPSAS studies, the specific endorsement of relevant associations was extremely useful in persuading institutions to cooperate. Endorsements from 14 professional associations have already been secured for B&B:08/09. These associations are listed in appendix F. We will contact the same associations to request endorsement for BPS:04/09 as well.
Minimizing burden. Different options for collecting transcripts for sampled students are offered. The coordinator is invited to select the methodology of greatest convenience to the institution. The optional strategies for obtaining the data are discussed later in this section.
Another strategy RTI can investigate to increase the efficiency of the transcript data collection, encourage participation, and minimize burden to individual institutions is to solicit support at a system-wide level and state agencies. A timely contact, together with enhanced verification procedures, is likely to reduce the number of remail requests, and minimize delay caused by misrouted requests.
Institution Coordinator Training. The purpose of an effective plan for training institution coordinators is two-fold: to make certain that procedures are understood and followed, and to motivate the coordinators. The project relies on these procedures to assure transcript collections are accurate and complete. Because institution coordinators are a critical element in this process, communicating instructions about their transcript collection tasks clearly is essential.
Institution coordinators will be trained by call center staff according to the method of data collection they have selected for their institution (refer to section 3.c). All institution coordinators will be provided information on the purposes of BPS and on their tasks for the study, and assured of our commitment to maintaining the confidentiality of institution and student data.
Transcript data will be requested for sampled students from all institutions attended since they first entered the sample in the 2003-04 academic year. Several methods will be used for obtaining the data including: (1) institution staff uploading electronic transcripts for sampled students to the secure study website; (2) institution staff sending electronic transcripts for sampled students by secure File Transfer Protocol; (3) institution staff sending electronic transcripts as encrypted attachments via email; (4) for institutions that already use this method, RTI requesting/collecting electronic transcripts via a dedicated server at the University of Texas at Austin; and as a last resort, (5) institution staff transmitting transcripts via a secure fax that is housed in a locked room at RTI, after sending a confirmed test page. Each method is described below. A complete transcript from the institution will be requested as well as the complete transcripts from transfer schools that the students attended, as applicable.
To track receipt of institution materials and student transcripts, we will add a Transcript Control System (TCS) to the IMS developed for BPS:04/09, similar to the TCS used successfully for B&B:08/09. The TCS will track the status of each catalog and transcript request, from initial mailout of the requests through follow up and final receipt.
Uploading electronic transcripts to the secure study website. Goals for BPS:04/09 include reducing the data collection burden on institutions (thereby reducing project costs), expediting data delivery, improving data quality, and ensuring data security. NPSAS:2000, 2004, and 2008 demonstrated the viability of a web-based approach for receiving student enrollment lists and student record abstraction (CADE) data files. We propose to use the same functionality for uploading data that was used in NPSAS:08.
Because the open internet is not conducive to transmitting confidential data, any internet-based data collection effort necessarily raises the question of security. However, we intend to incorporate the latest technology systems into our web application to ensure strict adherence to NCES confidentiality guidelines. Our web server will include a Secure-Sockets Layer (SSL) Certificate, and will be configured to force encrypted data transmission over the Internet. The SSL technology is most commonly deployed and recognizable in electronic commerce applications that alert users when they are entering a secure server environment, thereby protecting credit card numbers and other private information. Also, all of the data entry modules on this site are password protected, requiring the user to log in to the site before accessing confidential data. The system automatically logs the user out after 20 minutes of inactivity. This safeguard prevents an unauthorized user from browsing through the site. Additionally, we will stay attuned to technological advances to ensure the BPS:04/09 data are completely secure.
Files uploaded to the secure website will be immediately moved to a secure project folder that is only accessible to specific staff members. Access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. It is necessary for the files to be stored on the project share so that they can be backed up by ITS in case any problems occur that cause us to lose data. ITS will use their standard procedures for backing up data, so the backup files will exist for 3 months.
Institution staff sending electronic transcripts by secure File Transfer Protocol. FTPS (also called FTP-SSL) uses the FTP protocol on top of SSL or TSL. When using FTPS, the control session is always encrypted. The data session can optionally be encrypted if the file has not been pre-encrypted.
Files transmitted via FTPS will be copied to a secure project folder that is only accessible to specific staff members. As with uploaded files, access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. After being copied, the files will be immediately deleted from the FTP server. It is necessary for the files to be stored on the project share so that they can be backed up by ITS in case any problems occur that cause us to lose data. ITS will use their standard procedures for backing up data, so the backup files will exist for 3 months.
Institution staff sending electronic transcripts as encrypted attachments via email. RTI will provide guidelines on encryption and creating “strong” passwords. Encrypted electronic files sent via e-mail to a secure e-mail folder will only be accessible to a few staff members on the project team. These files will then be copied to a project folder that is only accessible to these same staff members. Access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. After being copied, the files will be deleted from the e-mail folder. The files will be stored on the network that is backed up regularly to avoid the need to recontact the institution to provide the data again should a loss occur. RTI’s information technology service (ITS) will use standard procedures for backing up data, so the backup files will exist for 3 months.
Institution staff transmitting transcripts via a secure fax. We expect that few institutions will ask to provide hardcopy transcripts. In such cases, we will encourage one of the secure electronic methods of transmission. If that is not possible, we will accept faxed transcripts. Although fax equipment and software does facilitate rapid transmission of information, this same equipment and software opens up the possibility that information could be misdirected or intercepted by individuals to whom access is not intended or authorized. To safeguard against this, as much as is practical, RTI protocol will only allow for transcripts to be faxed to a fax machine housed in a locked room and only if institutions cannot use one of the other options. To ensure the fax transmission is sent to the appropriate destination, we will require a test run with nonsensitive data prior to submitting the transcripts to eliminate errors in transmission from misdialing. RTI will provide schools with a fax cover page that includes a confidentiality statement to use when transmitting individually identifiable information.
Paper transcripts will be kept in a locked file cabinet in RTI’s secure data receipt facility. Only BPS:04/09 transcript staff will have access to the file cabinet.
Collecting electronic transcripts via a dedicated server at the University of Texas at Austin. We will also request and collect transcripts electronically via a dedicated server at the University of Texas at Austin for institutions that currently use this method. Approximately two hundred institutions currently send and receive academic transcripts in standardized electronic formats via a dedicated server at the University of Texas at Austin. The server supports Electronic Data Interchange (EDI) and XML formats. Nine (6 percent) of the field test institutions and approximately 70 (6 percent) of the likely full scale institutions are fully registered with the server. In addition, 14 of the field test institutions and approximately 60 of the likely full scale institutions are in the test phase with the server, which means that they are preparing and testing using the server but not currently using it to send data.
The dedicated server at the University of Texas at Austin supports the following methods of securely transmitting transcripts:
email as MIME attachment using PGP encryption
regular FTP using PGP encryption
Secure FTP (SFTP over ssh) and straight SFTP
FTPS (FTP over SSL/TLS)
Files collected via the dedicated server at the University of Texas at Austin will be copied to a secure project folder that is only accessible to specific staff members. The same access restrictions and storage protocol will be followed for these files as described above for files uploaded to the study website.
We do not anticipate that active student consent for the release of transcripts will be required for BPS:04/09. For certain agency purposes, the Family Educational Rights and Privacy Act of 1974 (FERPA) permits institutions to release student data to the U.S. Department of Education and its authorized agents without consent. In compliance with FERPA, a notation will be made in the student record that the transcript has been collected for use in the BPS:04/09 longitudinal study.
Despite the relatively routine nature of the transcript request, it is anticipated that telephone prompting will be required to obtain the desired number of transcripts. We will also use e-mail prompts, letters, and postcard prompts, which have proven to be effective tools in gaining cooperation. E-mail, in particular, has proven to be a low-cost and effective means of reaching institution officials who cannot be reached by phone. Because each transcript request will be accompanied by a voucher for any expenses incurred in handling the request, it is unlikely that refusals will become a significant problem. However, in the event that an institution expresses resistance to the transcript request, seasoned institutional contactors and other project staff are trained to sensitively listen to institutional concerns, address any roadblocks to participation, and negotiate with institution staff to resolve them.
Another strategy RTI can investigate to increase the efficiency of the transcript data collection, encourage participation, and minimize burden to individual institutions is to solicit support at a system-wide level. If needed, state and system-wide contacts made as part of NPSAS:04 will be asked to encourage the participation of institutions under its administration. While we are planning for transcript collection from each individual institution, we will also explore the possibility of collecting transcripts from state agencies or at a system-wide level whenever such an approach is practical, as well as the National Student Clearinghouse database which the B&B:08/09 field test transcript effort has shown is being used by some institutions. Based on our NELS:88/2000 experience, we expect that less than 5 percent of institutions will have closed, but it is often the case that transcripts can be collected for institutions that are technically closed. RTI will confirm the status of any closed institution with state departments of education, offices of higher education, or other appropriate state licensing agencies.
Transcript Collection Quality Control. As part of our quality control procedures, we will emphasize to registrars the importance of collecting complete transcript information for all sampled students. Transcripts will be reviewed for completeness. Institutional Contactors will contact the institutions to prompt for missing data and to resolve any problems or inconsistencies.
Transcripts received in hardcopy form via secure fax will be subject to a quick review prior to recording their receipt. Receipt control clerks will check transcripts for completeness and review transmittal documents to ensure that transcripts have been returned for each of the specified sample members. The disposition code for transcripts received will be entered into the TCS. Course catalogs will also be reviewed and their disposition status updated in the system in cases where this information is necessary and not available through CollegeSource Online. Hardcopy transcripts and course catalogs will be sorted and stored in a secure facility at RTI, organized by institution.
The procedures for electronic transcripts will be similar to those for hardcopy documents—receipt control personnel, assisted by programming staff, will verify that the transcript was received for the given requested sample member, record the information in the receipt control system, and check to make sure that a readable, complete electronic transcript has been received.
The initial transcript check-in procedure is designed to efficiently receipt returned materials into the TCS as they are received each day. The presence of an electronic catalog (obtained from CollegeSource Online) will be confirmed during the verification process for each institution and noted in the TCS. The remaining catalogs will be requested from the institutions directly and will be receipted in the TCS as they are received. Transcripts and supplementary materials received from institutions (including course catalogs) will be inventoried, assigned unique identifiers based on the IPEDS ID, reviewed for problems, and receipted into the TCS.
Data processing staff will be responsible for (1) sorting transcripts into alphabetical order to facilitate accurate review and receipt; (2) assigning the correct ID number to each document returned and affixing a transcript ID label to each; (3) reviewing the materials to identify missing, incomplete, or indecipherable transcripts; and (4) assigning appropriate TCS problem codes to each of the missing and problem transcripts plus providing detailed notes about each problem to facilitate follow-up by Institutional Contactors and project staff. Project staff will use daily monitoring reports to review the transcript problems and to identify approaches to solving the problems.
Web-based collection will allow timely quality control, as RTI central staff will be able to monitor data quality for participating schools closely and on a regular basis. When institutions call for technical or substantive support, we will be able to query the institution’s data and communicate much more effectively regarding any problems.
Transcript data, including paper transcripts, will be destroyed or shredded after the transcripts are keyed, coded, and quality checked at a time to be negotiated with NCES.
Transcript Keying and Coding. Once student transcripts and course catalogs are received and missing information is collected, keying and coding of transcripts and courses taken will take place. The taxonomy of coding will be modeled on those used in B&B:08/09, which itself is based on the taxonomy developed for NELS:88/2000 and B&B:93/94. A careful review of these taxonomies (based on Adelman’s College Course Map) will be carried out with refinements. Guidance will be provided by NCES, technical review panel members, and other key personnel on refining and reviewing the taxonomy for transcript coding and new courses and fields of study.
Keyer-Coders will have full access to all transcript-related documents including course catalogs or other course listings provided. All transcript-related documents will be thoroughly reviewed before data is abstracted from them.
Transcript Keying and Coding Quality Control. As part of our quality control procedures, we will ensure that all coders have earned a bachelor’s degree to ensure that they have firsthand knowledge of college courses, credits, and grade point averages. Emphasis is placed on recruiting professions with knowledge of transcripts and teaching curriculum as our expert keyer-coders.
A supervision and quality control plan will also be implemented. At least one supervisor will be onsite at all times to manage the effort and simultaneously perform QC checks and problem resolution. Verifications of transcript data keying and coding at the student level will be performed. Any errors will be recorded and corrected as needed.
Once the transcripts for each institution are keyed and coded, transcript course coding at the institution level will be reviewed by expert coders to ensure that (1) coding taxonomies have been applied consistently and data elements of interest have been coded properly within schools (2) program information has been coded consistently according to the program area and sequence level indicators in course titles (3) records of sample members who attended multiple institutions do not have duplicate entries for credits that transferred from one institution to another and (4) additional information has been noted and coded properly
No testing of procedures or methods is planned for the full-scale transcript collection. Given the timing of the request, a field test collection will not be made. The field test transcript collection for the 2008/09 Baccalaureate and Beyond Longitudinal Study (B&B:08/09), a sister study to BPS, being conducted currently by RTI will serve as an adequate pretest of collection methods and procedures.
Names of individuals consulted on statistical aspects of study design along with their affiliation and telephone numbers are provided below.
Name Affiliation Telephone
Dr. Sara Wheeless RTI 919/541-5891
Dr. Jennifer Wine RTI 919/541-6870
Ms. Melissa Cominole RTI 919/990-8456
Dr. Karol Krotki RTI 202/728-2485
Mr. Peter Siegel RTI 919/541-6348
Dr. Lutz Berkner MPR 510/849-4942
In addition to these statisticians and survey design experts, the following statisticians at NCES have also reviewed and approved the statistical aspects of the study: Dr. James Griffith, Dr. Tom Weko, and Dr. Tracy Hunt-White.
The study is being conducted by the Postsecondary Longitudinal and Sample Survey Studies unit of NCES, U.S. Department of Education. NCES’s prime contractor is RTI. RTI is being assisted through subcontracted activities by MPR Associates. Principal professional staff of the contractors not listed above, who are assigned to the study, are provided below:
Name Affiliation Telephone
Mr. John Doherty RTI 919/541-7120
Ms. Vicky Dingler MPR 510/849-4942
Ms. Kristin Dudley RTI 919/541-6855
Ms. Emily Forrest-Cataldi MPR 515/270-8457
Mr. Jeff Franklin RTI 919/485-2614
Mr. Joe Simpson RTI 919/541-5941
| File Type | application/msword |
| Author | Edith.McArthur |
| Last Modified By | doritha.ross |
| File Modified | 2008-08-28 |
| File Created | 2008-06-24 |