Supporting Statement B for
NLM Distance Learning Mentoring Program Study (NLM /LHNCBC)
Collection of Information Employing Statistical Methods
November 5, 2014
Craig Locatis
National Library of Medicine
Lister Hill National Center for Biomedical Communications
Office of High Performance computing and Communication
Building 38A, Room B1n30
(301) 435-32554 – phone
(301) 402-4080 – fax
Data are being collected from all students participating in an internship and follow-on distance learning program. Half the 50 students who will participate are from middle schools and half are from high schools in the San Juan, Puerto Rico metropolitan area. We expect that all of the students will respond since completion of the essays, inventories, and ratings are indicated as a requirement for participating in the summer internship and follow on distance learning program. Sampling is not being used because the number of subjects is low and we are only interested in how the participating students are responding to the internship and distance learning experience. Data will be analyzed for the group as a whole and for the middle school and high school sub-groups. There will no presentation of data by race or ethnicity other than an indication that that the students are form Puerto Rican schools. There will be no exclusion of participation because of race, gender, or ethnicity.
Data will be analyzed for all 50 students as a whole and the same analyses will be done again for the 25 participating middle and 25 participating high school students. Students will complete a general career interest inventory online, health science career knowledge and interest inventory, and will write an essay of 500 or fewer words about health careers. Each will be completed three times: 1) at the start of a summer internship to establish a baseline; 2) at the end of the week long summer internship to determine its impacts; and 3) at the end of the follow on distance learning and mentoring program. All three essays will be scored at the end of the program by two independent reviewers who will be blinded to the time during the study that the essay was written. The essay scores, general inventory and health science inventory results for the three measurement intervals will be used to determine the relative impacts of the internship and the follow on distance learning program respectively. In addition, the students will rate the quality of the instruction they received at the end of the internship and after each distant learning session.
There will be one hour distance learning mentoring sessions offered once a month for eight months by different health professionals, each describing a different health career, why they chose it, and other information about it, such as its educational requirements. These sessions will be delivered by videoconference to the participating students’ schools where students will view the presentations, ask questions, and otherwise participate using computers equipped with videoconferencing technology.
The essays and inventory results before and after the internship and the rating of internship instruction will be collected by staff while students attend the internship program on the University of Puerto Rico Medical Campus. The distance learning session ratings and the essays and health science career knowledge and interest inventory done at the end of the follow on program will be collected by a teacher at each student’s school and forwarded to the university. Since the general career inventory is online, the results of this inventory taken at the end of the follow on program will be accessed directly by researchers at the university.
The evaluation plan will be a pretest-posttest with follow-up design. The respondents will be all those students that chose to enroll in a special health sciences program. The four evaluation instruments will include a general career interest inventory (GCII), the Health Science Career Knowledge and Interest Inventory (HSCKII) which is a 20 item, 7-point Likert type agreement scale, the Student Essay Ratings scored 0 to 5 by two judges on each of 4 criteria with a maximum score of 20, and the 10-item Distance Learning Mentor Presenter Evaluation form. Approximately 50 students are expected to participate. Students will be categorized by educational level, middle school or high School. Means and standard deviations will be calculated for the HSCKII, and the Student Essay Rating instruments and presented in tables stratified by pre-internship, post internship, and final follow-up stages and educational level. Interest in a health science career will be categorized in binary form as “yes” or “no” as indicated by the results of the GCII. Tables of proportion of student interested in those careers will be present stratified by the three stages and by educational level. Mean Distance Learning Mentor Presenter Evaluation scores will be presented in a separate table to show the perceived quality of assistance during the Mentoring phase of the program. The primary statistical analysis will be a repeated measures t-test on the mean HSCKII and mean Student Essay scores comparing post workshop or follow-up scores with pre-program scores. To test for changes in student career interest based in the IPI a McNemar test for paired nominal data will be used. Exact tests may be used if data is sparse. A two-sided alpha significance level of .05 will be chosen for the criteria indicating significant change or differences in all analyses. The primary software used will be the Statistical Package for the Social Sciences (SPSS) software.
The students will be on campus for the internship and under teacher supervision for the follow on distance learning mentoring sessions and will be given time to complete the essays, inventories, and ratings. The response rate is expected to be very high as a result and there is no need for methods to maximize response rate that would normally apply to surveys involving random sampling of a given population. The response rate will be determined by dividing the number students who completed each instrument by the number who participated in the internship. Based on past experience in the educational setting, a response rate of 85% or better is expected with each questionnaire at each stage.
The health science career knowledge and interest inventory will be tried out with 3 to 6 summer students at NLM who will read the questions aloud and attempt to answer them. Students will be queried if the hesitate or exhibit behavior that they might not understand each question. Students will have an opportunity to ask questions for clarification. If more than one student indicates a problem understanding a question, it will be examined for possible wording changes eliminating any ambiguity.
Statistical consultant:
Richard Maisiak, PhD, (602) 923-7726; (602) 741-2434 (mobile)
Data collection and analysis:
Emma Fernandez-Repollet, Ph.D.
Department of Pharmacology, Lab A-331
University of Puerto Rico Medical Sciences Campus
PO Box 365067
San Juan, PR 00936-5067
Tel: (787) 763-9401
Fax: (787) 758-5206
E-mail: e.fernandez@upr.edu
Additional data analysis:
Richard Maisiak, PhD
Maisiak Associates
University of Alabama at Birmingham, Professor, retired
5444 E. Grovers Avenue
Scottsdale, AZ 85254
(602) 741-2434
Craig Locatis, PhD
Office of High Performance Computing and Communication
National Library of Medicine
8600 Rockville Pike
Bethesda, MD 20894
(301) 435-3254
clocatis@mail.nih.gov
| File Type | application/msword |
| File Title | Supporting Statement 'B' Preparation - 03/21/2011 |
| Subject | Supporting Statement 'B' Preparation - 03/21/2011 |
| Author | OD/USER |
| Last Modified By | Sharlip, David (NIH/NLM) [E] |
| File Modified | 2014-11-05 |
| File Created | 2014-11-05 |