Justification Statement A_032916

SUPPORTING STATEMENT FOR LEARNERS’ PERCEPTIONS SURVEY, Cont’d

This submission is for extension of a currently approved information collection.

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection of information.

Under the authority of Federal Law 38 U.S.C. Part V, Chapter 73, Section 7302, the Department of Veterans Affairs (VA) provides education and training to over 120,000 national cohort of health care trainees per year to assist in providing an adequate supply of health personnel for VA and the Nation. VA is further required to evaluate this program on a continuing basis and determine its effectiveness in achieving its goals (Federal Law, 38 U.S.C. Part I, Chapter 5, Section 527). In addition, the Government Performance and Results Act (GPRA) of 1993, requires Federal agencies to set goals, measure performance, and report on the accomplishments.

More recently, the Veterans Access, Choice, and Accountability Act of 2014 (PL 113-146), as signed into law on August 7, 2014 (128 Stat. 1754; 38 USC 101; HR3230), was enacted to “… improve the access of veterans to medical services from the Department of Veterans Affairs…” The Act requires the Secretary to: “… ensure that already established medical residency programs have a sufficient number of residency positions…” (at §301(b)(1) amending 38 U.S.C. 7308(e)(1)); to provide annual reports (directly to both House and Senate Veterans’ Affairs Committees) detailing its progress towards meeting the 1,500 new GME positions goal (at §301(b)(3)) and include the number of “positions filled,” “not filled,” “anticipating filling,” (at §301(b)(3)(B)(i)) provide the resident’s geographic location, academic affiliation (at §301(b)(3)(B)(ii)), “the policy at each medical facility … with respect to the ratio of medical residents to staff supervising medical residents” (at §301(b)(3)(B)(iii)), “… the number of individuals who declined an offer from the Department [of Veterans Affairs] to serve as a medical resident at a medical facility of the Department and the reason why each such individual declined such offer” (at §301(b)(3)(B)(iv)), descriptions of “… challenges … faced by the Department in filling graduate medical education residency positions…” (at §301(b)(3)(B)(v)(I)), the “actions … taken by the Department to address such challenges…” (at §301(b)(3)(B)(v)(II)), and finally “… efforts … to recruit and retain medical residents to work for the Veterans Health Administration as full-time employees.” (at §301(b)(3)(B)(vi)).

In addition, health professional training is one of the Veterans’ Health Administration’s (VHA) core missions for almost 70 years, since Policy Memorandum No. 2, January 20, 1946, established educational affiliations between Veterans Administration medical centers (VAMC) and schools of medicine. VA medical centers throughout the nation have served this mission by affiliating with accredited training programs in undergraduate and graduate medical education, nursing and associated health training programs such as pharmacy, psychology, social work and dentistry. Over this time period, the number of VA facilities with medical school affiliations has grown. In academic year 2014, 41,223 medical residents; 22,931 medical students; 1,398 dental residents and dental students received some or all of their clinical training at VA. Nearly two-thirds of all U.S. medical students train in VA facilities prior to their graduation. Nearly 80% of VA physicians have faculty appointments at an educational institution. VA’s affiliations and the educational environment contribute to VA's mission by enabling the recruitment of high quality faculty members to VA staff. About 99 percent of VA's over 2,300 graduate medical education programs are sponsored in the name of an affiliate - generally a medical school or teaching hospital. About one-third of VA's dental residency training programs are sponsored by VA, with 80 percent of these in general practice dentistry. The remaining two-thirds of the dental residency programs are sponsored by dental or medical schools, universities, or teaching hospitals.

In addition, VA is a leader in the training of associated health professionals. Through affiliation agreements with more than 1,800 unique accredited college and university programs, training stipends are provided annually to over 4,600 trainees in fifteen disciplines. Funding is available for clinical pastoral education, dietetics, optometry, physician assistants, podiatry, as well as a range of mental health and rehabilitation professions. More than 27,200 Associated Health trainees received training at a VA facility in FY14. Nearly 70 percent of current VA optometrists and psychologists participated in VA training programs prior to their employment at VA.

In addition, VA is a leader in nursing education. More than 27,300 Nursing trainees received training at a VA facility. OAA maintains academic partnerships with nursing schools around the country to provide highly educated nurses to the VA and the Nation. Through these partnerships, VA provides opportunities for educational and practice innovations, expands faculty and professional development, implements faculty practice which increases Veteran access to care, and increases recruitment and retention of VA nurses. OAA Nursing is currently overseeing three major Nursing education initiatives: the VA Nursing Academic Partnerships (VANAP) program with both an Undergraduate and a Graduate focus; the Post-Baccalaureate Nurse Residency (PBNR) program; and the Psychiatric/Mental Health Nurse Practitioner Residency (PMHNPR) program. VANAP has expanded to include 19 VA Facility/School of Nursing partnerships across the country. This program was designed to leverage academic and clinical resources to increase student enrollment, enhance VA Nursing faculty competencies and professional scholarship, revise academic curriculum to include Veteran specific health problems, develop inter-professional education, develop Veteran-centric practice and education initiatives, and increase the recruitment and retention of VA nurses. The PBNR program is a pilot hosted by 14 VA facilities and is designed to operate in line with VHA’s statutory health professions education mission enabling VA to enhance the clinical learning and practice environment for Nurses, recruit and retain high quality professional staff, and provide excellent care to Veterans. As of this past year, the PBNR program will be fully integrated into the VANAP Undergraduate program. OAA has committed to an expansion of clinical training in Mental Health. This initiative is consistent with the Institute of Medicine recommendation of providing clinical residency programs for nurse practitioners. Studies have shown that over half of new Nurse Practitioners do not perceive themselves as well prepared for practice following their graduate education. As such, OAA Nursing oversees three VA facilities operating a PMHNPR with plans to expand into Acute Care and Adult-Gerontology in FY2016.

In addition to the growth of programs within the VA, external forces have created a changing healthcare environment. Accrediting bodies, such as the Accreditation Council for Graduate Medical Education (ACGME) have imposed sweeping changes, including resident duty hour limits in the training environment that affect the VA training environments as well as the clinical care environments where veteran patients health care needs are served. Further changes in regulations governing clinical training programs in nursing, associated health, and physicians are expected to impact how health professionals are trained in VA. This changing landscape in the clinical education environment comes at a time when Veterans Health Administration (VHA) must rise to the challenge of treating a new generation of veterans. The interface with the clinical and the educational arenas necessitates a system for assessment of the education environment. Consistent with VHA leadership in other areas of measurement, such as performance measures to track and improve quality of patient care in the clinical realm, VHA has served to lead in measurement of learner satisfaction through the creation and implementation of the Learners’ Perceptions Survey. These include measuring how VA training programs influence acceptance of VA models of care including patient-centered care, interprofessional team care, known as Patient Aligned Care Teams and formerly known as Patient Centered Medical Homes, and health professions training helps with the recruitment and retention of professional staff, and how health professions training cares an academic environment necessary to support research on how to diagnosis, assess, and treat veterans with military service related clinical conditions.

The Learners’ Perceptions Survey (LPS) is a tool that was created by the VA to allow ongoing assessment of clinical training environments in order to help accomplish VA’s teaching mission. The survey identifies both strengths and opportunities for improvement for VHA in the clinical education environment and measures the satisfaction of VA clinical trainees who come in direct contact with our veteran patients and who contribute to their care on a daily basis. Clinical trainees who are satisfied with their clinical training can impact how patients view their care and patient satisfaction. Implicit in the survey is the identification of key drivers of clinical trainees’ satisfaction so as to develop and implement targeted improvements that will benefit both learners and patients in VHA. The survey also measures other aspects important to patient care services, including patient-centered care, interprofessional team care, psychological safety, quality of care, willingness of trainees to consider VA for future employment, and program requirements and accreditation standards for health professions education. The LPS survey has also been instrumental in meeting accreditation requirements for our academic affiliates, including medicine, nursing, and associated health.

This survey is consistent with VA’s oversight responsibilities and Government Performance and Results Act (GPRA), and VA has identified trainee satisfaction with clinical training experience as one of the VA national performance measures. The survey results are reported internally in VA and in the Annual Performance and Accountability Report submitted to Office of Management and Budget (OMB).

2. Indicate how, by whom, and for what purposes the information is to be used; indicate actual use the agency has made of the information received from current collection.

The results of the survey are used at every level in VHA from the national level to local in understanding elements and domains that impact the clinical training environment. At the VA medical center level, the survey data is available in the form of a data cube containing both adjusted and unadjusted scores that permit comparison across facilities, across VISNs, and over time. Such information informs local VA education leaders and program officials regarding areas of strength and opportunities for improvement in specific domains (quality of the faculty, learning environment, working environment, physical environment and personal experience) and allows facilities to trend data so that changes over time can be monitored. In order to maintain VA as a preferred training site for future health care professionals, it is important to know how trainees view VA training versus training in non-VA settings. The survey results provide data on the perceptions of trainees comparing their clinical training experiences at both VA and non-VA facilities.

At the VISN level, the results of the survey can focus VISN-leadership on the critical needs pertaining to the teaching mission of VA and provide information as to how the teaching environment can be improved. Survey results also indicate that the likelihood that trainees will consider employment with VA improves (almost doubles) if they had training at VA. Careful evaluation of areas of satisfaction and relative dissatisfaction may provide useful information to aid in recruitment and retention of quality health care professionals.

At the national level, the survey results call attention to the rigor with which the VA addresses the education mission as the results are reported widely in VA and in the Annual Performance and Accountability Report submitted to OMB. Over the past several years, data has been collected regarding the degree to which individual facilities and disciplines have used the survey results to make changes in their programs. Since drivers of satisfaction are different for different disciplines, special studies are completed for certain disciplines (i.e., pharmacy, nursing and physician residents) to determine system-wide trends and issues.

In addition, the LPS survey has been used as a tool to determine the impact training programs have had on recruitment of trainees to work for VA, impact of psychological safety on staff productivity and retention, impact of PACT and patient-centered and interprofessional team models of care, and impact of trainees on contribution to clinical workload net of supervision.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

Electronic, online completion of the survey reduces respondent and government burden, by providing a convenient, faster and less labor-intensive survey collection and data analysis than paper submission. The respondents do not have to mail paper questionnaires, and they can complete the survey from any Internet accessible computer, either at home or at a VA medical center. Data can be rolled up by facility, VISN, and nationally for analysis using a number of trainee categories (e.g., types of training programs and level of training).

To reduce burden, survey implementation will be limited to trainees having training rotations at the VA during the year, to be completed at the end of the VA rotations. For physician trainees, it is known that satisfaction varies over the course of the year, with biases both at the start (e.g. July/August) and end (e.g. May/June) of the academic year.

In addition, the psychometric properties of responses are constantly monitored, leading recently to a reduction of the survey by 20%, without loss of significant information. Results of psychometric analyses by domain is posted on the OAA Intranet website as the LPS Instructions and User Manual.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

Information at the facility or system level concerning trainees’ perceptions of their VA clinical training experience is not available elsewhere.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

The continuous improvement cycle requires local facilities to measure, intervene and re-measure. If the survey were to be conducted less frequently, VA could not institute timely changes and program oversight would be impaired. For example, Choice Act requires VA provide annual evaluations.

There are no technical or legal obstacles to reducing burden.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

Not applicable. There are no such special circumstances or requirements.

8. a. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

The notice of Proposed Information Collection Activity was published in the Federal Register on December 11, 2015 (Volume 80, Number 77083, Page 77083). We received no comments in response to this notice.

b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances which preclude consultation every three years with representatives of those from whom information is to be obtained.

LPS survey is guided by a National Evaluation Workgroup that meets weekly. The group is comprised of VA employees, including Associate Chiefs of Staff for Education, Informatics, and Designated Learning Officers from selected VA medical centers across a wide geographic area. The workgroup advised LPS in terms of survey design, format, and administration, data management, scoring, coding, and analyses, presentation of results, and interpretation of findings for VA leadership, including the health professions education community, research, and administrators and clinical practitioners in VA teaching settings.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift is provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The first line in the survey introduction is the following statement: “This is a confidential survey.”

Descriptive statistics (means, frequencies, standard deviation, modes, medians) are reported to the field using aggregated data only with no fewer than 8 respondents to a reporting unit. The likelihood that aggregate statistics can be used to re-identify respondents depends on the total number of trainees in the reportable unit, but is in the most extreme case (8 respondents and 8 total trainees) considered negligible.

The LPS is also used for evaluation purposes that are often required for a program to obtain, and sustain, accreditation from a national accrediting agency. Trainees who receive academic credit in unaccredited programs cannot use such credit for purposes of satisfying state licensing requirements. We plan to use the LPS for such purposes. However, since it is possible that as few as 2 respondents may be responding to the survey at the local level, respondents who are asked to take the survey for accreditation evaluation purposes are directed to a different website where they will take the LPS, but face a different instructions and cover page. Here, the opening paragraph to the introduction reads: “This is a confidential survey. Your responses will not be made available to your program directors, faculty, or clinical staff. However, your responses may be combined with those from one or more other respondents to compute aggregate information (means, frequencies). Aggregate information may be released to program directors, faculty, and clinical staff for program evaluation and administrative purposes. By taking the survey you have agreed to these terms.”

The completed surveys can be withheld if requested under the Freedom of Information Act (FOIA) under Exemption 5 of the FOIA according to VA General Counsel. The clinical trainees’ opinions in the survey are considered individual opinions gathered in a “pre-decisional” deliberation.

11. Provide additional justification for any questions of a sensitive nature (Information that, with a reasonable degree of medical certainty, is likely to have a serious adverse effect on an individual's mental or physical health if revealed to him or her), such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Not applicable. There are no questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information:

a. The annual burden is estimated to be

Respondents	Minutes	Equals	/ by 60	Annual Burden Hours
15,000	15	225,000	60	3,750

b. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB 83-I.

This request is for a web-based survey questionnaire for VHA healthcare trainees.

c. Provide estimates of annual cost to respondents for the hour burdens for collections of information. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14 of the OMB 83-I.

The respondent population for VA Form 10-0439 is a tool that was created by the VA to allow ongoing assessment of clinical training environments in order to help accomplish VA’s teaching mission. VHA cannot make further assumptions about the population of respondents because of the variability of factors such as the educational background and wage potential of respondents. Therefore, VHA used general wage data to estimate the respondents’ costs associated with completing the information collection.

The Bureau of Labor Statistics (BLS) gathers information on full-time wage and salary workers. According to the latest available BLS data, the median weekly earnings of full-time wage and salary workers are $800.00. Assuming a forty (40) hour work week, the median hourly wage is $20.00.

Legally, respondents may not pay a person or business for assistance in completing the information collection and a person or business may not accept payment for assisting a respondent in completing the information collection. Therefore, there are no expected overhead costs for completing the information collection. VHA estimates the total cost to all respondents to be $75,000 (3,750 burden hours x $20.00 per hour).

The total cost to the respondents annually is based on time spent on completing the survey and is estimated to be $75,000 based on a $20/hr student stipend cost.

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

The only cost to the respondent is the time spent in completing the survey. There is no capital, start-up, operational or maintenance costs to the respondent for completing the survey.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total estimated cost to the Government is estimated at $61,258.

.

Staff Salary	Function	Salary	Hours
GS 13/5	On-Line Survey Development	$40.95	80	$3,276
Overhead at 100% of salary = $3,276
GS 13/5	Report Preparation	$40.95	200	$8,190
Overhead at 100% of salary = $8,190
GS 13/5	SPSS Programming	$40.95	160	$6,552
Overhead at 100% of salary = $6,552
GS 13/5	Web Design	$40.95	40	$1,638
Overhead at 100% of salary = $1,638
GS 14/5	Statistical Analysis	$48.39	120	$5,807
Overhead at 100% of salary = $5,807
GS 12/5	Help Desk Assistance	$34.44	150	$5,166
Overhead at 100% of salary = $5,166
TOTAL				$61,258

Overhead costs are 100% of salary and are same as the wage listed above and the amounts are included in the total.

15. Explain the reason for any burden hour changes since the last submission.

The public burden hours in item 12 on a per person basis remained unchanged at 15 minutes.

The 2 surveys have been combined into 1 document to reflect the accurate number of burden hours and number of responses.

An expiration date placeholder has been added to the form.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The primary mission of conducting the LP survey is to identify strengths and opportunities for improvement in the VA clinical training program. Information collected is disseminated to VA facilities and VISNs in the form of a data cube with instructions and user manuals, current reports made available to VA medical centers, and special reports focusing on specific problems as requested by the field. The purpose of this information is to inform local facilities on problems, potential problems, actions they can take to correct such problems, and an assessment of whether such actions have been effective to correct the problem.

The aggregated information is disseminated nationally in VA reports and may be published. OAA has published results based upon the LPS and may do so in the future using appropriate statistical methodology. Such publications give credibility to LPS findings, while providing valuable information on ‘lessons learned’ to the broader health professions education community.

List of articles published and presentations provided regarding surveys:

Keitz, S., Holland, G.J., Melander, E.H., Bosworth, H., and Pincus, S.H. for the Learners’ Perceptions Working Group (Gilman, S.C., Mickey, D.D., Singh, D., et al). The Veterans Affairs Learners’ Perceptions Survey: The Foundation for Educational Quality Improvement. Academic Medicine 78(9):910-917, 2003.

Singh, D.K., Holland, G.J., Melander, E.H., Mickey, D.D., Pincus, S.H.: VA’s Role in U.S. Health Professions Workforce Planning. Proceedings of the 13th Federal Forecasters Conference of 2003:127-133, 2004.

Singh, D. K., Golterman, L., Holland, G. J., Johnson, L.D., and Melander, E. H., Proposed Forecasting Methodology for Pharmacy Residency Training, Proceedings of the 15^th Federal Forecasters Conference of 2005: 39-42, 2005.

Chang, Barbara K.; Kashner, T. Michael; and Holland, Gloria J. “Evidence-based Expansion and Realignment of Physician Resident Positions.” Presented at the 3^rd Annual Association of American Medical Colleges Physician Workforce Research Conference. Bethesda MD, May 2-4, 2007.

Chang, B.K.; Holland, G.J.; Kashner, T.M.; Flynn, T.C.; Gilman, S.C.; Sanders, K.M.; and Cox, M.   “Graduate Medical Education Enhancement in the VA.” Presented at the Association of American Medical Colleges Group on Resident Affairs Professional Development Meeting, Small Group Facilitated Discussion. Memphis TN, April 22-25, 2007.

Chang, B. K.; Kashner, T.M.; Holland, G.J.  “Allocation Methods to Enhance Graduate Medical Education.” Presented at the International Medical Workforce Collaborative. Vancouver B.C., Canada, March 21-24 2007.

Cannon, Grant W.; Keitz, Sheri A.; Holland, Gloria J.; Chang, Barbara K.; Byrne, John M.; Tomolo, Anne; Aron, David C.; Wicker, Annie B.; and Kashner, T. Michael. “Factors Determining Medical Students’ and Residents’ Satisfaction during VA-Based Training: Findings from the VA Learners’ Perceptions Survey.” Academic Medicine. vol. 83, no. 6 (June 2008), pp. 611-620.

Chang, Barbara K.; Cox, Malcolm; Sanders, Karen M.; Kashner, T. Michael; and Holland, Gloria J. Expanding and Redirecting Physician Resident Position by the US Department of Veterans Affairs.” Presented at the 11^th International Medical Workforce Collaborative, Royal College of Surgeons of Edinburgh, Edinburgh UK, September 17, 2008.

Golden, Richard M.; Henley, Steven S.; White Jr., Halbert L.; and Kashner, T. Michael. "Correct Statistical Inferences using Misspecified Models with Missing Data with Application to the Learners’ Perceptions Survey." Presented at the Joint Annual Convention of the 42^nd Annual Meeting of the Society for Mathematical Psychology and the 40^th Annual Conference of the European Mathematical Psychology Group, Amsterdam, Netherlands, August 1-4, 2009.

Kashner, T. Michael; Henley, Steven S.; Golden, Richard M.; Byrne, John M.; Keitz, Sheri A.; Cannon, Grant W.; Chang, Barbara K.; Holland, Gloria J.; Aron, David C.; Muchmore, Elaine A.; Wicker, Annie; and White Jr., Halbert L. “Studying the Effects of ACGME Duty Hours Limits on Resident Satisfaction: Results from VA Learners’ Perceptions Survey.” Academic Medicine. vol. 85, no. 7 (July, 2010), pp. 1130-1139.

Golden, Richard M.; Henley, Steven S.; White Jr., Halbert L.; and Kashner, T. Michael. “Application of a Robust Differencing Variable (RDV) Technique to the Department of Veterans Affairs Learners’ Perceptions Survey.” Presented at the 43^rd Annual Meeting of the Society for Mathematical Psychology, Portland, OR, August 7-10, 2010.

Kaminetzky, Catherine P.; Keitz, Sheri A.; Kashner, Michael; Aron, David C.; Byrne, John M.; Chang, Barbara K. ; Clarke, Christopher; Gilman, Stuart C.; Holland, Gloria J.; Wicker, Annie; and Cannon, Grant W. “Training Satisfaction for Subspecialty Fellows in Internal Medicine: Findings from the Veterans Affairs (VA) Learners’ Perceptions Survey.” BMC Medical Education. vol. 11, no. 21 (2011), pp. 1-9 (http://www.biomedcentral.com/1472-6920/11/21).

Kashner, T. Michael; and Chang, Barbara K. “VA Residents Improve Access and Financial Value.” Presented at the Annual Meeting of the Association of American Medical Colleges, Denver, CO, November 4-9, 2011.

Lam, Hwai-Tai C.; O’Toole, Terry G.; Arola, Patricia E.; Kashner, T. Michael; and Chang, Barbara K. “Factors Associated with the Satisfaction of Millennial Generation Dental Residents.” Journal of Dental Education, vol. 76, no. 11 (November, 2012), pp. 1416-1426.

Byrne, John M.; Chang, Barbara K.; Gilman, Stuart; Keitz, Sheri A.; Kaminetzky, Cathy; Aron, David; Baz, Sam; Cannon, Grant; Zeiss, Robert A.; and Kashner, T. Michael. “The Primary Care-Learners’ Perceptions Survey: Assessing Resident Perceptions of Internal Medicine Continuity Clinics and Patient-Centered Care.” Journal of Graduate Medical Education, vol. 5, no. 4 (December, 2013), pp. 587-593.

Chang, Barbara; Muchmore, Elaine; and Kashner, T. Michael. “Taking the Pulse of Your GME Training Programs.” Presented at the 2014 (AAMC) Group on Resident Affairs Spring Meeting, Phoenix, AZ, May 4-7, 2014.

Byrne, John M.; Kashner, T. Michael; Gilman, Stuart C.; Wicker, Annie B.; Bernett, David S.; Aron, David C.; Brannen, Judy L.; Cannon, Grant W.; Chang, Barbara K.; Hettler, Debbie L.; Kaminetzky, Catherine P.; Keitz, Sheri A.; Zeiss, Robert A.; Golden, Richard M.; Paik, Dae-Hyun; and Henley, Steven S. “Do Patient Aligned Medical Team Models of Care Impact VA’s Clinical Learning Environments.” Presented at the 2015 Health Services Research and Development / Quality Enhancement Research Initiative (HSR&D/QUERI) National Conference, Philadelphia, PA, July 8-10, 2015.

Perez, Elena V.; Byrne, John M.; Durkin, Rob; Wicker, Annie B.; Henley, Steven S.; Golden, Richard M.; Hoffman, Keith A.; Hinson, Robert S.; Aron, David C.; Baz, Samuel; Loo, Lawrence K.; Velasco, Erwin D.; McKay, Tracy; and Kashner, T. Michael. “Clinical Supervision Index: Measuring Supervision of Physician Residents in VA Medical Centers.” Presented at the 2015 Health Services Research and Development / Quality Enhancement Research Initiative (HSR&D/QUERI) National Conference, Philadelphia, PA, July 8-10, 2015.

Kashner, T. Michael; Hettler, Debbie L.; Zeiss, Robert A.; with Aron, David C.; Brannen, Judy L.; Byrne, John M.; Cannon, Grant W.; Chang, Barbara K.; Dougherty, Mary B.; Gilman, Stuart C.; Holland, Gloria J.; Kaminetzky, Catherine P.; Wicker, Annie B.; Bernett, David S.; and Keitz, Sheri A. “Has Interprofessional Education Changed Learning Preferences? A National Perspective,” invited resubmission to Health Services Research.

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The expiration date placeholder will be included on the form.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

Page 13

Submitted 09/2015