Supporting Statement Part A

AHRQ’s mission. As described in its 1999 reauthorizing legislation, Congress directed the Agency for Healthcare Research and Quality (AHRQ) to enhance the quality, appropriateness, and effectiveness of health services, as well as access to such services, by establishing a broad base of scientific research and promoting clinical and health systems practice improvements.¹ The legislation also directed AHRQ to “conduct and support research, evaluations, and training, support demonstration projects, research networks, and multidisciplinary centers, provide technical assistance, and disseminate information on healthcare and on systems for the delivery of such care, including activities with respect to health statistics, surveys, database development, and epidemiology.”²

Furthermore, AHRQ shall conduct and support research “to provide objective clinical information to healthcare practitioners and other clinicians of healthcare goods or services; identify the causes of preventable healthcare errors and patient injury in healthcare delivery; develop, demonstrate, and evaluate strategies for reducing errors and improving patient safety; and disseminate such effective strategies throughout the healthcare industry.”³

The safety of healthcare workers has been the focus of much research and regulation in the U.S. Healthcare workers face a wide range of hazards in the workplace, including, harmful exposures to chemicals and hazardous drugs, back injuries, latex allergy, violence, and stress.⁴ The Bureau of Labor Statistics (BLS) regarded nursing homes as one of the most dangerous workplaces in the U.S.⁵ In 2020, BLS reported that private industry nursing and residential care had a rate of 791.7 cases per 10,000 full-time equivalent (FTE) cases involving days away from work, an increase from a rate of 170.9 in 2019.⁶ Additionally, occupational stress and worker burnout are being recognized as significant threats to patient safety and quality of care.⁷ These injuries result in numerous indirect and less visible costs including employee turnover, training, overtime, incident investigation time, productivity, and morale.

Improving the safety of the healthcare system includes improving the safety of healthcare workers, which is a necessary precondition to advancing patient safety. In May 2018, the Institute for Healthcare Improvement (IHI) re-launched the patient safety agenda by convening the National Steering Committee for Patient Safety (NSC) to develop a national action plan for reducing harm in the delivery of health care.

The NSC was convened to leverage the knowledge of various stakeholders — influential federal agencies, leading health care delivery organizations and associations, patient and family leaders, and respected industry experts — into a set of actionable and effective recommendations.⁸ AHRQ is co-leading the NSC, which is co-chaired by Jeffrey Brady, M.D., Director of AHRQ’s Center for Quality Improvement and Patient Safety, and Tejal Gandhi, M.D., Chief Safety and Transformation Officer at Press Ganey. The NSC convened four subcommittees, with workforce safety being one of the four foundational areas:

Culture, Leadership, and Governance—focusing on the role of leaders, governing bodies, and policymakers to establish safety as a core value and promote cultures of safety.
Workforce Safety—focusing on the safety of the healthcare workforce as a necessary precondition to advancing patient safety.
Patient and Family Engagement—focusing on co-designing and co-producing care with patients and families.
Learning Systems—focusing on learning systems within and across health care organizations.

While the safety of healthcare workers has been a longstanding concern and is a core component of the NSC’s efforts, the recent emergence of the COVID-19 pandemic in early 2020 has brought the issue of healthcare worker safety even more to the forefront around the world. Shortages of proper personal protective equipment (PPE) have made healthcare workers more vulnerable to exposure to the virus. In addition, the physical and psychological well-being of healthcare workers is being tested as patient loads have increased and co-workers become infected with COVID-19, contributing significantly to burnout and a decline in the mental health of healthcare workers.⁹

Given the foundational importance of workplace safety as a precondition for patient safety, and renewed attention on the criticality of workplace safety as a result of the ongoing COVID-19 pandemic, AHRQ is undertaking timely work to develop survey items that will help nursing homes identify and improve workplace safety. The workplace safety survey items will assess the extent to which the nursing home’s culture supports workplace safety. The items will be developed as a new supplemental item set that can optionally be administered at the end of the AHRQ Surveys on Patient Safety Culture^™ (SOPS^®) Nursing Home Survey. By developing a supplemental item set, AHRQ will build this new measure of workplace safety upon its existing and highly successful SOPS survey program. SOPS surveys are completed by providers, staff, and administrators within healthcare organizations to assess the extent to which their organizational culture supports patient safety. There are SOPS surveys for hospitals, medical offices, nursing homes, ambulatory surgery centers, and community pharmacies. There are also supplemental items on health information technology patient safety for the hospital setting, diagnostic safety in the medical office setting, and value and efficiency in both the hospital and medical office setting.

In November 2021, AHRQ released a supplemental item set on workplace safety for the hospital setting. Now there is interest in developing a reliable, public-use item set that is limited in scope so it can be used in conjunction with the AHRQ SOPS Nursing Home Survey. The goal is to develop 15-20 items across 5-6 composite measures rather than developing a full-length survey.

The supplemental item set will be used by nursing homes to enable them to assess the organizational culture factors that contribute to workplace safety, and help them identify strengths and areas for improvement to efficiently target resources to improve workplace safety.

Most existing surveys on workplace safety focus on organizational, management, and provider and staff compliance with regulations, and are more like safety checklists. There is currently a dearth of validated survey items focusing on provider and staff perspectives about aspects of organizational culture that support workplace safety. We have been unable to find instruments that address workplace safety for the nursing home setting, in one, brief instrument, development of which is the objective of this research. To achieve this objective, we propose the following activities:

Conduct cognitive testing – The purpose of cognitive testing is to understand the cognitive processes respondents engage in when answering each draft item on the survey, which will help refine the survey instrument. Cognitive testing will be conducted with a mix of nursing home personnel--including clinicians, nurses, certified nursing assistants, and other types of staff such as administrative and support staff.

Cognitive interviews will be conducted with individual respondents to test the feasibility and applicability of the workplace safety supplemental items in nursing home settings. English cognitive interviews will be conducted in two rounds via Zoom.gov for up to 20 individuals from a nursing home setting. Limited Spanish cognitive testing will also be conducted with up to 5 individuals from nursing home settings.

Respondents will complete the draft supplemental items (see Attachment A). The cognitive interview guide found in Attachment B will be used during these interviews.

Feedback obtained from these interviews will be used to refine the supplemental items. The results of the cognitive testing, along with the proposed revisions, will be reviewed by the SOPS Technical Expert Panel and subject matter experts before proceeding with pilot testing.

Conduct pilot test data collection. A pilot test will be done in up to 75 nursing homes, and based on an analysis of results, we will determine which survey items to retain and refine the questionnaire accordingly.

We will plan one data collection effort of the draft supplemental items aimed at assessing the psychometric properties of the items and composite measures. We will assess the non-response, variability, reliability, factor structure and construct validity of the draft supplemental items and composite measures, allowing for their further refinement (see Part A, Section 16 for analysis plan description). The draft supplemental items (see Attachment A) will be added to the SOPS Nursing Home Survey and be administered to approximately 7,015 clinicians and staff from 75 nursing homes to facilitate analysis of the data. A nursing home point of contact (POC) will be recruited to publicize the pilot test of the supplemental items and assemble a list of sample clinicians and staff. We will recruit 75 nursing home points of contact (POC) or one per nursing home to publicize the pilot test of the nursing home supplemental item set. Instructions for the POCs are included in Attachment C, and Exhibit 2 includes a burden estimate for the POCs’ time in assisting with the pilot test. The supplemental item set will be administered via web or paper survey. Clinicians and staff will receive notification of the supplemental item set and reminders via email for a web survey and by hard copy for a paper survey. The draft pilot test notification and the weekly follow-up reminder notice are included in Attachment D.

Obtain Technical Expert Panel (TEP) and Subject Matter Expert (SME) feedback. The existing SOPS Technical Expert Panel (TEP) and additional subject matter experts (SMEs) in workplace safety will provide input to guide the development of the workplace safety supplemental item set. The TEP and SMEs will be engaged to provide feedback on survey drafts, and upon completion of the pilot test, to review results from the pilot test and finalize the supplemental items. The TEP and SMEs are discussed in more detail in Section 8. This activity does not impose a burden on the public and is therefore not included in the burden estimates in Section 12.

Disseminate the Survey Items. The final supplemental items will be made publicly available through the AHRQ website and we will conduct promotion and dissemination activities to encourage adoption and uptake by healthcare organizations. This dissemination activity does not impose a burden on the public and is therefore not included in the burden estimates in Section 12.

This work is being conducted by AHRQ through its contractor, Westat, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

2. Purpose and Use of Information

The responses from the cognitive testing of the draft workplace safety supplemental items will be used by project staff to test and improve the items and composite measures. Following cognitive testing results, we will revise the survey for pilot testing. Further, the information collected in the pilot test data collection effort will be used to test and improve draft supplemental items. Psychometric analysis will be conducted on the supplemental items data to examine item nonresponse, item response variability, factor structure, reliability, and construct validity of the items and composite measures. Because the items are being developed to measure specific aspects or composite measures of workplace safety, the factor structure of the items will be evaluated through confirmatory factor analysis. On the basis of the data analyses, items or composite measures may be dropped to create the final supplemental item set.

Nursing homes participating in the pilot test data collection effort will receive a report of their nursing home-specific results. This feedback report serves as one incentive for participation, and saves the nursing homes time and effort to analyze their own results.

The final supplemental item set will be made publicly available for nursing homes to assess how their culture supports workplace safety. The supplemental items can be used by nursing homes to identify areas for improvement related to their workplace safety culture. Researchers are also likely to use the supplemental items to assess the impact of workplace safety improvement initiatives.

3. Use of Improved Information Technology

The pilot test data collection will be conducted using a web or paper survey. According to the 2019 AHRQ SOPS Nursing Home Database, 43 percent of nursing homes administered the survey by web only, 42 percent administered the surveys by both paper and web, and 14 percent administered the survey by paper only¹⁰. In addition to reducing the burden associated with survey administration (printing and tracking paper surveys), a web-based survey will be offered to increase security of responses and eliminate data entry expense; however if the nursing home has a large percentage of staff who do not have email access while at work, we will offer that nursing home the opportunity to administer paper surveys.

4. Efforts to Avoid Duplication

We conducted a review of the literature, searching for staff surveys that measure workplace safety in nursing home settings. In reviewing the literature, we identified a number of topics that contribute to workplace safety: (1) exposure to workplace hazards; (2) moving, transferring, or lifting residents; (3) workplace aggression; (4) supervisor support for workplace safety; (5) management support for workplace safety; (6) workplace safety and reporting; (7) work stress/burnout; and (8) overall rating on workplace safety for staff. Listed below is information on related surveys that measure various aspects of workplace safety. Among the surveys found, there is less information on leader/management support for workplace safety, and reporting of hazards and incidents. There is a lack of a validated staff survey with a focus on organizational culture, rather than compliance with regulations, that addresses all of these relevant areas for use in the nursing home setting, succinctly captured in one, brief instrument.

Exposure to Workplace Hazards

The Health and Safety Practices Survey of Healthcare Workers¹¹ consists of one screener survey; one core survey; and seven module surveys, which each focus on a different type of exposure, such as aerosolized medication and surgical smoke. The core survey is 65 items and includes questions about exposure experiences with specific chemical agents and other hazardous materials. Because of the specificity of the questions, the survey is not as widely applicable as the AHRQ workplace safety supplemental items will aim to be.

Peterson, et al¹² included questions that focused on management commitment to safety, management feedback on safety procedures, coworker safety norms, worker involvement in health and safety issues, and worker training. In a report by the U.S. Bureau of Reclamation¹³, several items were included to measure worker safety perceptions. Several of these questions are specifically focused on correct use of respirators. Others are more general and ask about management and health and safety issues. These items are more similar to questions that may be included in the proposed AHRQ Nursing Home Workplace Safety supplemental item set, but focus on health and safety versus just safety. Finally, these items did not include all the topics on workplace safety we intend to capture.

Workplace Violence Surveys

There are many publications and surveys¹⁴^,¹⁵^,¹⁶^,¹⁷^,¹⁸ on workplace violence that measure specific incidences of physical, verbal, psychological, or sexual violence. However, none of these focused on the culture around healthcare workplace violence. Some had questions related to reporting or training, but they did not capture the workplace culture that could impact either of those elements.

The NY State Workplace Violence Prevention Program Guidelines¹⁹ included a list of sample questions that did touch on some culture aspects (via yes/no response option), however, the questions were limited to violence only while the proposed item set intends to cover more topics. Further, we have heard from experts that violence and/or aggression are not the appropriate terms when referring to behaviors from nursing home residents, as it puts intent in their actions, when these behaviors are often due to dementia or confusion.

Work Stress/Burnout

Several surveys assess burnout and stress among healthcare workers. We reviewed the Mini Z Burnout Survey²⁰, Maslach Burnout Inventory²¹, Nursing Occupational Stressor Scale (NOSS)²², Job Stress Scale²³, and American Psychological Association Workplace Survey²⁴. While all of these surveys measure burnout or stress, they do not include questions on the breadth of topics that the AHRQ SOPS Nursing Home Workplace Safety item set will measure. We have adapted a single item validated measure from the Mini Z to use in the item set to assess work stress and burnout with permission from the survey developer.

Other Components of Workplace Safety Surveys

Two surveys, the NDNQI RN Survey with Practice Environment Scale²⁵ and the Safe Patient Handling Staff Assessment Survey²⁶, both measure safe patient handling. The NDNQI survey asked about a specific program, the Safe Patient Handling and Mobility program. The wording for these items is too complex or specific to be used in the AHRQ Nursing Home Workplace Safety item set. The Safe Patient Handling Staff Assessment Survey is a collection of generally worded items questions on injury/incident reporting, safe patient handling training, and equipment availability. While these items could be adapted and tailored to the nursing home setting; they are not directly applicable. Further, these surveys were limited in measuring safe patient handling.

The Civil Money Penalty Reinvestment Program (CMPRP) and the Centers for Medicare and Medicaid Services (CMS) collaborated to develop the Nursing Home Employee Satisfaction Survey²⁷. The survey is a 38-item scale that measures employee satisfaction across four topic areas: job satisfaction; team building communication; scheduling and staffing training; and management and leadership. One survey item measures an aspect of workplace safety (My workplace is safe and well maintained) however is double barreled asking about if it is safe and if it is well-maintained. Further, the proposed supplemental item set intends to cover more topics.

General Well-being Surveys

The American Nurses Association (ANA) Enterprise launched a Healthy Nurse, Healthy Nation™ Grand Challenge in May 2017²⁸. Its purpose was to change the nation’s health by improving the health of nurses. Data was gathered through a 99 question survey that comprehensively assessed many aspects of health, safety, and wellness of the nursing workplace. While this survey asks questions that are applicable to nurses and workplace safety, it does not assess perceptions from all staff and providers in healthcare settings. The unique perspectives of other healthcare workers is important to capture in order to get an overall sense of the organizational culture around workplace safety.

National Nursing Assistant Survey²⁹, part of the National Nursing Home Survey, was the first national study of nursing assistants working in nursing homes. The survey, funded by the Office of the Assistant Secretary for Planning and Evaluation (ASPE), was conducted by the Centers for Disease Control and Prevention’s National Center for Health Statistics. The survey was conducted using a computer-assisted telephone interviewing (CATI) system with a sample of employees who provided daily living activity assistance to residents. The survey included 123 questions across 11 modules including management and supervision; client relations; organizational commitment and job satisfaction; workplace environment; and work-related injuries. The goal of the survey was to identify nursing assistant priorities and ways to meet those priorities, as well as identify ways to prevent future staffing shortages. While some questions are similar to questions that may be included in the proposed supplemental item set (e.g., “How often is a lifting device available when you actually need to use one”), these items generally assess descriptive types of information (e.g., “During the past 12 months how many times were you accidentally stuck with a needle while working”) rather than aspects of culture. Further, these items did not include all the topics on workplace safety we intend to include.

Compliance Surveys

Two surveys, the OHS Vulnerability Scale^³⁰ and the Health and Safety Practices Survey of Healthcare Workers, are used to measure, in part, healthcare workers’ compliance with using personal protective equipment (PPE). The OHS Vulnerability Scale assesses occupational health and safety (OHS) in hazard exposure; workplace policies and procedures; worker awareness of hazards and OHS rights and responsibilities; and worker empowerment to participate in injury and illness prevention. The draft items do include a section on Protection from Workplace Hazards, however they focus on the culture supporting protections, rather than strictly examining compliance.

In summary, while there are many surveys that assess specific components of workplace safety, we did not discover a brief, validated survey, covering the breadth of areas needed, that could be administered to all providers and staff as a supplement to the AHRQ SOPS Nursing Home Survey for assessing the organizational culture around workplace safety. Where possible we have examined the types of concepts in these surveys that are relevant, to develop the proposed supplemental items. As noted above, we have taken one survey item from the Mini Z to assess work stress and burnout with permission from the author. In conclusion this development and data collection effort is not duplicative of existing measures.

5. Involvement of Small Businesses

AHRQ designed the data collection instrument and procedures to minimize burden on individual nursing home respondents. The data requested of nursing homes represents the absolute minimum information required for the intended uses, without unduly burdening small nursing homes or others businesses.

6. Consequences if Information Collected Less Frequently

This effort is a one-time data collection.

7. Special Circumstances

The data collection efforts will be consistent with the guidelines at 5 CFR 1320.5(d)(2). No special circumstances apply.

8. Consultation Outside the Agency

The SOPS Technical Expert Panel (TEP) will be consulted to guide the development of the Workplace Safety supplemental items. As part of our standard survey development process, we engage the TEP on a regular basis. The TEP will review drafts of the supplemental items and the feedback from the cognitive interviews to assist in finalizing the supplemental items. The SOPS TEP includes 19 members from various parts of the health sector covered by the patient safety culture surveys, including nursing homes (see Attachment E).

9. Payments/Gifts to Respondents

Cognitive Interview Respondents. Incentives or honoraria are often offered to healthcare staff for participation in research. Respondents are being recruited for specific characteristics (e.g., a specific staff position within a nursing home setting). Recruitment of clinical staff is particularly challenging as there is a high turnover rate that has been exacerbated by COVID-19 and incentives can help increase interest so these individuals are able to be recruited in a timely manner. Additionally, cognitive interviews require a high level of effort for the respondent given that they are asked to explain their thought processes for selecting an answer to each proposed item, discuss what it means to them, and if there was anything confusing.

The amounts proposed for cognitive interview incentives are based on the average amounts typically paid by research recruitment vendors for these positions. As these vendors attest per their experience, healthcare positions, and physicians particularly, are very difficult to recruit for research participation unless incentives are sufficient. Therefore, to successfully recruit 25 cognitive interview participants for 1-hour cognitive interviews using established vendors, we propose $300 cash incentive for each of an estimated 3 physicians, $200 cash incentive for each of an estimated 8 registered nurses, and $150 cash incentive for each of an estimated 14 staff members (12 nursing assistants, 2 administrators). The amount for the incentives totals $4,600. These amounts are consistent with the amounts recently paid for the cognitive interviews used for testing the AHRQ Workplace Safety Supplemental Items for the Hospital Survey (as approved under OMB control # 0935-0124, expiry: 01-31-2024).

Pilot Test Sites. A $1,500 incentive is proposed for active nursing home participation. The first portion ($1,000) will be awarded at the start of data collection and the remaining $500 will be awarded if a 50 percent or higher response rate is achieved. Participating pilot nursing homes will have the AHRQ SOPS Nursing Home Survey and workplace safety supplemental items administered to providers and staff free-of-charge and will receive a customized feedback report that compares their results with the aggregated results from other participating nursing homes.

10. Assurance of Confidentiality

Individuals and organizations will be assured limitation on use of certain information under Section 944(c) of the Public Health Service Act, 42 USC 299c-3(c). This law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.

11. Questions of a Sensitive Nature

We do not consider survey questions related to the culture of workplace safety as particularly sensitive; however, if during cognitive testing we discover any sensitivities, we will modify or delete these questions accordingly and develop any modified survey protocols as needed.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated annualized burden hours for the participants’ time to take part in this research. Cognitive interviews for the supplemental items will be conducted with 25 individuals (approximately 3 physicians, 8 registered nurses, 12 nursing assistants, and 2 administrators) and require approximately one hour to complete.

For the pilot test, the supplemental items will be administered to about 7,015 individuals from 75 nursing homes and require 25 minutes to complete. Assuming a response rate of 60 percent, this data collection effort will yield a total of approximately 4,209 completed questionnaires. We expect an estimated 75 POCs, each representing a single nursing home, will complete the nursing home background information form (Attachment F) (completion is estimated to take about 3 minutes).

We estimate the total annualized burden is 1,431.75 hours.

Exhibit 2 shows the estimated annualized cost burden associated with the participants’ time to take part in this research. The total cost burden is estimated to be $33,219.70.

Exhibit 1. Estimated annualized burden hours

Form Name/Activity	Number of respondents/POCs	Number of responses per respondent	Hours per response	Total burden hours
Cognitive interviews	25	1	1	25
Nursing home background information form	75	1	3/60	3.75
Pilot test	4,209	1	20/60	1,403
Total	4,309	na	na	1,431.75

Exhibit 2. Estimated annualized cost burden

Form Name/Activity	Total burden hours	Average hourly wage rate*	Total cost burden
Cognitive interviews^a	25	$30.89	$772.16
Nursing home information survey^b	3.75	$40.65	$152.44
Pilot test^c	1,403	$23.09	$32,396.73
Total	1,431.75	na	$33,219.70

^a Based on the weighted average wages for 3 General Internal Medicine Physicians (29-1216; $79.30), 8 Registered Nurses (29-1141; $34.36), 12 Nursing Assistants (31-1131; $14.84), 1 Administrative Services and Facilities Manager (11-3010; $40.65);

^b Based on the average wages for Administrative Services and Facilities Managers (11-3010; $40.65) in the nursing home setting;

^c Based on the weighted average wages for 130 General Internal Medicine Physicians (29-1216; $79.30), 854 Registered Nurses (29-1141; $34.36), 2,850 Nursing Assistants (31-1131; $14.84), 375 Administrative Services and Facilities Managers (11-3010; $40.65), in the nursing home setting;

* National Occupational Employment and Wage Estimates in the United States, May 2020, “U.S. Department of Labor, Bureau of Labor Statistics” (available at https://www.bls.gov/oes/current/naics3_623000.htm [for nursing home setting]

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.

14. Estimates of Annualized Cost to the Government

Exhibit 3 shows the estimated annualized cost to the Government, which is estimated at $21,570.

Exhibit 3. Estimated Annual cost to AHRQ for Project Oversite [AHRQ TO COMPLETE]

AHRQ Position	% Time	Annualized Cost
GS 15 Step-5	5%	$8,264
GS 14 Step 5	5%	$7,253
GS 13 Step 5	5%	$6,053
Total		$21,570

15. Changes in Hour Burden

This data collection effort is a new activity.

16. Time Schedule, Publication and Analysis Plan

As soon as OMB approval is received, cognitive testing activities will begin. The estimated time schedule to conduct these activities is shown below:

Complete up to 25 cognitive interviews (3 months)
Pilot test data collection (4 months)
Data analysis, feedback report production, and development of technical reports (6 months)
Final Nursing Home Workplace Safety supplemental item set and development of toolkit materials (1 month)

The final version of the Nursing Home Workplace Safety supplemental item set, technical reports, and accompanying toolkit materials will be made publicly available on the AHRQ website.

This section describes the specific analyses that we will conduct on the pilot test data.

Pilot Test Psychometric Analysis. Psychometric analysis will be conducted to examine item nonresponse, item response variability, factor structure, reliability, and construct validity of the items. Because the supplemental items are being developed to measure specific aspects or composite measures of culture of workplace safety, the factor structure of the supplemental items will be evaluated through confirmatory factor analysis. On the basis of the data analyses, items or composite measures may be dropped.

Descriptive Statistics

We will examine item frequencies to review response variability and identify items with high percentages of missing data or Does Not Apply/Don’t Know (DNA/DK) responses. Items with little response variability may not be helpful in differentiating higher-scoring from lower-scoring nursing homes. For example, any items with more than 90 percent of respondents responding positively (e.g., those answering Strongly agree/Agree or Always/Most) may be considered to have low variability. Additionally, if more than 30% of respondents leave an item missing or answer DNA/DK, the item may be considered poorly functioning, as it may not be relevant to a large proportion of respondents.

Confirmatory Factor Analysis

A confirmatory factor analysis will be conducted to initially examine whether groups of items intended to measure a specific composite measure are interrelated, ignoring the nesting of respondent data within nursing homes. Factor loadings for each item in an a priori composite measure will be considered as having an adequate contribution to a particular composite measure or factor if the strength of the item's relationship to that factor (i.e., its factor loading), is 0.40 or greater.

We will also examine overall model fit indices using standard fit statistics: the chi-square, comparative fit index (CFI), and the standardized root mean square residual (SRMR). For chi-square statistics, lower and non-significant chi-squares indicate good fit. The factor structure is determined to adequately fit the data if the CFI is at least 0.90. A value of zero for the SRMR indicates perfect fit, but a value less than 0.08 is considered a good fit.

Site-level Reliability

To examine the variability of item scores within nursing homes compared to between nursing homes we will compute site-level reliability. Site-level reliability indicates the extent to which responses given by staff within the same nursing home are more similar to each other than they are to responses given by staff from other nursing homes. In other words, site-level reliability helps to assess how well survey measures differentiate nursing homes. It does so by comparing between-site variability to within-site variability of survey items. Values of 0.70 or higher are considered acceptable for site-level reliability.

Internal Consistency Reliability

Reliability analyses will then be performed on the composite measures to examine whether individuals responded consistently to the items within each composite measure. Internal consistency reliability will be calculated using Cronbach's alpha. The minimum criterion for acceptable reliability is an alpha of at least 0.70.

Intercorrelations

Intercorrelations among the supplemental item set’s composite measures will also be examined. Intercorrelations will be explored at the nursing home level of analysis. The survey measures should be intercorrelated, as they are designed to assess aspects of culture focusing on workplace safety. Moderate to moderately high correlations typically indicate a correspondence or convergence among similar concepts.

The above analyses will be used to determine which, if any, items and composite measures are functioning poorly and remove them from the survey to derive a final set of items and composite measures with good psychometric properties and reduce the overall length of the final supplemental item set. The Technical Expert Panel and subject matter experts will be informed of the data analysis results and asked to advise Westat on which items to retain or drop (when the psychometric results do not provide enough guidance and decisions can be made on the content value of the items).

The final supplemental item set will be made publicly available on the SOPS pages of the AHRQ website for use by healthcare organizations and researchers.

17. Exemption for Display of Expiration Date

No exemption is requested.

List of Attachments

Attachment A: Draft Workplace Safety Supplemental Item Set

Attachment B: Draft Workplace Safety Cognitive Interview Guide

Attachment C: Nursing Home Point of Contact (POC) Instructions

Attachment D: Survey Invitation and Reminder Notices

Attachment E: Subject Matter Experts and Technical Expert Panel Members

Attachment F: Nursing Home Background Information Form

1 Healthcare Research and Quality Act of 1999. Available at https://www.ahrq.gov/policymakers/hrqa99a.html. Last accessed 10/21/2021.

2 See Section 902, (a) (8) of the Healthcare Research and Quality Act of 1999. Available at https://www.ahrq.gov/policymakers/hrqa99a.html. Last accessed 10/21/2021.

3 See Section 912, (b) (2) (A) (ii) (I) and (iii) (II) and (c) (1) (2) and (3) of the Healthcare Research and Quality Act of 1999. Available at http://www.ahrq.gov/policymakers/hrqa99b.html. Last accessed 10/21/2021.

4 Centers for Disease Control/National Institute for Occupational Safety and Health (NIOSH). Available at https://www.cdc.gov/niosh/topics/healthcare/default.html. Last accessed 10/21/2021.

5 Lapane, K., Dube, C., & Jesdale, B. (2016). Worker Injuries in Nursing Homes: Is Safe Patient Handling Legislation The Solution? J Nurs Home Res Sci. Author manuscript, October 28; 2: 110-117.

6 Bureau of Labor Statistics News Release. November 3, 2021. Available at Employer-Reported Workplace Injuries and Illnesses – 2020 (bls.gov). Last accessed 11/18/2021.

7 White, E., Aiken, L., and McHugh, M. (2019). Registered Nurse Burnout, Job Dissatisfaction, and Missed Care in Nursing Homes. Journal American Geriatric Society. Author manuscript, October: 67(10): 2065-2071.

8 A National Strategy for Reducing Harm. Institute for Healthcare Improvement, Boston, MA. Available at http://www.ihi.org/Engage/Initiatives/National-Steering-Committee-Patient-Safety/Pages/default.aspx. Last accessed 10/21/2021.

9 Ehrlich, H., McKenney, M., & Elkbuli, A. (2020). Protecting our healthcare workers during the COVID-19 pandemic. The American Journal of Emergency Medicine, 38(7), 1527–1528 https://doi.org/10.1016/j.ajem.2020.04.024)

10 AHRQ SOPS Nursing Home Survey on Patient Safety Culture 2019 User Database Report. Available at Nursing Home Survey on Patient Safety Culture: 2019 User Database Report Part I (ahrq.gov). Last accessed 10/21/2021.

11 The National Institute for Occupational Safety and Health (NIOSH) (2014). Health and Safety Practices Survey of Healthcare Workers [Survey]. Available at https://www.cdc.gov/niosh/topics/healthcarehsps/aboutstudy.html. Last accessed 10/21/2021.

12 Peterson, K., Rogers, B.M.E., Brosseau, L.M., Payne, J., Conney, J., Joe, L., & Novak, D. (2016). Differences in hospital managers’, unit managers’, and health care workers’ perceptions of the safety climate for respiratory protection. Workplace Health Saf., 64(7), 326-336

13 U.S. Bureau of Reclamation. (2016). Baseline assessment of employee perceptions of safety culture. https://www.usbr.gov/safety/sap/documents/teamreports/Team14.pdf. Last accessed 10/21/2021.

14 The American College of Emergency Physicians. (2018). ACEP Emergency Department Violence Poll research results. https://www.emergencyphysicians.org/globalassets/files/pdfs/2018acep-emergency-department-violence-pollresults-2.pdf. Last accessed 10/21/2021.

15 Texas Center for Nursing Workforce Studies. (2016). Workplace Violence Against Nurses Survey (Publication No. 25-14922). Texas Department of State Health Services. https://www.dshs.state.tx.us/chs/cnws/WorkforceReports/2016_WPVAN.pdf

16 Appendix 1. SOVES-A questionnaire. (2013). Clinical Medicine, 13(6), 1. https://doi.org/10.7861/clinmedicine.13-6-1

17 International Labour Office, International Council of Nurses, World Health Organization, Public Services International. (2003). Workplace Violence in the Health Sector Country Case Study – Questionnaire. Available at

https://www.who.int/violence_injury_prevention/violence/interpersonal/en/WVquestionnaire.pdf?ua=1. Last accessed 10/21/2021.

18 Emergency Nurses Association (2011). Emergency Department Violence Surveillance Study. Institute for Emergency Nursing Research. https://www.ena.org/docs/default-source/resource-library/practice-resources/workplace-violence/2011-emergency-department-violence-surveillance-report.pdf?sfvrsn=5ad81911_4. Last accessed 10/21/2021.

19 New York Department of Labor. (2009). NY State Workplace Violence Prevention Program Guidelines.

https://dol.ny.gov/workplace-violence-prevention-information

20 Mini-Z Burnout Survey. Available at https://www.christushealth.org/-/media/files/for-providers/ama-mini-z-burnout-survey.ashx?la=en. Last accessed 10/21/2021.

21 Maslach Burnout Inventory™. Available at https://www.mindgarden.com/117-maslach-burnout-inventory. Last accessed 10/21/2021.

22 Chen, Y., Guo, Y.L. Lin, L., Lee, Y., Hu, P., Ho, J. & Shaio, J.S. (2020). Development of the Nurses Occupational Stressor Scale. International Journal of Environmental Research and Public Health., 17, 649.

23 Job Stress Scale. Available at https://www.tandfonline.com/doi/full/10.1080/23311975.2015.1134034. Last accessed 10/21/2021.

24 APA Workplace Survey. Available at https://www.apa.org/pubs/reports/work-well-being Last accessed 11/2/2021.

25 Press Ganey (2018). NDNQI RN Survey with Practice Environment Scale^©[Survey]. Available at https://www.upstate.edu/nursing/documents/ndnqi_rn_survey_with_pratice_environment_scale.pdf. Last accessed 10/21/2021.

26 Lax, M., Zoeckler, J., Goessling, K., Greetham, S. (2019). New York State’s Safe Patient Handling Law. Occupational Safety and Health Working Group. http://ohccupstate.org/index_htm_files/Safe%20Patient%20Handling%20Survey%20Report%20April%202019%20FINAL.pdf. Last accessed 10/21/2021.

27 The Civil Money Penalty Reinvestment Program and the Centers for Medicare and Medicaid Services Nursing Home Employee Satisfaction Survey. Available at https: https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/CMPRP-Toolkit2-Employee-Satisfaction-Survey.pdf . Last accessed 10/21/2021.

28 American Nurses Association Healthy Nurse, Healthy Nation™. Available at https://www.healthynursehealthynation.org/about/survey/. Last accessed 11/2/2021.

29 Centers for Disease Control and Prevention’s National Center for Health Statistics (2014). National Nursing Assistant Survey. Funded by the Office of the Assistant Secretary for Planning and Evaluation. https://www.cdc.gov/nchs/nnhs/nnas.htm. Last accessed 10/21/2021.

30 Institute for Work & Health (2015). OHS Vulnerability Measure [Survey]. Available at https://www.iwh.on.ca/sites/iwh/files/iwh/tools/ohs_vulnerability_measure_guide_2017.pdf. Last accessed 10/21/2021.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2024-07-28

Supporting Statement Part A