SUPPORTING STATEMENT
Part A
Safety Program in Perinatal Care (SPPC)-II Demonstration Project
Version: June 23, 2020
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 6
3. Use of Improved Information Technology 7
4. Efforts to Identify Duplication 7
5. Involvement of Small Entities 7
6. Consequences if Information Collected Less Frequently 7
7. Special Circumstances 8
8. Consultation outside the Agency 8
9. Payments/Gifts to Respondents 9
10. Assurance of Confidentiality 9
11. Questions of a Sensitive Nature 10
12. Estimates of Annualized Burden Hours and Costs 10
13. Estimates of Annualized Respondent Capital and Maintenance Costs 12
14. Estimates of Annualized Cost to the Government 12
15. Changes in Hour Burden 12
16. Time Schedule, Publication and Analysis Plans 13
17. Exemption for Display of Expiration Date 13
List of Attachments 14
This request is for an update to previously submitted OMB supporting statements for revisions to the OMB clearance for the data collection of the Safety Program in Perinatal Care Demonstration Project (SPPC-II), which was approved in November 2019 and has an expiration date of November 30, 2022. Revisions are requested at this time due to the impact of the pandemic on our study population (i.e., hospital staff) in Oklahoma and Texas.
We propose updating the SPPC-II data collection by 1) changing the data collection intervals of the clinical staff self-administered implementation survey from 6, 18, and 30 months to 6, 12, and 18 months, 2) adding questions to the approved qualitative interview guide at 3-4 months (Appendix J now Appendix J1) to include pandemic-related questions to better understand the implementation context, 3) adding an additional qualitative interview collection at 15-16 months with a new interview guide (Appendix J2) to better understand the implementation context, and 4) increasing the total number of qualitative interview participants from 25 to 30 participants to account for the two qualitative interview collections at 3-4 months and 15-16 months. The total estimated annual burden hours for SPPC-II will increase from 54,654 hours in the previous clearance to 54,659 hours in this clearance request, an increase of 5 hours.
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
Maternal mortality and severe maternal morbidity (SMM) increased significantly and continuously in the United States (US) over the past 30 years.1 A considerable proportion of these adverse events are attributable to preventable harm and unintended consequences arising from clinical practice and the system of delivering perinatal care.2 To address these alarming trends, AHRQ has developed the Safety Program in Perinatal Care (SPPC).3 During its initial phase (SPPC-I), the program was comprised of three pillars: teamwork and communication, patient safety bundles, and in situ simulations.3 Despite several promising results, the evaluation of SPPC-I revealed considerable hospital attrition due to heavy data burden and competing safety initiatives.4,5 Also, differences in the local adaptation of the SPPC-I patient safety bundles selected by implementation sites thwarted a meaningful cross-site comparison of programmatic impact.4,5
The current, second phase of the program (SPPC-II), focuses on integrating the teamwork and communication pillar into patient safety bundles developed by key professional organizations and implemented in 20+ US states with technical assistance by the Alliance for Innovation on Maternal Health (AIM)6 program and funding from the Health Resources and Services Administration (HRSA). Of note, the model used by AIM to implement these bundles is through statewide perinatal quality collaboratives (PQC) aiming to enroll all birthing hospitals in the state in the PQC.
During the Planning Phase of SPPC-II, the contractor, Johns Hopkins University (JHU), developed SPPC-II Training Toolkits for two AIM patient safety bundles: obstetric hemorrhage and severe hypertension in pregnancy. The aim of the SPPC-II Demonstration Project is to implement and evaluate an integrated AIM-SPPC II program that overlays the SPPC-II Training Toolkits and the AIM patient safety bundles and program infrastructure in two states -- Oklahoma (OK), currently implementing the severe hypertension bundle; and Texas (TX), currently implementing the hemorrhage bundle.
Over the next five years, with additional funding from HRSA, the AIM program is expected to cover about two thirds of US states. Therefore, there is need to determine the feasibility and impact of the proposed integrated AIM-SPPC II program, and inform future government funding decisions regarding these two programs.
To this end, the SPPC-II Demonstration Project has the following goals:
1) To implement the integrated AIM-SPPC II program in birthing hospitals in OK and TX in coordination with AIM and the respective state PQC;
2) To assess the implementation of the integrated AIM-SPPC II program in these hospitals; and
3) To ascertain the short- and medium-term impact of the integrated AIM-SPPC II program on hospital (i.e. perinatal unit) teamwork and communication, patient safety, and key maternal health outcomes.
To achieve the goals of this project the following data collections will be implemented:
Training of AIM Team Leads from 48 birthing hospitals in OK and 210 birthing hospitals in TX (i.e., all birthing hospitals enrolled in the respective state PQC) on using teamwork and communication tools and strategies in clinical obstetric practice. The training will be conducted in-person, through a full-day workshop organized in collaboration and coordination with the AIM program and state PQCs, and led by JHU. Only one such training workshop will be conducted in OK using the SPPC-II Toolkit for severe hypertension in pregnancy. Given the size of the state, potential long distances to be traveled by trainees, and the cost-efficiency of coordinating with back-to-back regional PQC meetings planned in TX this fall, five training workshops will be conducted in this state using the SPPC-II Toolkit for obstetric hemorrhage. We expect about half of the birthing hospitals in both states to send 2 hospital champions, of which one to be designated as AIM Team Lead, for training. JHU will keep and bi-annually update a roster of AIM Team Leads in each hospital to assess the need for training of new AIM Team Leads if turnover occurs (Appendix A). Training workshop evaluation forms (Appendix B) will be distributed for completion by trainees on a voluntary basis to assess the perceived utility of training workshops.
Training of all frontline clinical staff in 48 birthing hospitals in OK and 210 birthing hospitals in TX on teamwork and communication tools and strategies will be coordinated by AIM Team Leads in each hospital by: a) providing unique trainee IDs and information for them to access 8 training e-modules online (with option to leave voluntary comments/suggestions), and b) using the JHU-developed facilitator guide included in the SPPC-II Toolkits to facilitate brief, in-person demonstration sessions on how to use the information from the training e-modules in clinical practice. Each of the eight training e-modules will take about 15 minutes to complete online, for a total of about 120 minutes. Because these training e-modules will be accessed and completed online, tracking of e-module completion and re-take, needed to assess overall staff exposure to training, is possible through the online training platform. The specific tracking measures are included in a list in the Appendix (Appendix C).
Coaching calls will be organized monthly and led by JHU to address program implementation questions and assist with potential challenges. AIM Team Leads in all Demonstration Project hospitals will be invited to join these calls and ask questions. A list of coaching call participants and topics addressed will be maintained by JHU (Appendix D).
AIM Team Lead self-administered baseline surveys will be made available 2-3 weeks before the AIM Team Leads training workshop (Appendix E), together with a corresponding consent form (Appendix F). The purpose of this survey is to assess key characteristics of project hospitals, including human resources, processes in place for AIM bundle implementation, and use of teamwork and communication tools in clinical practice. Respondents will have the option to complete the survey online or on paper, in line with the current administration of the Hospital Survey on Patient Safety Culture.7 The expected response rate for this survey is 95% in both states.
Clinical staff self-administered baseline surveys will be made available about a month before the SPPC-II implementation start date (Appendix G), together with a corresponding consent form (Appendix H). The purpose of this survey is to assess baseline levels of previous teamwork and communication training, overall use of teamwork and communication tools and strategies, teamwork and communication perceptions, experience with AIM bundle implementation. Three respondents will be randomly selected in each hospital using comprehensive lists of clinical staff developed by the AIM Team Leads (Appendix I). These lists will be updated by AIM Team Leads on a quarterly basis to capture new hires and staff turnover. Respondents will be given the option to complete the survey online or on paper, in line with the administration of the national Hospital Survey on Patient Safety Culture.7 The expected response rate for this survey is 85% in both states.
Qualitative, semi-structured interviews with AIM Team Leads will be conducted by phone about 3-4 months and 15-16 months after the SPPC-II implementation start date to assess the perceived utility of the training and assistance needed with the rollout of training to all frontline clinical staff using the e-modules and facilitation sessions to consolidate the information, and to better understand the implementation context (including barriers, facilitators, and strategies). An interview guide developed based on the Consolidated Framework for Implementation Research framework will be used to conduct the interviews (Appendices J1, revised, and J2, new), together with a corresponding consent form (Appendix K).
Clinical staff self-administered implementation surveys will be made available at about 6, 12, and 18 months after the SPPC-II implementation start date (Appendices L-N), together with a corresponding consent forms (Appendices O-Q), to assess training knowledge, transfer, and results such as use of teamwork and communication tools and strategies, teamwork and communication perceptions, experience with AIM bundle implementation overlaid with the teamwork and communication tools. The time points were chosen to assess: early adoption and results of the training (6-month survey); adoption and results of the training at the time when unit culture changes are expected per available implementation research8-12 (12-month survey); and medium-term program sustainability (18-month survey). For each survey, three respondents will be randomly selected in each hospital using the most up to date comprehensive lists of clinical staff. Respondents will have the option to complete these surveys online or on paper, in line with the administration of the national Hospital Survey on Patient Safety Culture.7 The expected response rates are 80%, 77.5% and 75% for surveys completed at 6, 12 and 18 months after the SPPC-II implementation start date, respectively.
AIM program data will be obtained under data use agreements (DUA) with coordinating bodies of state PQCs or individual hospitals, as needed (Appendix R includes key provisions for these DUAs; additional provisions may be added at the time of signature). The list of mandatory measures already reported to the AIM program on a quarterly basis by all hospitals is included in Appendix S; additional measures may be added and required for reporting by the AIM program or the state PQCs over the course of our project. These data are needed for the evaluation of the SPPC-II Demonstration Project to assess changes in key SPPC-II program processes and maternal health outcomes, such as severe maternal morbidity, throughout the project.
This study is being conducted by AHRQ through its contractor, Johns Hopkins University (JHU) and the AIM program, JHU’s subcontractor, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
The information collected for this Demonstration Project will be used to evaluate the implementation and impact of the SPPC-II program overlaid with AIM safety patient bundles in birthing hospitals in OK and TX. More specifically, the project will:
provide information on whether the proposed integration of AIM and SPPC-II programs can be implemented as intended, i.e. through the use of a two-tier approach for training all clinical staff in all hospitals, coordination by the AIM Team Lead of the rollout of training clinical staff using e-modules on teamwork and communication, facilitation by AIM Team Leads of in-person sessions to practice teamwork and communication tools and strategies; or, what changes are needed to better facilitate program implementation;
provide information regarding the impact of the integrated AIM-SPPC II program on use of teamwork and communication tools and strategies, teamwork and communication metrics, patient safety culture changes, AIM bundle implementation, and key maternal health outcomes; and
provide information regarding the sustainability of the integrated AIM-SPPC II program 18 months after implementation.
Results from the evaluation of the SPPC-II Demonstration Project will be disseminated widely through conference presentations at professional organization and state PQC meetings, publication in peer-reviewed journals, and reports published on the AHRQ website. This will facilitate the wide dissemination of lessons learned and potential challenges with implementation of quality improvement initiatives in obstetrics in the US. There is limited knowledge regarding the impact of implementing AIM patient safety bundles, yet there are promising preliminary results from both SPPC-I and AIM program implementation in California, Illinois and Florida.13 Therefore, the information from SPPC-II Demonstration Project will be used by two federal agencies, AHRQ and HRSA, as they consider the next steps with and funding for the SPPC and AIM programs, respectively.
Because training of frontline clinical staff will be online using the training e-modules developed by JHU, tracking of unique staff IDs to assess e-module completion or re-take is automated, reducing on the burden of collecting this information on paper.
All survey respondents will be given the option to complete the surveys online, thus permitting the electronic submission of responses and reducing the burden of data entry and minimizing potential related errors. Respondents are given the option to complete the surveys on paper as such administration approaches have been shown to yield higher response rates than online completion.7
Also, data to be obtained from the SPPC-II hospitals or state PQCs will be shared electronically, which minimizes burden and avoids data entry errors.
There is no other study that has been conducted to assess the implementation and impact of integrating teamwork and communication trainings within AIM patient safety bundles. This is known from direct communication with AIM program leadership, which would need to be involved in any such research efforts. However, some of the data of key interest for the evaluation of the SPPC-II Demonstration Project (i.e. both process and outcome measures; see Appendix S) are collected by the AIM program. To avoid duplication in data collection, these data will be obtained from the state PQCs or SPPC-II hospitals under data use agreements.
The SPPC-II Demonstration Project does not involve collection of information from small entities.
AIM Team Lead self-administered baseline surveys are a one-time data collection.
Clinical staff self-administered baseline surveys are a one-time data collection.
Qualitative, semi-structured interviews with AIM Team Leads are a two-time data collection: 3-4 months and 15-16 months after the SPPC-II implementation start date.
Clinical staff self-administered implementation surveys will be conducted three times. Given that most of these surveys are expected to be completed online, the burden involved by this data collection frequency is reduced considerably. The timing proposed for these surveys is considered minimum, yet sufficient, to assess all: a) early adoption and results of the training (6-month survey); b) adoption and results of the training at the time when unit culture changes are expected per available implementation research8-12 (12-month survey); and, c) medium-term program sustainability (18-month survey). If conducted less frequently, in light of the covid-19 pandemic, some of this information will be missing, leading to a gap in our knowledge and learning from this project.
AIM program data will be obtained on the same schedule by which they are reported by hospitals, i.e. quarterly basis. The burden associated with the electronic sharing of these data is minimal.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), a notice was published in the Federal Register July 16th on page 43239 for 60 days (see Attachment X). AHRQ did not receive comments on this notice.
During the Planning Phase of SPPC-II, AHRQ and JHU consulted regularly with AIM program leadership and HRSA to obtain information about the availability of data relevant to this Demonstration Project, the proposed data collection strategies, the frequency of data collection, specific data elements to be obtained through the proposed data collection. Of note, the AIM program is a subcontractor of JHU for the Demonstration Project of SPPC-II, a relationship that involves weekly meetings between AIM and JHU, and monthly meetings with all AIM, JHU, AHRQ and, HRSA representatives. Moreover, throughout the Demonstration Project, JHU has a contractual obligation to convene four meetings of a Stakeholder Panel comprised of AHRQ, HRSA, JHU, AIM, OK-PQC, TX-PQC representatives as well as six external panelists with expertise in obstetrics, quality improvement, data and evaluation methods (Appendix T). These panelists will provide guidance on all Demonstration Project implementation and evaluation activities.
All study participants will be offered a $10 Amazon gift card as a token of appreciation for their time and participation in the study. This decision is based on experience with other AHRQ data collection; findings from the published literature; and the need to equally compensate respondents irrespective of the manner in which they complete the surveys (online or on paper).
For AHRQ’s Medical Expenditure Panel Survey (MEPS), a monetary gift has been offered to respondents since 1996; the current amount is $50 for the five times when contacted for the surveys – this corresponds to $10 per time spent on the MEPS at each time.
Findings from recent (2016-2018) high-quality studies show:
provision of compensation to Research Participant Perception Survey respondents increased survey completion rate from 54% to 71% (p<0.001); 14
a $10 monetary incentive encouraged initially reluctant participants to participate in the longitudinal World Trade Center Health Registry, increased the likelihood of returning a survey by 30% for those who received an incentive than not (AOR=1.3, 95% CI: 1.1-1.4), and increased the number of returned surveys by 18%;15
conversely, modest non-monetary incentives were not found to increase response rates certain categories of healthcare providers. For example, of providers who were offered a book, 11.6% responded compared with 10.7% who were not (OR=1.10, 95% CI 0.87-1.38, P=0.42).16
The choice of offering a $10 gift card to the most widely used online store has the benefit of being available to the respondents for use immediately after completing the interviews, and to reduce logistical complications and time burden associated with distributing cash (or any other type of incentives) to staff in 250+ hospitals in 2 states.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.
Except for hospital names and clinical staff names and email addresses, no other personally identifiable information (PII) will be collected during the SPPC-II Demonstration Project. Names of hospitals and clinical staff will be recorded, but replaced with anonymized hospital IDs and staff IDs; email addresses for staff are needed to share information about access to training e-modules and survey questionnaires if these are to be completed online. Data obtained from hospitals or state PQCs will be de-identified (hospital names will be replaced with our hospital study IDs), and a data use agreement for receiving and using these data will be signed by representatives at all hospitals before data are shared. No hospital or staff names or email addresses will be included in any reports, presentations, or other publications emerging from this project. Only aggregated, de-identified results will be displayed in any reports.
All project data, including the files linking hospital and staff names and study IDs, will be stored in a secure, password-protected electronic shared folder (JHUBox), which is widely and routinely used by JHU faculty and staff for government-funded projects given the secure access it provides. A specific folder for the SPCC-II Demonstration Project will be developed to which only project staff who need to use the data or have managerial roles will have access. Only JHU team investigators will have access to project data on the secure JHUBox folder. Moreover, access to the JHUBox SPCC-II folder with project data will only be granted to project staff who need to use the data or have managerial roles. All project staff must have an official JHU email address to be granted access to the secure JHUBox folder. Protocols for data collection, storage, and analyses will be approved by the JHU Institutional Review Board and followed accordingly. All Demonstration Project data will be destroyed 3 years following completion of the project.
Confidentiality statements will be printed on respondent materials (e.g. questionnaires) using the following text: “Your responses will be kept confidential to the extent permitted by law, including AHRQ’s confidentiality statute, 42 USC 299c-3(c).”
Also, to describe the AHRQ statute, we also added the following sentence: “That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied unless you consent to the use of the information for another purpose.”
Exhibit 1 shows the estimated annualized burden ours for the respondents’ time to participate in the SPPC-II Demonstration Project.
An estimated 387 AIM Team Leads from the 258 Demonstration Project sites will be trained during 8-hour workshops using the SPPC-II Toolkit. An evaluation form, which will take approximately 5 minutes to complete, will be distributed to them at the end of the workshop, and about 75% of them (290 AIM Team Leads) are expected to complete the evaluation. They will also be asked to extract from an available human resources computerized database and update bi-annually rosters of frontline clinical staff in their units – first extraction and each update is expected to take about 5 minutes.
An estimated 15,480 frontline clinical staff are expected to be trained using the training e-modules in the SPPC-II Toolkit. Completion of the 8 e-modules will take about 2 hours. These trainings will be complemented by four 15-min facilitation sessions led by AIM Team Leads in their respective units. The AIM Team Leads will track attendance of the facilitation session, work estimated to take about 15 minutes after each session.
Monthly 1-hour coaching calls will be organized during the first 18 months of the project and at least one representative from about half of the sites is expected to participate at each coaching call.
Several surveys will be administered throughout the Demonstration Project, specifically: baseline, 20-minute surveys with AIM Team Leads at each of 258 sites; baseline, 25-minute surveys with frontline clinical staff at each SPPC-II hospital; 30-minute implementation surveys with frontline clinical staff at each SPPC-II hospital will be conducted at 6, 12, and 18 months after the initial training workshops in both states. In addition, one-hour qualitative interviews will be conducted with a total of 30 AIM Team Leads in the 2 states about 3-4 months and 15-16 months after the SPPC-II implementation start date.
We will inform AIM Team Leads of the DUAs put in place to access their hospital's AIM program data – this will take about 5 minutes.
The total annual burden hours are estimated to be 54, 654 hours.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Training of AIM Team Leads |
387 |
1 |
8 |
3,096 |
Frontline staff rosters developed by AIM Team Leads |
258 |
6 |
0.08 |
124 |
Evaluation form for training of AIM Team Leads |
290 |
1 |
0.08 |
23 |
Training of frontline clinical staff |
15,480 |
1 |
2.00 |
30,960 |
Facilitation sessions |
15,480 |
4 |
0.25 |
15,480 |
Tracking attendance of facilitation sessions |
258 |
4 |
1.00 |
1,032 |
Coaching calls |
129 |
18 |
1.00 |
2,322 |
Self-administered baseline surveys with AIM Team Leads |
258 |
1 |
0.33 |
85 |
Self-administered baseline surveys with clinical staff |
774 |
1 |
0.42 |
325 |
Qualitative semi-structured interviews with AIM Team Leads |
30 |
1 |
1.00 |
30 |
Self-administered implementation surveys with clinical staff at 6 months |
774 |
1 |
0.50 |
387 |
Self-administered implementation surveys with clinical staff at 18 months |
774 |
1 |
0.50 |
387 |
Self-administered implementation surveys with clinical staff at 30 months |
774 |
1 |
0.50 |
387 |
DUA for AIM data |
258 |
1 |
0.08 |
21 |
Total |
35,924 |
NA |
NA |
Exhibit 2 shows the estimated annualized cost burden based on the respondents’ time to submit their data. The cost burden is estimated to be $1,489,998.34 annually.
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of respondents
|
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Training of AIM Team Leads |
387 |
3,096 |
$ 49.83 |
$ 154,273.68 |
Frontline staff rosters developed by AIM Team Leads |
258 |
124 |
$ 49.83 |
$ 6,178.92 |
Evaluation form for training of AIM Team Leads |
290 |
23 |
$ 49.83 |
$ 1,146.09 |
Training of frontline clinical staff |
15480 |
30960 |
$ 66.32 |
$ 2,053,267.20 |
Facilitation sessions |
15480 |
15480 |
$ 66.32 |
$ 1,026,633.60 |
Tracking attendance of facilitation sessions |
258 |
1032 |
$ 49.83 |
$ 51,424.56 |
Coaching calls |
129 |
2322 |
$ 66.32 |
$ 153,995.04 |
Self-administered baseline surveys with AIM Team Leads |
258 |
85 |
$ 49.83 |
$ 4,235.55 |
Self-administered baseline surveys with clinical staff |
774 |
325 |
$ 66.32 |
$ 21,554 |
Qualitative semi-structured interviews with AIM Team Leads |
30 |
30 |
$ 49.83 |
$ 1,494.90 |
Self-administered implementation surveys with clinical staff at 6 months |
774 |
387 |
$ 66.32 |
$ 25,665.84 |
Self-administered implementation surveys with clinical staff at 18 months |
774 |
387 |
$ 66.32 |
$ 25,665.84 |
Self-administered implementation surveys with clinical staff at 30 months |
774 |
387 |
$ 66.32 |
$ 25,665.84 |
DUA for AIM data |
258 |
21 |
$ 49.83 |
$ 1,046.43 |
Total |
35,919 |
54716 |
|
$ 1,489,998.34 |
* National Compensation Survey: Occupational wages in the United States May 2017 “U.S. Department of Labor, Bureau of Labor Statistics.”
a Hourly wage for nurse-midwives ($48.36; occupation code 29-1161).
b Weighted mean hourly wage for obstetrician-gynecologists ($113.10; occupation code 29-1064; 30%); nurse-midwives ($49.83; occupation code 29-1161; 30%); registered nurses ($35.36; occupation code 29-1161; 20%); and nurse practitioners ($51.86; occupation code 29-1171; 20%).
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
The total contractor cost to the government to implement and evaluation the Demonstration Project is estimated to be $1,890,056.98. As shown in Exhibit 3a, this amount includes costs for project development ($461,586.66), collecting the data ($58,790.00); analyzing the data ($520,376.66); and reporting the findings ($5,000).
Exhibit 3a. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Project Development |
$461,586.66 |
$115,396.67 |
Data Collection Activities |
$58,790.00 |
$14,697.50 |
Data Processing and Analysis |
$520,376.66 |
$130,094.17 |
Publication of Results |
$5,000 |
$1,250.00 |
Project Management |
$520,376.66 |
$130,094.17 |
Overhead |
$328,927 |
$82,231.75 |
Total |
$1,890,056.98 |
$473,764.25 |
Three Health Scientist Administrators (a Project Lead (GS-15) at 5% FTE, a Contracting Officer (GS-14) Representative at 20% FTE, and Project Co-Lead at 20% FTE (GS-14)) will be responsible for project management and oversight. This will include oversight of the data collection activities during the implementation and evaluation phase and review of the report of summarized results. The estimated cost to the Federal Government for these activities is provided in Exhibit 3b. The average hourly salary for the position of the Health Scientist Administrator at the GS-15 grade level, Step 3 is $70.45 per hour and the average hourly salary for the position of the Health Scientist Administrator at the GS-14 grade level, Step 3 is $59.90. The Federal hourly salary information is available on the OPM website at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/19Tables/html/DCB_h.aspx.
Exhibit 3b. Federal Government Personnel Cost
Federal Personnel |
Staff Count |
Hourly Rate |
Estimated Hours/year |
Cost/year |
Grade 14 |
2 |
$59.90 |
832 |
$49.836.80 |
Grade 15 |
1 |
$70.45 |
104 |
$7,326.80 |
Total |
$57,163.60 |
|||
The estimated total annualized cost for this activity is $530,927.85. This cost includes contractor costs ($473,764.25) and Federal personnel costs ($57,163.60).
The increase in hours is due to the 5 additional qualitative interviews that will be conducted with AIM Team Leads.
Baseline surveys with AIM Team Leads and frontline clinical staff occurred January-March, 2020. Training workshops for AIM Team Leads took place in February 2020 in both states. Qualitative interviews with AIM Team Leads will be conducted 3-4 months after the SPPC-II implementation start date, which due to delays from the covid-19 pandemic is now planned for January-February 2021, and 15-16 months after the SPPC-II implementation start date. Implementation surveys will be conducted 6, 12, and 18 months after SPPC-II implementation start date, thus mid-March 2021, mid-September 2021, and mid-March 2022, respectively. Coaching calls will be conducted for the 18 months of program implementation, between September 2020 and March 2022. AIM Program data will be obtained quarterly.
Qualitative data from AIM Team Leads will be coded using NVivo10 (QSR) and thematically analyzed to study organizational elements of successful implementation. This analysis will be conducted within three months of interviews’ completion. Survey data collected will be used in a variety of descriptive analyses, stratified by key characteristics of the hospitals and by state. For these and all other analyses, we will employ hospital weights derived using key hospital characteristics such as level of maternity care offered, teaching status, number of annul deliveries. Descriptive analyses will be conducted separately for each data collection activity within one month of the activity’s completion. To evaluate program implementation and trainings, descriptive statistics will summarize hospitals’ and trainees’ characteristics that could potentially be associated with key process outcomes. Associations between participants’ characteristics and training outcomes (in line with Kirkpatrick’s training evaluation framework17 for learning, knowledge transfer, and results) will be assessed at 6, 12 and 18 months after program implementation. Absolute and relative changes in training outcomes, including results, will be estimated between the various time points of data collection.
Between January and September 2022, JHU will conduct longitudinal data analyses. Of note, due to the relatively short period of program implementation, assessment of program’s impact on maternal health outcomes will only be conducted at the end of the project, after completion of all data collection activities. JHU will use interrupted time-series and segmented regression analysis methods with autoregressive error models to account for correlations of the data across time points and adjust for hospital characteristics. Regression analyses with linear and logistic models fitted for binary and continuous, respectively, process and outcome measures will identify key hospital and clinical staff characteristics associated with the program impact.
Information about the implementation of the SPPC-II Demonstration Project and data collected for its evaluation will be disseminated through reports on the AHRQ website; conference and PQC meeting presentations; and in the peer-reviewed, obstetric and patient safety literature. Journal articles to be developed using the Demonstration Project data will be similar to the recently published analysis of SPPC-I data.5
AHRQ does not seek this exemption.
List of Attachments:
Attachment A Roster of AIM Team Leads trained in each hospital
Attachment B AIM Team Leads training workshop evaluation form
Attachment C List of tracking measures for online trainings of clinical staff
Attachment D List of coaching call participants and topics addressed
Attachment E AIM Team Lead self-administered baseline survey
Attachment F AIM Team Lead self-administered baseline consent form
Attachment G Clinical staff self-administered baseline survey
Attachment H Clinical staff self-administered baseline consent form
Attachment I Comprehensive list of clinical staff
Attachment J1 Interview guide for qualitative interviews with AIM Team Leads, 3-4mo
(revised)
Attachment J2 Interview guide for qualitative interviews with AIM Team Leads, 15-16mo (new)
Attachment K Consent form for qualitative interviews with AIM Team Leads
Attachment L Clinical staff self-administered implementation survey at 6 months
Attachment M Clinical staff self-administered implementation survey at 12 months
Attachment N Clinical staff self-administered implementation survey at 18 months
Attachment O Consent form for clinical staff self-administered implementation survey at 6 months
Attachment P Consent form for clinical staff self-administered implementation survey at 12 months Attachment Q Consent form for clinical staff self-administered implementation survey at 18 months
Attachment R Data use agreement for AIM program data
Attachment S List of measures reported by hospitals for the AIM program
Attachment T List of Stakeholder Panel Members
Attachment X Federal Register Notice
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |