SUPPORTING STATEMENT
Part A
Programmatic Information Collection for the AHRQ Initiative to Support Primary Care to Advance Cardiovascular Health in States with High Prevalence of Preventable CVD Events
Version: July 10, 2020
Agency for Healthcare Research and Quality (AHRQ)
Table of contents
1. Circumstances that make the collection of information necessary 3
2. Purpose and Use of Information 7
3. Use of Improved Information Technology 8
4. Efforts to Identify Duplication 8
5. Involvement of Small Entities 8
6. Consequences if Information Collected Less Frequently 8
8. Federal Register Notice and Outside Consultations 9
9. Payments/Gifts to Respondents 9
10. Assurance of Confidentiality 9
11. Questions of a Sensitive Nature 9
12. Estimates of Annualized Burden Hours and Costs 9
13. Estimates of Annualized Respondent Capital and Maintenance Costs 11
14. Estimates of Annualized Cost to the Government 11
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Despite improvements in recent years, coronary heart disease (CVD) is a significant national health burden and the leading cause of death, involved in nearly one of every three deaths. Modifiable risk factors for CVD, such as high blood pressure, high cholesterol, and smoking, remain poorly controlled. PCOR evidence shows that increasing the delivery of the ABCS of heart health – Aspirin in high-risk individuals, Blood pressure control, Cholesterol management, and Smoking cessation – can reduce risk and save lives.
In 2010, Congress established the Patient-Centered Outcomes Research (PCOR) Trust Fund and instructed AHRQ to support the dissemination and implementation of PCOR findings. In accordance with its mandated role, AHRQ issued a Request for Applications (RFA) entitled Supporting Primary Care to Advance Cardiovascular Health in States with High Prevalence of Preventable CVD Events.1 AHRQ anticipates investing up to $18 million to support a maximum of 4 awards. The expected earliest start date for the grants is December 30, 2020.
This initiative has the following goals:
To improve heart health and help reduce CVD disparities by engaging with primary care practices, and disseminating and implementing PCOR findings to improve care delivery.
To learn how to develop sustainable state-level primary care QI infrastructure to improve the uptake of PCOR evidence in primary care.
To disseminate lessons learned, which take into consideration the context in which each program operated, on how to replicate successes and avoid challenges.
Grantees are expected to develop multi-organizational, state-level entities called Cooperatives, which could include a broad range of entities, such as universities, primary care organizations, public health agencies, payers, health systems, and advocacy groups. The Cooperatives will develop a network of primary care practices in the state, and from this network recruit at least 50 to engage in a QI evidence-based project to improve heart health. The program builds on AHRQ’s previous efforts, notably the EvidenceNOW (ENOW) and IMPaCT initiatives that identified states as key drivers of sustainable primary care practice improvement. 2,3
As specified in the RFA, each grantee will conduct an evaluation of its own program. AHRQ will use information supplied by the grantees as part of AHRQ’s standard monitoring and management of the grants (e.g., grantee progress reports), as well as information collected from key informant interviews (the subject of this information request). AHRQ’s information collection will explore each grantee’s primary care quality improvement, including their members and partners; and their experiences, achievements. This information will be used by AHRQ to develop case studies and resources that will serve as models for other States that are interested in improving their capacity to support the dissemination of PCOR and quality improvement in primary care.
To achieve the goals of this project the following data collections will be implemented:
Key informant interviews. AHRQ will conduct phone interviews with a variety of state-level organizations involved in primary care support and with primary practices. This information will be used to develop case studies for each Cooperative as well as program-level generalizations and lessons learned that might inform other efforts to improve care delivery.
Member check-in sessions: AHRQ will conduct group phone discussions with subset of participants in the key informant interviews to corroborate case studies and lessons learned, and to provide additional shared insights across participants.
Individual and Group Interviews
As noted above AHRQ is proposing to conduct interviews with key stakeholders participating in its new initiative Support Primary Care to Advance Cardiovascular Health in States with High Prevalence of Preventable CVD. This section describes the individual and group interviews that AHRQ is proposing to conduct (detailed interview protocols are provided in the attachments A-D).
Key Informant Interviews
Individual key informant interviews will be conducted with the following groups:
Grantee and Cooperative leadership, and Cooperative partners – about decision to participate in the project, prior collaborations, organization and governance of the Cooperative, nature and extent of partnerships, what worked well and barriers, changes to the Cooperative and their impact on provision of QI support, QI support strategies and their perceived effectiveness, successful strategies for recruiting practices and types of practices recruited, success in establishing state-level capacity to provide QI support, factors associated with successful implementation of QI, longer-term impact of the grant and sustainability of capacity developed, suggestions for improvement and lessons learned from the project. (See Attachments A and B)
Unaffiliated organizations involved in or knowledgeable about primary care in the states – nature and extent of connection to the Cooperatives, awareness of the project, views about the organization and effectiveness of the Cooperatives and their networks, other local activities that may have affected the work of the Cooperatives, views on changes in practice capacity to deliver better care and on sustainability of improvements, benefits to and any potential adverse consequences for patients, suggestions for improvement and lessons learned from the project. (See Attachment C)
Practices within the network not participating in the Heart Health QI project – prior collaboration and experience of recruitment to the network and Heart Health QI project, decision to participate, nature of engagement with the Cooperative and network, benefits and drawbacks of network participation, interest in participating in Heart Health QI project, strategies employed to improve heart health, knowledge of and views on QI strategies at participating practices, concurrent efforts to improve care delivery, plans to continue participating in the network, suggestions for improvement and lessons learned (See Attachment D)
Practices within the network participating in the Heart Health QI project – prior collaboration and experience of recruitment to the network and Heart Health QI project, decision to participate, nature of engagement with the Cooperative and network, benefits and drawbacks of network participation, weaknesses in care delivery that QI strategies are designed to address and how the practices handle these, expectations for improvements stemming from QI projects and any potential challenges, nature of and satisfaction with support for Heart Health QI project, contribution of QI support to practice capacity to improve heart health outcomes, concurrent efforts to improve care delivery, plans to continue implementing the intervention, other benefits of participation in the Heart Health QI project, plans to remain in the project, suggestions for improvement and lessons learned. (See Attachment D)
A total of 200 interviews is anticipated over the course of three years (Table 1).
All interviews will be conducted by telephone and are expected to take 45-60 minutes. Grantee and Cooperative leadership and Cooperative partner groups will be interviewed annually for three years, while the grants are active. Unaffiliated organizations and network practices will be interviewed in years 2 and 3 of the grants and practices participating in the Heart Health QI project in year 3 only. This schedule of interviews reflects the anticipated evolution of the state-level entity, development of new partnerships, recruitment of practices to the network, and implementation of Heart Health QI project.
All interviews will include at least one lead interviewer and a note-taker and will be recorded with respondents’ permission as a back-up. Detailed notes will be prepared after each interview.
Member Checking Sessions
In addition to key informant interviews, AHRQ will hold three group sessions to conduct member checking.4,5,6 The first session will take place at the conclusion of the grant year 1 and will include grantee leadership, Cooperative leadership, and key network partners (Table 2). The goal of this session (#1) is to corroborate AHRQ’s characterization of their organizational models and typology of QI support strategies they plan to deliver. The protocol is included in Attachment E. The remaining two member checking sessions, with (#2) Grantee/Cooperative leadership/network partners as well as organizations unaffiliated but familiar with the Cooperative and its work, and with (#3) representatives of network practices, both those participating and not participating in heart health QI project, will be convened at the end of the grant funding period (year 3). The goal of these sessions is to engage participants in validating the findings and/or providing additional insights on the common themes identified. The protocol is included in Attachment F.
All three sessions, expected to include groups of 9-12 participants each (the maximum total of 36), will be conducted by WebEx or another tool for remote communication, and will take 60-90 minutes.
The timing of interview and member checking session and the number of respondents by group are shown in Tables 1 and 2 (based on the assumption that AHRQ will award four grants).
Table 1: Number of and Timing of Individual Interviews
Respondents |
Sample Sizes and Timing |
||
Gant Year 1 (2021) |
Grant Year 2 (2022) |
Grant Year 3 (2023) |
|
Grantee leadership |
12 |
12 |
12 |
Cooperative leadership |
12 |
12 |
12 |
Cooperative partners |
12 |
24 |
24 |
Unaffiliated organizations |
0 |
12 |
12 |
Practices in network not participating in Heart Health QI project |
0 |
8 |
8 |
Practices in network participating in Heart Health QI project |
0 |
8 |
20 |
TOTAL |
36 |
76 |
88 |
Note: it is assumed that the networks will not be fully developed until year 2 and the practices will not be fully recruited until year 3.
Table 2. Number of and Timing for Member Checking Sessions
Respondents |
Gant Year 1 (2021) |
Grant Year 3 (2023) |
Session 1: Cooperative Models and Typology of QI Support Strategies Respondents: Grantee and Cooperative leadership, key Cooperative organizations and partners |
12 |
|
Session: Case Studies and Lessons Learned Respondents: Grantee and Cooperative leadership, key Cooperative organizations and partners, and unaffiliated organizations |
|
12 |
Session 3: Case Studies and Lessons Learned
Respondents: Network practices (those participating and not participating in heart health QI project) |
|
12 |
TOTAL |
12 |
24 |
Secondary Data Sources
The following secondary data will also be available to augment this information collection effort.
Grant applications
These documents will allow capturing the range of models and strategies envisioned by the grantees and interpreting their accomplishments in the context of this vision. In addition, the applications are expected to include a description of grantee approaches for internal evaluations and enable the AHRQ team to coordinate their activities with those of the grantees. Finally, grant applications will help refine the probes for the year 1 interviews and the member checking session.
Technical assistance (TA) documents
Throughout the initiative AHRQ will be collaborating with and providing technical assistance to grantees. The details of the support, documented in a TA log and in meeting notes, will offer insights into grantee needs and challenges, and how these evolved over time.
Meeting Notes
AHRQ will have regular calls and in-person meetings with grantees. Like TA support, notes from these interactions will provide information on grantee progress, challenges, and needs.
Quarterly Grantee Progress Reports
As part of its grantee monitoring role, AHRQ will collect information from quarterly progress reports to:
Learn about Cooperative members and their partners, such as their roles and responsibilities, governance and communication strategies, and the network of practices they have formed, and, in the third year, about their Heart Health QI projects
Provide grantees with technical assistance with the implementation of their respective projects
Identify lessons learned and challenges faced by the grantees
Tailor the interview prompts to grantees’ actual experiences. (This is explained further in the interview protocols provided in Attachments A-D)
Learn about grantees’ evaluations and their findings
Grantee progress reports will also contain other informative documents, such as Cooperatives’ memoranda of understanding, governance policies, training materials, Technical Assistance (TA) resources, and publications.
All this information will supplement insights from key interviews and will be used to develop comparative case studies that explore which grantee-level characteristics are associated with success in launching the Cooperatives and networks; to document program-level accomplishments, challenges, and lessons learned by aggregating the information across the grantees; and develop resources that could be used by states to assist with their primary care quality improvement efforts.
This project is being conducted by AHRQ through its contractor, Abt Associates, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
The purpose of the proposed information collection effort is to explore each grantee’s primary care quality improvement, including their members and partners; and their experiences and achievements. Additionally, this information collection will serve to help synthesize insights from across grantees, identify key themes, and distill lessons learned, taking consideration of the context in which each program operated.
The following knowledge will be generated to understand the contribution of the program to developing sustainable state-level capacity to implement PCOR findings in primary care and the pros and cons of various Cooperative models, as well as lessons learned about approaches to assisting practices in implementing evidence to improve care
All information collection for the project will use information technology. Interviews and member checking sessions will be conducted via video-conferencing to facilitate participation and reduce costs.
AHRQ made every effort to avoid duplication and to collect only such information needed to develop case studies and lessons learned. Because this is a new program, there are no existing datasets or other sources which could be reused.
In addition to the information collection, the study will make full use of program data from grantees. While important, these sources will not contain all of the data required to address the aims herein. Anticipated gaps in extant data include reflections on the participants experience, pros and cons of various strategies to establish and use the Cooperatives and the networks, satisfaction of participating practices, efforts to ensure sustainability, and other topics. These gaps in will be filled using new information collection described in this request.
This project does not intend to intentionally involve nor exclude or impact any small entities. However, if an identified and recruited participating organization is a small entity, they will be involved in the study in the same way as other organizations, and we expect the impact to be no different. The instruments and procedures used to collect information are designed to minimize the burden on all respondents.
This project is a one-time information collection effort. The information collection period will last for approximately 3 years. Many of the respondents will be interviewed up to three times (on an annual basis). This repetition is necessary to capture the development and function of the Cooperatives and their networks of practices. Not collecting the information, or shortening the information collection period will cause a risk of not collecting adequate information to understand the pros and cons of the models developed by the grantees and consequently will limit the utility of the study to AHRQ and other users.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on August 5th, 2020 for 60 days on Page 47374, Volume 85, and Number 152. (see Attachment G). AHRQ received no substantive comments.
There are no outside consultants being used
No payments or gifts will be offered to respondents.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.
The information collection protocols do not contain any questions concerning political affiliations and attitudes; respondents’ mental or psychological problems; illegal, antisocial, self-incriminating or demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships; legally privileged relationships; or records describing how an individual exercises First Amendment rights. Nor do they contain questions related to sexual behavior and attitudes, religious beliefs, income or proprietary business information. However, interviews may elicit information that reflects negatively on staff or organization performance. Respondents will be explicitly informed that their participation is voluntary, information they provide is confidential to the extent provided by law, and they may choose to withdraw from the study or not respond to specific items without penalty.
Table 4 presents estimates of the reporting burden hours for the information collection efforts. Time estimates are based on prior experiences and what can reasonably be requested of participating entities.
Key informant interviews. In-depth interviews will be conducted with the total of up to 88 individuals. Respondents from Grantee and Cooperative leadership and Cooperative partner groups will be interviewed every year for three years. Respondents from unaffiliated organizations and non-participating practices will be interviewed twice, in years 2 and 3, and respondents from participating practices once in year 3. The interviews are expected to last for up to 1 hour. The protocols are included in Attachments A – D.
Member checking sessions. Three member checking sessions will be conducted with the total of up to 36 participants. Grantee and Cooperative leadership will participate in two sessions, in year 1 and year 3, and Cooperative partners only in year 3. The sessions are expected to last for up to 1.5 hours. The protocols are included in Attachments E and F.
Table 3. Estimated annualized burden hours
Data Collection Method or Project Activity |
A. Number of respondents |
B. Number of responses per respondent |
C. Hours per response |
D. Total burden hours A*B*C |
Key Informant Interviews |
|
|
|
|
Grantee leadership (Attachment A) |
12 |
3 |
1 |
36 |
Cooperative leadership (Attachment A) |
12 |
3 |
1 |
36 |
Cooperative partners (Attachment B) |
24 |
2.5* |
1 |
60 |
Unaffiliated organizations (Attachment C) |
12 |
2 |
1 |
24 |
Practices in network not participating in Heart Health QI project (Attachment D) |
8 |
2 |
1 |
16 |
Practices in network participating in Heart Health QI project (Attachment D) |
20 |
1.4** |
1 |
28 |
Member Checking Sessions |
|
|
|
|
Grantee leadership (Attachments E and F) |
4 |
2 |
1.5 |
12 |
Cooperative leadership (Attachments E and F) |
4 |
2 |
1.5 |
12 |
Cooperative partners (Attachments E and F) |
2 |
2 |
1.5 |
6 |
Unaffiliated organizations (Attachment F) |
2 |
2 |
1.5 |
6 |
Network practices (Attachment F) |
12 |
1 |
1.5 |
18 |
TOTAL |
112 |
|
|
254 |
*Note: This number reflects that in Year 1 we will only interview 12 respondents, but 24 in years 2 and 3, hence 2.5 # of responses.
**This number reflects that in Year 2 we will interview 8 respondents and in year 3 we will interview 20 respondents.
Table 4 presents the estimated annualized cost burden associated with the respondents’ time to participate in this research. The total cost burden is estimated to be $29, 260.96.
Table 4. Estimated annualized cost burden
Data Collection Method or Project Activity |
A. Number of respondents |
B. Total burden hours |
C. Average hourly rate |
D. Total cost burden B*C |
Key Informant Interviews |
|
|
|
|
Grantee leadership |
12 |
36 |
$110.74 |
$3,986.64 |
Cooperative leadership |
12 |
36 |
$110.74 |
$3,986.64 |
Cooperative partners |
24 |
60 |
$110.74 |
$6,644.40 |
Unaffiliated organizations |
12 |
24 |
$110.74 |
$2,657.76 |
Practices in network not participating in Heart Health QI project |
8 |
16 |
$136.49 |
$2,183.84 |
Practices in network participating in Heart Health QI project |
20 |
28 |
$136.49 |
$3,821.72 |
Member Checking Sessions |
|
|
|
|
Grantee leadership |
4 |
12 |
$110.74 |
$1,328.88 |
Cooperative leadership |
4 |
12 |
$110.74 |
$1,328.88 |
Cooperative partners |
4 |
6 |
$110.74 |
$664.44 |
Unaffiliated organizations |
2 |
6 |
$110.74 |
$664.44 |
Network practices |
12 |
18 |
$110.74 |
$1,993.32 |
TOTAL |
112 |
254 |
|
$29,260.96 |
Note: the rates were based on the mean hourly wages from the Bureau of Labor & Statistics for the closest categories of respondents and doubled to account for overhead and fringe.
The mean hourly wage rates were obtained from the Bureau of Labor & Statistics7 and doubled to account for overhead and fringe benefits. The occupational codes used were as follows:
For grantee and cooperative leadership, partners, and unaffiliated organizations – medical and health service managers (11-9111, $53.37)
For practices – an average of physicians (29-1228, $97.81), medical and health services managers (11-9111, $53.37), and nurse practitioners (29-1171, $53.77)
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this information collection. There are no direct costs to respondents other than their time to participate in the study.
Table 6. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Instrument development |
$30,000 |
$30,000 |
Information collection |
$300,000 |
$100,000 |
Information processing and analysis |
$180,000 |
$60,000 |
Reporting of results |
$60,000 |
$20,000 |
Project management |
$30,000 |
$10,000 |
Total |
$600,000 |
$200,000 |
Table 7. Government Personnel Cost
Tasks/Personnel |
Annual Salary |
% of Time |
Cost |
PRE OMB Approval Costs |
|||
Social Science Analyst – GS15, Step 9 |
$164,200 |
1% |
$1,642 |
POST OMB Approval Costs |
|||
Social Science Analyst – GS15, Step 9 |
$164,200 |
2% |
$3,284 |
Total |
|
|
$4,926 |
This is a new information collection.
Description |
Timing |
Interviews with grantees’ leadership |
Jan – Mar 2021 Jan – Mar 2022 Jan – Mar 2023 |
Member checking session 1 with grantee and cooperative leadership and cooperative partners |
Jul – Sept 2021 |
Interviews with cooperative leadership |
Apr – Jun 2021 Jul – Sept 2022 Jul – Sept 2023 |
Interviews with cooperative partners |
Apr – Jun 2021 Jul – Sept 2022 Jul – Sept 2023 |
Interviews with unaffiliated organizations |
Jul – Sept 2022 Jul – Sept 2023 |
Interviews with not participating practices |
Jul – Sept 2022 Jul – Sept 2023 |
Interviews with participating practices |
Jul – Sept 2023 |
Member checking session 2 with grantee and cooperative leadership |
Oct – Dec 2023 |
Member checking session 3 with cooperative partners |
Oct – Dec 2023 |
Publication Plan
AHRQ intends to use the information
collected from this information collection to develop case studies
and other resources related to primary care improvement. AHRQ will
also publish findings about the information collected from the
interviews in peer-reviewed publications.
Analysis Plan
Qualitative
information from interviews will be coded and analyzed using NVivo
qualitative analytic software. The coding will be iterative and
include deductive codes established a priori from the research
questions and inductive codes which emerge from the information. Some
qualitative codes, such as the type of organization recruited to
participate in the Cooperative, may be transformed into quantitative
codes that can be used as covariates in the quantitative analyses.
Coding quality assurance procedures will be used (e.g., debriefs on
coding). The resulting information will be triangulated with
secondary sources and examined to create a case study of each
grantee, which will support both comparisons and program-wide
understanding of effects. We will create a typology of the QI support
interventions and implementation strategies, similar to what was done
for EvidenceNOW. We will also identify lessons learned and models
used to structure the organization of, and define the function of,
the Cooperatives and networks.
AHRQ does not seek this exemption.
List of Attachments:
Attachment A: Grantee and Cooperative
Leadership - Interview Protocol
Attachment B: Cooperative Partners - Interview Protocol
Attachment C: Unaffiliated
Organizations - Interview Protocol
Attachment D: Practices
Participating and Non-participating in Implementing a QI Strategy –
Interview Protocol
Attachment E: Member Checking
Session to Validate QI Infrastructure Models and Typology of QI
Support Interventions
Attachment F: Member Checking
Sessions 2 and 3 to Validate Qualitative and Mixed-Methods Findings
Attachment G: 60-Day Federal
Register Notice
2 The EvidenceNOW Model: Providing External Support for Primary Care. Content last reviewed August 2019. Agency for Healthcare Research and Quality, Rockville, MD.
3 Kaufman, A. et al. The Role of Health Extension in Practice Transformation and Community Health Improvement: Lessons From 5 Case Studies. Ann Fam Med August 2019 vol. 17 no. Suppl 1 S67-S72
4 Creswell, J. W., & Miller, D. L. (2000). Getting good qualitative data to improve educational practice, Theory Into Practice, 39(3), 124-130.
5 Candela AG. Exploring the function of member checking. The Qualitative Report. 2019 Mar 1;24(3):619-28.
6Cohen DJ, Balasubramanian BA, Gordon L, Marino M, Ono S, Solberg LI, Crabtree BF, Stange KC, Davis M, Miller WL, Damschroder LJ. A national evaluation of a dissemination and implementation initiative to enhance primary care practice capacity and improve cardiovascular disease care: the ESCALATES study protocol. Implementation Science. 2015 Dec;11(1):86.
7 May 2019 National Occupational Employment and Wage Estimates
United States. URL: http://www.bls.gov/oes/current/oes_nat.htm#b29-0000.htm
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |