Supporting Statement Part B -- 0935-0196 Medical Office SOPS_Extension

SUPPORTING STATEMENT

Part B

Collection of Information for

Agency for Healthcare Research and Quality’s (AHRQ)

Surveys on Patient Safety Culture® (SOPS®) Medical Office Survey Database

October 6, 2024

Extension without change of the SOPS Medical Office Survey Database,

OMB NO. 0935-0196

Table of Contents

B. Collection of Information Employing Statistical Methods 2

1. Respondent Universe and Sampling Methods 2

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 4

4. Tests of Procedures 5

5. Statistical Consultants 5

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

AHRQ developed and pilot tested the SOPS Medical Office Survey in 2007 with OMB approval (OMB #0935-0132; approved on July 5, 2007). The AHRQ SOPS Medical Office Database serves as a central U.S. repository for data from the survey. The original SOPS Medical Office Database, OMB #0935-0196, was first approved on June 12, 2012.

According to the U.S. Census Bureau 2021 Economic Census (2021 NAICS code 6211 “Offices of physicians”), there were 213,098 physicians’ offices in the U.S.¹ However, less than 1% of these medical offices (1,164 medical offices) participated in the 2024 Database as noted in the Database Report on the AHRQ website². The database is comprised of data that are voluntarily submitted by medical offices that have administered the survey, and is not a statistically selected sample, nor is it a representative sample of all U.S. medical offices. Estimates based on this self-selected sample may produce biased estimates of the population. Because of this, it is not possible to compute estimates of precision that apply to the population. While the survey results in the Database Report represent the largest known compilation of publicly available patient safety culture data for medical offices, the report contains a Data Limitations section at the end that acknowledges the limitations noted above.

Medical offices that submit data to the Database receive a free, customized feedback report that displays their medical office’s results against the aggregated and de-identified results of other medical offices submitting data to the Database. Medical offices that do not submit data to the database can still view the SOPS Medical Office Database report for aggregate results of those medical offices who voluntarily submitted their data to the Database. As part of a toolkit of support materials for the SOPS Medical Office Survey, medical offices can also use a Microsoft^® Excel-based Data Entry and Analysis Tool that is an Excel file with macros that will automatically produce graphs and charts of a medical office’s results once data are entered into a data sheet. Many medical offices use this tool to produce their results.

In the overall database report, medical offices are provided with a detailed description and explanation of the results that are presented and given examples and guidance on how to calculate their own scores for comparison.

2. Information Collection Procedures

Information collection for the AHRQ Medical Office Survey on Patient Safety Culture Database occurs in a periodic data collection cycle approximately every other year in September. The next submission period is September 2025. Information collection procedures for submitting and processing data are shown in Figure 1.

Figure 1. SOPS Medical Office Database Data Submission

Step 1: Call for Data Submission. Announcements about the opening of voluntary data submission for sites that have administered or plan to administer the SOPS Medical Office Survey go out through various publicity sources. AHRQ’s News Now and patient safety electronic newsletters target approximately 191,000 and 120,000 subscribers respectively. In addition, the AHRQ Surveys on Patient Safety Culture Medical Office Survey listserv targets approximately 81,000 subscribers. An example of email announcements calling for data submission is shown in Attachment D, Email # 1 and # 3. Through these efforts, U.S. medical offices are made aware of and invited to submit their survey data to the database.

As the administrator of the database and under contract with AHRQ, Westat provides free technical assistance to submitting medical offices through a dedicated email address (DatabasesOnSafetyCulture@westat.com) and toll-free phone number (1-888-324-9790).

Step 2: Registration of Potential Participants. A secure data submission website allows interested parties such as medical offices and medical groups to register and submit data. Registration takes about 3 minutes to complete and asks for contact and other basic information (see Attachment A). After registering, if registrants are deemed eligible to submit data, an automated email is sent to authenticate the account and update the user password (see Attachment D, Email # 2).

Once users are registered and have a password, they can enter the main page of the website. Information about eligibility requirements, data use agreements, and data file specifications regarding how to prepare their data for inclusion in the SOPS Database is posted on the main page and can be reviewed.

Step 3: Enter Medical Office Information and Upload Questionnaire. At this step, users provide information about each of their medical offices, such as point-of-contact, method of survey administration, overall response rate, and other characteristics that we use to analyze collected data (e.g., specialty, number of staff and providers, and ownership) (see Attachment C). They also upload their survey questionnaire that they administered to enable us to determine whether any changes were made to the survey (see Attachment G, Figure 1).

Step 4: Submit Data Use Agreement (DUA). To protect the privacy of all participating medical offices, a duly authorized representative from the medical office must sign a Data Use Agreement (DUA) (see Attachment B). The DUA language was reviewed and approved by AHRQ’s general counsel and asserts that the medical office’s data will be handled in a secure manner using necessary administrative, technical and physical safeguards to limit access to it and maintain its confidentiality. In addition, the DUA explains that the data are used for the purposes of the database, that only aggregated results are reported, and that the medical office will not be identified by name. Data are not included in the database without this signed DUA. Users can fax, mail, or upload a copy of the signed agreement.

Step 5: Upload Data File(s). At this step, users are asked to upload their de-identified individual-level survey data for each medical office (see Attachment G, Figure 2). Data submitted through the secure data submission website are encrypted to ensure secure transmission of the survey data. Data are accepted in Microsoft Excel^® format since this is the format preferred by medical offices. Users must upload one data file per medical office. If a user has multiple offices within a medical group, users can upload one data file that identifies all of the offices in their group. The data file specifications (see Attachment E) are provided to data submitters to ensure that users submit standardized and consistent data in the way variables are named, coded, and formatted.

Once a data file is uploaded, a separate load program developed in Visual Basic (VB) reads the submitted files and loads them into the SQL database that stores the data. A data quality report is then produced and made available to the participant. This report displays item frequencies and flags out-of-range values and incorrectly reverse-coded items. If there are no problems with the data, an acknowledgement of data upload and acceptance will be granted during the user session. If data are improperly coded, the user is informed by having a message post on the screen indicating that the data file failed during the user session. Users are expected to fix any errors and resubmit their data file(s) for processing. Once there are no problems, the user is informed of the acceptance of data during the user session with an online message of acceptance.

Step 6: Approve Data Submission. Once all the information required for submission has been submitted and approved, an email is sent to the medical office contact indicating that their data have received final acceptance.

3. Methods to Maximize Response Rates

AHRQ makes a number of toolkit materials available to assist medical offices with the SOPS surveys. The SOPS Medical Office Survey has a Survey User’s Guide that gives users guidance and tips about survey administration on the following topics: planning; selecting a sample; determining their data collection method; data collection procedures (including a section on web surveys); and analyzing data and producing reports³. The Survey User’s Guide also gives medical offices tips about how to increase response rates through publicity efforts, top management support, use of incentives, and following all steps of proper data collection protocols. Of the medical offices that voluntarily submitted their data for the 2024 SOPS Medical Office Database, the average response rate across 1,164 medical offices was 58%, with a minimum of 6% and a maximum of 100%.

The SOPS User Network promotes the database by reaching out to promote submission through:

1. AHRQ and AHRQ SOPS email listservs;

2. Organizational partners and stakeholders that have national reach to primary care and specialty practices;

3. Past Medical Office Database submitters;

4. Users that have contacted the SOPS technical assistance helpline about the medical office survey or its Excel Tool;

5. Other outlets such as webcasts and conferences.

As noted earlier in this document under Information Collection Procedures, Step 1 – Call for Data Submission, announcements about the opening of data submission go out through various publicity sources as a way to boost medical office participation in the database. AHRQ, through its contractor Westat, provides free technical assistance to users through a dedicated email box and toll-free phone number. In addition, reminders are sent to database registrants to remind them of the deadline for data submission.

4. Tests of Procedures

Input and Feedback for the Development of the SOPS Database Submission System.

Because the Surveys on Patient Safety Culture are public-use instruments, the SOPS program has generally modeled its data submission processes after those used by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Database that has been in operation for many years. SOPS staff consulted with CAHPS Database staff and programmers to determine best practices for data submission. This information, as well as feedback obtained during the provision of technical assistance each year the database has been running, has been used to improve the SOPS online data submission system and process over time.

5. Statistical Consultants

Joann Sorra, PhD

Westat

1600 Research Blvd.

Rockville, MD 20850

joannsorra@westat.com

Naomi Yount, PhD

Westat

1600 Research Blvd.

Rockville, MD 20850

naomiyount@westat.com

1 U.S. Census Bureau, 2021 Economic Census. https://data.census.gov/cedsci/. Last accessed 1/18/2024

2 SOPS Medical Office Database. Content last reviewed August 2024. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/sops/databases/medical-office/index.html. Last accessed 9/7/2024.

3 Medical Office Survey on Patient Safety Culture. Content last reviewed July 2024. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/sops/surveys/medical-office/index.html. Last accessed 9/7/2024.