Supporting Statement Part B -- MOSOPS Comparative Database 5-15-2012

SUPPORTING STATEMENT

Part B

Medical Office Survey on Patient Safety Culture Comparative Database

February 2, 2012

Agency of Healthcare Research and Quality (AHRQ)

Table of Contents

B. Collection of Information Employing Statistical Methods 2

1. Respondent universe and sampling methods 2

2. Information Collection Procedures 7

3. Methods to Maximize Response Rates 9

4. Tests of Procedures 9

5. Statistical Consultants 10

B. Collection of Information Employing Statistical Methods

1. Respondent universe and sampling methods

The AHRQ Medical Office Survey on Patient Safety Culture (Medical Office SOPS) Comparative Database serves as a central U.S. repository for data from the survey. As such, AHRQ houses the largest database of the survey’s results. However, the database comprises data that are voluntarily submitted by medical offices that have administered the survey and is not a statistically selected sample, nor is it a representative sample of all U.S. medical offices.

A Medical Office SOPS Comparative Database Report is produced annually, at least initially, based on data submitted to the database. Currently, preliminary comparative results from 470 medical offices are presented that include medical offices that participated in a pilot test of the survey in 2007 as well as Practice-Based Research Network (PBRN) medical offices that administered the survey in 2009. A copy of the 2010 Preliminary Comparative Results: Medical Office Survey on Patient Safety Culture report is available at http://www.ahrq.gov/qual/mosurvey10/moreslttab1.htm. Similar to the Hospital Survey on Patient Safety Culture Comparative Database Report (http://www.ahrq.gov/qual/hospsurvey10/), a section entitled “Data Limitations” is included in the 2010 Medical Office SOPS Preliminary Comparative Results report that outlines the limitations of the data and makes it clear that the data are not a statistically selected sample of U.S. medical offices.

Universe of medical offices and representativeness of the data. Quantifying the universe of medical offices is not straightforward. Medical offices are typically characterized as either those with 1 or 2 physicians or group medical practices consisting of 3 or more physicians. According to the U.S. Census Bureau 2007 Economic Census (2007 NAICS code 6211 “Offices of physicians”), there were 220,131 physicians’ offices in the U.S. (http://factfinder.census.gov/servlet/IBQTable?_bm=y&-geo_id=D&-ds_name=EC0762A1&-_lang=en). These offices consist of those with only 1 or 2 physicians as well as offices from medical groups with 3 or more physicians. Because the AHRQ Medical Office SOPS is a survey, it is recommended that it only be implemented in group medical offices with 3 or more providers and staff to protect the confidentiality of medical office staff respondents when survey results are summarized. Smaller offices with only 1 or 2 physicians are advised to use the survey as a discussion tool rather than administer it as a survey. Therefore, the population most likely to submit data to the Medical Office SOPS Comparative Database is group medical offices.

A 2005 Health Affairs article examining electronic health record adoption in group medical practices, whose lead author is from the Medical Group Management Association states: “The total number of U.S. group practices is not known, but we estimate it to be somewhat larger than the 34,490 practices we identified, perhaps in the range of 40,000–50,000” (David Gans, John Kralewski, Terry Hammons, and Bryan Dowd, “Medical Groups’ Adoption Of Electronic Health Records And Information Systems,” Health Affairs, 2005, Vol 24 (5), pp. 1323-1333).

The most relevant and thorough source of data on the population of medical group practices in the U.S. is the American Medical Association’s (AMA) 1999 edition of “Medical Group Practices in the U.S.: A Survey of Practice Characteristics.” This report is the only and most recent source that reports information about the characteristics of medical group practices in the U.S., with groups defined as those with 3 or more physicians. A total of 34,066 eligible medical groups were identified and a census of them was conducted by the AMA in 1996. The problem with comparing group practices to individual medical offices is that a single group practice can comprise several different medical office locations. The AMA report only includes data from the parent or primary location of group practices, and therefore is also an underestimate of the actual number of group medical offices.

A more recent report from the National Center for Health Statistics in 2008 (http://www.cdc.gov/nchs/data/series/sr_13/sr13_166.pdf) presents estimates of the number and characteristics of medical practices with which physicians are associated. These data, from the 2005-2006 National Ambulatory Medical Care Surveys (NAMCS), are physician-based rather than office-based, and do not allow direct comparisons with the Medical Office SOPS database medical offices. However, we present the NAMCS geographic region data in Table 1 below. Finally, the NAMCS report estimates that during 2005-2006 there were 163,700 medical practices in the United States, which is considerably lower than the 220,131 physicians’ offices in the U.S. Census Bureau 2007 Economic Census.

Table 1 shows the geographic distribution of the AHRQ Medical Office SOPS database medical offices compared to the distribution of offices of physicians based on the 2007 U.S. Economic Census (http://www.census.gov/econ/industry/geo/g6211.htm) and the NAMCS estimates of the number of office-based medical practices in 2005-2006. The AHRQ database overrepresents medical offices in the Midwest region and slightly underrepresents medical offices in the South and West.

Table 1. Distribution of Offices of Physicians in AHRQ Database Medical Offices (2010), U.S. Economic Census, Offices of Physicians (2007), and NAMCS Office-Based Medical Practices (2005-2006) by Region

Census Region	AHRQ MO SOPS Database Medical Offices (2010)		U.S. Economic Census, Offices of Physicians (2007)		NAMCS Office-Based Medical Practices (2005-2006)
	Number	Percent	Number	Percent	Number	Percent
South	128	27%	84,424	38%	60,700	37%
Northeast	100	21%	44,605	20%	36,300	22%
Midwest	180	38%	38,951	18%	30,100	18%
West	62	13%	52,151	24%	36,600	22%
TOTAL	470	100%	220,131	100%	163,700	100%

• Note: Column percent totals may not add to exactly 100% because of rounding.

Because there is not a recent and comprehensive source of data describing the population of group medical offices in the U.S., we do not present comparisons of the database medical offices to any other population statistics. Only descriptive statistics about the database medical offices are provided. Participants represent a spectrum of characteristics by number of providers, single vs. multi-specialty, specific specialty, implementation of electronic tools, ownership, region, and number of locations.

Statistics From the Medical Office SOPS Preliminary Comparative Results Report. The following tables provide data from the 470 medical offices included in the Preliminary Comparative Results Report in 2010.

Medical office characteristics were obtained from a designated point of contact in each medical office. Tables 2 to 4 show the distribution of medical offices by total number of providers and staff, type of specialty, and number of medical offices by specialty. The vast majority of medical offices (77%) had a total of 40 or fewer providers and staff.

Table 2. Distribution of Medical Offices by Number of Providers and Staff, 2010

Total Number of Providers and Staff	All Medical Offices
	Number	Percent
5–10	84	18%
11–20	140	30%
21–30	90	19%
31–40	47	10%
41–50	40	9%
51–60	22	5%
61–70	11	2%
More than 70	36	8%
Total	470	100%

*Note: Column percent totals may not add to exactly 100% because of rounding.

As shown in Table 3, over half of medical offices (56%) were single specialty.

Table 3. Distribution of Medical Offices by Type of Specialty, 2010

Type of Specialty	All Medical Offices
	Number	Percent
Single specialty	263	56%
Multispecialty with primary care only (e.g., family medicine, internal medicine, pediatrics, OB/GYN, general practice)	114	24%
Multispecialty with primary care and specialty care	81	17%
Multispecialty with specialty care only	11	2%
Total	469	100%
Missing	1

*Note: Column percent totals may not add to exactly 100% because of rounding.

The 470 medical offices represent a wide range of specialties, with most categorized as Family Practice/Family Medicine (291 offices) (Table 4).

Table 4. Number of Medical Offices by Specialty, 2010

Specialty	Number of Medical Offices	Specialty	Number of Medical Offices
Allergy/Immunology	15	Neurology	11
Anesthesiology	2	Nuclear Medicine	2
Cardiology	22	OB/GYN or GYN	63
Child & Adolescent Psychiatry	6	Ophthalmology	15
Dermatology	11	Orthopedics	23
Diagnostic Radiology	8	Otolaryngology	12
Emergency Medicine	7	Pathology – Anatomic/Clinical	1
Endocrinology/Metabolism	22	Pediatrics	95
Family Practice/Family Medicine	291	Physical Medicine & Rehabilitation	9
Forensic Pathology	1	Psychiatry	26
Gastroenterology	11	Public Health & Rehabilitation	2
General Practice	16	Pulmonary Medicine	9
General Preventive Medicine	7	Radiology	6
General Surgery	21	Rheumatology	11
Geriatrics	14	Surgery (All)	11
Hematology/Oncology	12	Urology	12
Internal Medicine	102	Vascular Medicine	3
Medical Genetics	2	Other specialty	47
Nephrology	9

*Note: The total number of medical offices will not necessarily sum to 470 as some medical offices may categorize themselves as more than one type of specialty.

Table 5 shows the distribution of medical offices by number of locations. Approximately two-thirds of medical offices (67%) had a single location, while the rest (33%) had multiple locations.

Medical offices with more than one location had an average of 9 locations (ranging from 2 to 40 locations). Of the 152 medical offices with multiple locations, 48 indicated they were the primary/parent location and 103 indicated they were a satellite location.

Table 5. Distribution of Medical Offices by Number of Locations, 2010

Number of Locations	All Medical Offices
	Number	Percent
One Location	310	67%
Multiple Locations	152	33%
Total	462	100%
Missing	8

Table 6 shows that almost half of medical offices (47%) were owned by a hospital or health system.

Table 6. Distribution of Medical Offices by Majority Ownership, 2010

Majority Ownership	All Medical Offices
	Number	Percent
Provider(s) and/or Physician(s)	134	29%
Managed Care or Health Maintenance Organization	3	1%
University or Medical School or Academic Medical Institution	75	16%
Hospital or health system	220	47%
Federal, state or local government, community board, etc	23	5%
Other	10	2%
Total	465	100%
Missing	5

Comparative results and explanation of how results are calculated. The 2010 Medical Office SOPS Preliminary Comparative Results report presents average percent positive scores for each of the 12 patient safety culture composites and for the survey’s 52 items plus two additional questions on respondents’ overall rating of their medical office’s health care quality and patient safety. The average percent positive scores are calculated by averaging composite-level percent positive scores across all medical offices in the database, as well as averaging item-level percent positive scores across medical offices. Since the percent positive is displayed as an overall average, scores from each medical office are weighted equally in their contribution to the calculation of the average. An alternative method would be to report a straight percent of positive response across all respondents, but this method would give greater weight to respondents from larger medical offices.

Percentages are presented rather than mean scores because medical office administrators have indicated that percentages are more easily understood and interpreted (for example, indicating that 75% of staff responded positively to an item rather than reporting that the mean score on the item was 4.00 out of 5). In addition, the minimum and maximum percent positive scores are presented along with percentile scores for the 10^th, 25^th, 50^th, 75^th, and 90^th percentiles to present information about the distribution of scores across database medical offices.

2. Information Collection Procedures

Information collection for the AHRQ Medical Office Survey on Patient Safety Culture Comparative Database occurs in a periodic data collection cycle each year from September to October. Information collection procedures for submitting and processing data are shown in Figure 1.

Figure 1. Medical Office SOPS Comparative Database Data Submission

Step 1: Call for Data Submission. Announcements about the opening of data submission go out through various publicity sources. AHRQ’s patient safety and electronic newsletters target approximately 50,000 subscribers. In addition, email announcements are sent to approximately 3,000 survey users who have at some point requested technical assistance or who have used the Medical Office survey before. An example of an email announcement calling for data submission is shown in Attachment I, Email # 1. Reminder emails are sent one and two weeks after the initial email announcing the call for data submission (see Attachment I, email #2). In addition, the AHRQ Web site has public information about the yearly timeline and instructions for data submission (http://www.ahrq.gov/qual/mosurvey11/mosubinfo.htm). Through these efforts, U.S. medical offices are made aware of and invited to submit their survey data to the database.

As the administrator of the database and under contract with AHRQ, Westat provides free technical assistance to submitting medical offices through a dedicated email address (DatabasesOnSafetyCulture@ahrq.hhs.gov) which routes to Westat and toll-free phone number (1-888-852-8277).

Step 2: Registration for Potential Participants. A secure data submission Web site allows interested parties such as medical offices and medical groups to register and submit data. Login for registration is one page that takes about 3 minutes to complete and asks for contact and other basic information (see Attachment J). If registrants are deemed eligible to submit data, two separate, automated emails are sent to provide them with a username and password and information needed in the next steps of the data submission process (see Attachment I, Emails # 3 and #4).

Once users have a username and password, they can enter the main page menu of the Web site. Information about eligibility requirements, data use agreements, and data file specifications regarding how to prepare their data for inclusion in the SOPS database is posted and can be reviewed.

Step 3: Enter Medical Office Information and Upload Questionnaire. Here, users provide information about each of their medical offices, such as point of contact, method of survey administration, overall response rate, and other medical office characteristics (specialty, number of staff and providers, and ownership) (see Attachment D). They also upload their survey questionnaire that they administered to enable us to determine whether any changes were made to the survey (see Attachment J).

Step 4: Submit Data Use Agreement (DUA). To protect the confidentiality of all participating medical offices, a duly authorized representative from the medical office must sign a Data Use Agreement (DUA) (see Attachment C). The DUA language was reviewed and approved by AHRQ’s general counsel. The DUA states that the medical office’s data are handled in a secure manner using necessary administrative, technical and physical safeguards to limit access to it and maintain its confidentiality. In addition, the DUA states the data are used for the purposes of the database, that only aggregated results are reported, and that the medical office is not identified by name. Data are not included in the database without this signed data use agreement. Users fax, email and/or mail a signed copy of the DUA.

Step 5: Upload Data File(s). At this step, users are asked to upload their individual-level survey data for each medical office. Data are submitted through a secure Web site and are encrypted to ensure secure, confidential transmission(see Attachment K). Data are accepted in Microsoft Excel® format since this is the format preferred by medical offices. If a user has multiple offices within a medical group, users can upload one data file that identifies all of the offices in their group. The data file specifications (see Attachment K) are provided to data submitters to ensure consistency in the way variables are named, coded, and formatted.

Once a data file is uploaded, a program developed in Visual Basic (VB) reads the submitted files and loads them into the SQL database. A data quality report is then produced and made available to the participant. This report displays item frequencies and flags out-of-range values and incorrectly reverse-coded items. If there are no problems with the data, an acknowledgement of data upload is granted via an automatic email. If data are improperly coded, an automatic email informs the participant of the problem. Users are expected to fix any errors and resubmit. Once there are no problems, an email is sent to the medical office contact via the database submission Web site indicating their data have received final acceptance.

Step 6: Approve Data Submission. Once all of the information required for submission is submitted and approved, an email is sent to the medical office contact indicating that their data have received final acceptance.

3. Methods to Maximize Response Rates

AHRQ makes a number of toolkit materials available to assist medical offices with the SOPS surveys. The Medical Office SOPS has a Survey User’s Guide that gives users guidance and tips about survey administration on the following topics: planning; selecting a sample; determining their data collection method; data collection procedures (including a section on Web surveys); and analyzing data and producing reports (at http://www.ahrq.gov/qual/patientsafetyculture/mosurvindex.htm). The Survey User’s Guide also gives tips about how to increase response rates through publicity efforts, top management support, use of incentives, and following all steps of proper data collection protocols.

As noted earlier in this document under Information Collection Procedures, announcements about the opening of data submission go out through various publicity sources as a way to boost medical office participation. AHRQ’s patient safety and electronic newsletters target approximately 50,000 subscribers. In addition, email announcements are sent to approximately 3,000 survey users who have at some point requested technical assistance or who have used the survey before. AHRQ, through its contractor Westat, provides free technical assistance to users through a dedicated email box and toll-free phone number. In addition, reminders are sent to database registrants to remind them of the deadline for data submission.

4. Tests of Procedures

Input and Feedback for the Development of the SOPS Database Submission System. Because the Surveys on Patient Safety Culture are public-use instruments, the SOPS program has generally modeled its data submission processes after those utilized by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) database that has been in operation for many years. SOPS staff consulted with CAHPS database staff and programmers to determine best practices for data submission. This information, as well as feedback obtained during the provision of technical assistance each year, has been used to develop the SOPS online data submission system and processes.

5. Statistical Consultants

Joann Sorra, PhD

Westat

1600 Research Blvd.

Rockville, MD 20850

joannsorra@westat.com

Naomi Dyer, PhD

Westat

1600 Research Blvd.

Rockville, MD 20850

naomidyer@westat.com

Draft 2-2-12

2