Supporting Statement: Part B
National Ambulatory Medical Care Survey (NAMCS) Supplement of Primary Care Policies (NSPCP) for Managing Patients with High Blood Pressure, High Cholesterol, or Diabetes
OMB No. 0920-NEW
Contact Information:
Christine A. Lucas, MPH, MSW
Ambulatory and Hospital Care Statistics Branch
National Center for Health Statistics
Centers for Disease Control and Prevention
3311 Toledo Road, Room 3317
Hyattsville, MD 20782
Voice: 301-458-4071
May 1, 2015
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Respondent Universe and Sampling Methods 3
2. Procedures for the Collection of Information 10
3. Methods to Maximize Response Rates and Deal with Nonresponse 12
4. Test of Procedures or Methods to be Undertaken 12
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 13
Target Population
The target population for the National Ambulatory Medical Care Survey (NAMCS) Supplement of Primary Care Policies for Managing Patients with High Blood Pressure, High Cholesterol, or Diabetes (NSPCP) consists of physicians in the United States who provide primary care to adults and whose specialties are Internal Medicine (IM) and Family Practice (FP). The specialty of Family Practice is also referred to as Family Medicine (FM). However, there is also interest in learning about the practices that employ IMs and/or FPs. Thus, data will be obtained to permit both physician level and practice level estimation. The eligibility criteria for physicians in the study are:
specialty is IM or FP;
employer is not the federal government; and
physician is routinely involved in patient care (i.e., not engaged solely in teaching, administration, or research), though they may not be principally engaged in patient care activities (i.e., a physician could be engaged in patient care activities less than half-time, for instance, less than 20 hours in a typical work week).
Participants in the National Ambulatory Medical Care Survey (NAMCS), a national survey of patient visits to office-based physicians also conducted by NCHS (OMB No. 0920-0234 expires 12/31/2017) will be excluded from participation in the NAMCS NSPCP.
Choosing a Sampling Frame
In order to develop both physician level estimates and practice level estimates for the primary care specialties of IM and FP, as well as practice-level estimates of practices that employ these specialties, a sampling frame that provides very high coverage of the two specialties is needed. The sampling design will follow an approach discussed by DiGaetano1. A stratified systematic random sample of physicians will be selected from strata defined by the two specialties. In addition to collecting the analytic data of interest, we will obtain the number of physicians in each of the IM and FP specialties working at the same organization as the sampled physician. In so doing, we will have an indirect sample of such organizations selected with probability proportionate to size (pps), where the size measure reflects the number of physicians in each organization in the two specialties.
This has added benefits with respect to the sample allocation of practices, since practices with a single physician are expected to be disproportionately high in terms of the population of practices compared to the proportion such physicians represent among the population of physicians. The number of solo physician practices has declined (the percent change between 2007 and 2012 is -6.2)2 but there are still expected to be relatively more of them in family practice and internal medicine specialties.
For this survey, we will use the American Medical Association (AMA) Masterfile as the source of our sampling frame. It is frequently used for such purposes, including for the NAMCS. The AMA file is expected to provide very high coverage of the population of physicians in the U.S. AMA establishes records for students who are entering accredited medical schools, helping to ensure high coverage of those ultimately obtaining a medical degree. In addition, AMA has provisions to include physicians from foreign countries who are practicing in the U.S. To reduce burden, the sampled physicians for the NAMCS NSPCP will not be included in the sample for NAMCS. In other words, even though the NAMCS NSPCP and NAMCS samples are drawn from the same AMA Masterfile frame, physicians participating in the NAMCS NSPCP will not be asked to participate in NAMCS as well.
Virtually all physicians on the AMA Masterfile in the IM and FP specialties will appear on the Sampling Frame in order to retain high coverage of the targeted specialties. The exceptions are those flagged as “retired” or who are characterized as inactive for reasons other than temporary absence or retirement.
In the table below we indicate the expected number of physicians that will appear on the Sampling Frame based on recent data provided by MMS, the vendor that provided the AMA’s Masterfile for recent sampling frames used in NCHS physician surveys. There are close to 115,000 FPs and over 137,000 IMs expected to be on the sampling frame.
Physician Specialty |
AMA Master file Count of Physicians on Sampling Frame April 2014 |
Projected Number on Frame in Target Population |
Family Practice |
113,902 |
55,000-75,000 |
Internal Medicine |
137,014 |
45,000-55,000 |
We have also projected the number of the physicians found on the sampling frame expected to be members of the target population for this study: between 55,000 to 75,000 FMs and 45,000 to 55,000 IMs. These numbers were projected based on the latest available NAMCS estimates (2010) as well as estimates from a survey of Oral Health conducted by the contractor3 in 2013. The NAMCS estimates were 57,803 FPs and GPs (the NAMCS does not distinguish between the two specialties for estimation purposes although the vast majority would be FPs) and 44,882 for Internal Medicine. The Oral Health survey estimates were 69,288 FPs and 48,261 IMs.
Stratification
The primary sampling strata will be defined by physician specialty, family practice and internal medicine. The primary care physicians (PCPs) in these two specialties account for most of the primary care provided to the U.S. adult population, and the PCPs in these two specialties are of greatest analytic interest to this study. As a result, we will use specialty as the primary sort variable prior to sample selection rather than select samples from each specialty separately.
The AMA Masterfile has a number of variables that can be used for sorting purposes within specialty for implicit stratification achieved by sorting when systematic sampling is used Primary candidates for this sorting will be the variables of age and specialty. Other candidates are gender, degree (MD, DO), and/or geography (e.g., region of country, MSA status).
Sample Allocation and Selection
We plan to select, using systematic random sampling, a total of 3,000 physicians -- 1,500 from each of the two specialties for fielding purposes. Response rates for the NAMCS NSPCP are expected to be about the same for the physicians sampled from the two specialties, roughly 60 percent. However, there are expected to be substantial differences in eligibility rates with roughly 65 percent of sampled FPs and 40 percent of sampled IMs expected to be members of the survey target population.
As a result, the sample distribution of eligible respondents is expected to be close to the population distribution across the specialties so that the design effect associated with differential sampling rates is expected to be small. The table below indicates the expected sample sizes and yields.
Expected Sample Sizes and Yields
Specialty |
Expected Sample size (Number to be Selected) |
Expected Field Response rate |
Expected Eligibility Rate |
Expected Yield (number of eligible respondents) |
Expected Target Population Distribution between Specialties |
FP |
1,500 |
0.60 |
0.65 |
585 |
0.58 |
IM |
1,500 |
0.60 |
0.40 |
360 |
0.42 |
Total |
3,000 |
|
|
945 |
1.00 |
We plan to sample physicians in order to survey the practices to which they belong. An equal probability sample of physicians will produce a pps sample of practices. This approach is described in detail by Kish4 and was the basis for the methodology used for the report by Hing and Burt4. This sampling design serves to oversample practices with more than one physician in the targeted population, and these are the ones considered more likely to have policies in place that are of interest to the study.
Estimation
Physician Level Estimation
The probability of selection of each sampled physician will be known at the time of sample selection and will be a constant. The reciprocal of this probability will serve as the initial or base weight of each sampled physician.
The final survey weight for physicians will be formed after adjustment for nonresponse at the screener and survey levels. The adjustment process is described in the following two sections.
Adjusting for Screener Nonresponse
Screener nonresponse can be incurred if a sampled physician cannot be located or if, when the sampled physician’s place of employment has been identified, insufficient information is gained as part of the screening process to enable us to contact a physician determined to be eligible to receive a survey questionnaire.
Variables available for purposes of weight adjustment for screener nonresponse from the sampling frame include specialty, age, gender, type of practice, region of country, etc.
An evaluation of these variables will be undertaken to identify those that appear most effective in distinguishing between subgroups with different propensities to respond. We plan to employ Chi-squared Automatic Interaction Detection CHAID for this purpose. CHAID is a commonly used tree-based algorithm for studying the relationship between a dependent variable and a series of predictor variables.
Once the CHAID analysis is completed, cells will be formed from the variables identified as effective in distinguishing between response propensities, and the weights of participating physicians associated with the cell will be adjusted to compensate for those in the same cell who do not participate.
Survey Nonresponse
As part of the screening process, additional information will be collected that can serve to adjust the weights of screener participants for nonresponse to the survey instrument. For example, we plan to collect information on the size of the practice. Any such variables in addition to those used to adjust for screener nonresponse will also be analyzed via CHAID models to determine useful cells to form to adjust for nonresponse to the survey among screener respondents.
Practice Level Estimation
Base Weight
The probability of selection of the physician will be known at the time of sample selection. The probability of selection of the practice will be determined from the survey data where we will have collected the number of FPs and IMs seeing patients at the practice. It will be computed reflecting the differential probabilities of selection for FPs and IMs as well as the number of FPs and IMs identified at a particular practice (e.g., if a practice reports having 4 FPs and 3 IMs and the probabilities of selection for the two specialties are RF and RI, respectively, the probability of selection of the practice can be computed as 1- (1-4RF)(1-3RI)). For illustration purposes we will suppose that the probability of selection of physicians is .012 for both FPs and IMs. Then, if a participating practice reports 4 FPs and 3 IMs who are members of the practice, the probability of selection of the practice will be roughly .082.
The base weight for a practice will simply be the reciprocal of its chance of selection. In the example above, the base weight for the physician is roughly 83.3, and the resulting base weight for the practice will be roughly 12.2 (=1/.082).
Implicit in this approach is the assumption that the sampling frame provides high coverage of IMs and FPs, so that physicians counted by a practice can be expected to be on the sampling frame. As discussed earlier, it is a reasonable expectation that the full AMA file provides high coverage.
Since the probability of selection of nonresponding practices may not be known for those without a completed screener (the number of FPs and IMs may not have been obtained), special efforts will be made to obtain such information with data retrieval efforts (e.g., phone calls, internet searches, etc.).
Adjusting the Practice Base Weight for Nonresponse
As with the physician level estimates, practice level estimates will be adjusted for nonresponse. The information on the sampling frame pertains to the sample physician rather than sampled practices. Sampling physicians to select a probability sample of practices is an example of indirect sampling. Methods of dealing with survey nonresponse for indirect sampling involve modeling and we will use such an approach.
Variance Estimation
For purposes of variance estimation and analyses, either replicate weights will be created using a jackknife replication methodology or “stratum” and “PSU” variables will be created for use with a Taylor Series approach 5to variance estimation as applied in SUDAAN.
Expected Levels of Precision
The expected number of responding eligible primary care physicians (FPs plus IMs) is 945 (the expected effective sample size for estimates of 0.5 of total targeted physicians, resulting from the use of differential sampling rates, is about 937). At the 95% confidence level for an estimated proportion of .5, the corresponding margin of error is .032. For an estimated proportion of .2, the corresponding margin of error is .026.
For practice level estimates the expected number of responding practices is about the same as the number of responding physicians. However, we are selecting a pps sample of practices expected to result in a non-trivial design effect associated with variable weights. The size of this design effect is uncertain since the size distribution in terms of practices/organizations employing IMs and FPs is unknown. The estimated size distribution will be of methodological interest for future studies interested in practice level estimates.
We have attempted to assess what this design effect might be using estimates appearing in the report by Hing and Burt2. They provided both an estimated population distribution of practices by size category (1, 2, 3-5, 6-10, and 11+) and physicians who are members of practices within these size categories. While these data do not apply directly to practices/organizations that employ FPs, IMs, or both and are somewhat out of date, we felt it may be of interest to attempt to get some sense of what the design effect might be. From this information we have projected that the size of the design effect for estimated proportions for the population of practices as a whole may be in the vicinity of 1.5. Thus, the effective sample size (the size of a simple random sample with the same level of precision) with 945 respondents could be roughly 630 (=945/1.5). The resulting margin of error for a 95% confidence interval for an estimated proportion of .5 for the population of practices would be about .039.
In addition, as mentioned earlier, it is expected that the practices with the policies of interest to the study will be disproportionately concentrated in practices that have more than one physician. Thus, by oversampling these practices, we expect to have increased, perhaps substantially, the precision of estimates about the characteristics of practices that have a particular policy in place compared to what would be obtained by an equal probability sample of practices from a frame with high coverage of all practices in the country (if one existed).
Nonresponse Bias Analyses
Because the response rate for this study is expected to be lower than 80 percent, we plan to conduct some nonresponse bias analyses to help assess the extent to which there is potential for nonresponse bias to arise in the survey estimates. The process of adjusting the weights for screener nonresponse serves as one vehicle for these analyses. Subgroups or adjustment cells where response rates are particularly low can be readily identified through the CHAID analyses. Size of practice is expected to be correlated with propensity to respond to the survey itself, and information collected through the screening process will permit us to examine this in particular.
The contractor has undertaken a number of physician studies that include FP and IM primary care physicians, and some estimates from these studies as well as the NAMCS may provide estimates for comparison purposes. Most of these studies restricted the sampling frame to those physicians flagged as office-based on the AMA, but their eligibility requirements were similar to those employed for the NAMCS NSPCP. NAMCS NSPCP estimates can be restricted to respondents flagged as office based to achieve comparability when needed.
The studies done by the contractor have all used incentives to help increase survey participation. NAMCS related mail surveys do not employ an incentive, and no incentive is planned for the NAMCS NSPCP.
Response Rates
The contractor has carried out a number of physician surveys involving the specialties of Family Practice and Internal Medicine. For instance, the Cancer Screening Survey (CSS) was carried out by the contractor in the fall of 2006 for the National Cancer Institute (NCI) (see National Survey of Primary Care Physicians for Breast, Cervical, Colorectal and Lung Cancer Screening, OMB No. 0925-0562, exp. date 7/31/2009).
The CSS was a mail survey as well but included an incentive of $50 provided to all sampled physicians. For the CSS, following the NAMCS model, the AMA Masterfile was used as the source of the sampling frame and only physicians flagged as office-based on the AMA file were included. The overall response rates for the CSS were on the order of 70 percent for both the FPs and IMs.
Based on these considerations, we expect the NAMCS NSPCP overall response rate, after accounting for the “no contacts”, to be roughly 55 percent.
2. Procedures for the Collection of Information
Data Collection
Tracing
Following sample selection, information for selected physicians will be reviewed for completeness. Any selected physician for which there is not a telephone number and/or mailing address on the file will be traced to try to identify that information.
Screening
Up to four contact attempts will be made to conduct telephone screening for each sampled physician. It is anticipated that the screening will likely take place with office staff, rather than the physician him/herself. Please see Attachment 3a for the screening instrument. The primary purposes of the screening are to:
Confirm that the sampled physician works at the practice that has been contacted. If a physician no longer works at that practice, the telephone interviewer will try to obtain new contact information for the physician. If new contact information cannot be obtained over the phone, the case will be assigned to tracing in order to try to find new contact information for the selected physician. Once new contact information has been identified, the screening process will resume.
Confirm that the selected physician’s specialty is either IM or FP. Cases where the physician is reported to have a specialty other than IM or FP will be further investigated to verify the specialty. This verification will involve looking at practice websites, looking at NPI listings, etc. If the specialty is verified as something other than IM or FP, the physician will be designated ineligible for the study and will not receive the mailings. If the specialty cannot be verified, the physician will remain eligible for the study and will receive survey mailings.
Confirm that the physician is still practicing. If the person answering the phone reports that the physician is retired, deceased or no longer practicing medicine for some reason, the information will be noted and the physician will be deemed ineligible for the study.
Determine the number of IM and FP practitioners at the practice. Practice level estimates, while not the primary focus of the study, are of analytic interest. This information is needed in order to determine the chance of selection of the practices where the sampled physicians work, as discussed earlier.
Previous physician surveys using the basic screening approach described above have included more eligibility questions about the physician. However, the respondent to these screener questions is almost always an office staff person rather than the physician him/herself. There is some uncertainty about the reliability of some of the information about the physician provided by the office staff. We plan to conduct an evaluation of this type of eligibility screener question in order to determine the quality of the data collected. This will involve comparing the responses on the screener to the same information collected on the survey instrument completed by the sampled physician. Physician eligibility provided by office staff for criteria considered highly likely to be accurate (e.g., deceased, retired, no longer in practice) will be accepted and no survey questionnaire will be sent out. For those eligibility criteria obtained during the screening where there is some uncertainty related to the accuracy of the information obtained, a survey questionnaire to the sampled physician will be mailed so that eligibility criteria can be addressed by the physician him/herself. In so doing, we will be able to compare physician versus non-physician responses related to the following criteria.
Whether the physician sees all of his/her patients in:
a hospital;
an urgent care or immediate care facility;
a Federal facility such as a VA office, a military clinic, a Public Health Service clinic, or an Indian Health Service clinic; or
a nursing home, rehabilitation center or correctional facility.
The extent to which there is consistency between physician and non-physician responses for those where we have both can help guide our nonresponse adjustment process in the development of the weights for sample physicians.
Mail Survey
Respondents not coded as ineligible during screening will be sent the survey instrument via Priority Mail along with a cover letter. Two weeks after the initial invitation, nonrespondents will be sent a second survey. Two weeks after the second set of surveys, a third set of surveys will be sent to nonrespondents. If a respondent completes the survey at any point, he/she will be removed from additional mailings or contacts. Please see Attachments 5a and 5b for the cover letters and Attachment 3b for the draft survey instrument.
Telephone Calls
Two weeks after the third mailing, telephone calls will be made to all non-responders to confirm receipt of the survey in the office and to request that non-respondents complete the survey. If willing, the survey will be completed over the phone with the selected physician.
3. Methods to Maximize Response Rates and Deal with No Response
The data collection procedures described above was developed to help maximize response rates. Tracing prior to the start of the field period is intended to provide the most accurate starting sample possible. Tracing as needed throughout the field period is designed to ensure that all sampled respondents are contacted and offered an opportunity to participate. The repeated mailings (with letters slightly different content) are intended to motivate respondents to participate in the survey. A reminder in a different mode (the reminder phone call) is designed to capture the attention of respondents who have not responded to mailed invitations. Additionally, respondents who have not responded to any of the mailings will be invited to complete the survey over the phone.
4. Tests of Procedures or Methods to be Undertaken
The NAMCS NSPCP will undergo pretesting before the instrument is fielded in final form. Because the questions on the NAMCS NSPCP survey instrument are almost all new, it is anticipated that additional testing will be needed to ensure that they are capturing the intended information and that they are understandable by respondents. In addition, the format of the instrument will be tested to ensure that respondents understand the skip patterns and grid items. This additional testing will consist of 30 cognitive interviews. All testing will be conducted with persons identified as key respondents in primary care practices that include a FP or IM physician. The cognitive interview participants will be purposively recruited from primary care medical practices with differing numbers of physicians and different types of practice ownership (physician or physician group, HMOs, and hospital-owned). Recruiting techniques will include requests for assistance from organizations such as the Medical Group Management Association and large health systems, the distribution of recruiting flyers. Any resulting changes to the data collection instrument are anticipated to be either cuts or minor wording changes and will be submitted to OMB prior to fielding the NAMCS NSPCP. Please see Attachment 4a for the testing protocol for the 30 cognitive interviews.
5. Individuals Consulted on Statistical Aspects and the Individuals
The statistician responsible for the survey sample design is:
Iris Shimizu, Ph.D.
Mathematical Statistician
Statistical Research and Survey Design Staff
Office of Research and Methodology
National Center for Health Statistics
Telephone: 301/458‑4497
The data collection agent is Westat, Inc and the contact person is:
Terisa Davis, Project Director
Westat, Inc
1600 Research Blvd
Rockville, MD 20850
Telephone: 301/251-1500
The data will be analyzed under the direction of:
Denys T. Lau, Ph.D.,
Deputy Director, Division of Health Care Statistics
National Center for Health Statistics
Centers for Disease Control and Prevention, DHHS
Telephone: 301/458-4802
Mr. Ralph DiGaetano is the statistical consultant for the survey. Mr. DiGaetano will be overseeing the data collection and topline analysis of the NSPCP Survey.
Ralph DiGaetano, MA
Senior Statistician, Westat, Inc.
301-294-2062
Ms. Martha Franklin is the survey design consultant for the survey:
Martha Franklin, MA
Instrument Design Specialist, Westat, Inc.
301-610-4826
Ms. Hilary Wall is the primary technical consultant from CDC who provided much of the survey content expertise:
Hilary Wall, MPH
Health Scientist, DHDSP/NCCDPHP/CDC
770-488-8172
Dr. Julie Will provided statistical expertise on survey design, sampling, and analysis plans:
Julie Will, PhD, MPH
Senior Epidemiologist, DHDSP/NCCDPHP/CDC
770-488-4835
Dr. Denys Lau is responsible for receiving and approving contract deliverables:
Denys Lau, Ph.D.
Deputy Director, Division of Health Care Statistics
National Center for Health Statistics
301-458-4802
Ms. Christine Lucas is the COR on this project:
Christine Lucas, MPH, MSW
Survey Statistician, Ambulatory and Hospital Care Statistics Branch
Division of Health Care Statistics
National Center for Health Statistics
301-459-4071
LIST OF ATTACHMENTS
1. Authorizing Legislation: Public Health Services Act
2. Federal Register Notice
2a. 60-day FRN
2b. Summary of Public Comment
3. Data Collection Instruments
3a. Screener
3b. Survey
4. Cognitive Testing Materials
4a. Protocol
4b. Screener
4c. Consent Form
4d. Cognitive Recruitment Documents
5. Contact Materials
5a. Invitation Cover Letter
5b. Reminder Cover letters
6. Review Approvals
6a. NCHS ERB Approval
6b. Contractor IRB Approvals
7. Consultations Outside the Agency
1 DiGaetano, R.; Sample Frame and Related Sample Design Issues for Surveys of Physicians and Physician Practices; doi: 10.1177/0163278713496566. Eval Health Prof September 2013 vol. 36 no. 3 296-329.
2Hsaio,C, et.al.: Trends in Electronic Health Record System Use Among Office-based Physicians: United States, 2007-2012; National Health Statistics Reports Number 75, May 20,2014.
3 Westat, Survey of Primary Care Physicians on Oral Health: Methodology Report. Report submitted to the US Department of Health and Human Services, Office of Women’s Health. August 30, 2013.
4 Kish, L. (1965). Sampling Organizations and Groups of Unequal Sizes. American Sociological Review, Vol. 30, No 4, pp. 564-572.
4 Hing, E. Burt, C.; Office-based Medical Practices: Methods and Estimates from the National Ambulatory Medical Care Survey; Advance Data from Vital and Health Statistics; Number 383; March 12, 2007.
5Research Triangle Institute. SUDAAN Language Manual, Volumes 1 & 2, Release 11.0. Research Triangle Park, NC:, 2011
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |