0720-0031_ssb_2.1.2022

SUPPORTING STATEMENT – PART B

TRICARE SELECT SURVEY OF CIVILIAN PROVIDERS (TSS-P) - #0720-0031

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

If the collection of information employs statistical methods, the following information should be provided in this Supporting Statement:

1.  Description of the Activity

The TRICARE Select Survey of Civilian Providers (TSS-P) is an annual survey designed to gather data on providers (physicians [including primary care physicians, specialist, and mental health providers] and non-physician behavioral health providers) to assess the extent to which they are aware of the overall TRICARE program, accept new TRICARE patients, the extent to which these providers accept Medicare patients, and the reason if they do not. The survey is sent to a sample of 50,000 providers. The expected number of responding mental health providers is equal to the expected number of responding physicians.

The original legislation directing this information collection was Section 723 of Fiscal Year (FY) 2004 National Defense Authorization Act (NDAA), later modified by Section 711 FY06 NDAA (requiring collection from 2005-2007), subsequently amended by Section 711 of FY08 NDAA (P.L. 110-181 requiring collection from 2008-2011), and extended by Section 721 of FY12 NDAA, (Public Law (PL) 112-81) requiring collection from 2012-2015. Section 712 of FY15 NDAA has extended the requirement again to continue the survey from 2017 through 2020.

Section 701 of the FY17 NDAA established TRICARE Select as the replacement for TRICARE Standard as of January 1, 2018. TRICARE Select brings together the features of TRICARE Standard and TRICARE Extra in a single plan. Select enrollees may obtain care from any TRICARE authorized provider without a referral or authorization. The goal is to broaden access for beneficiaries to network providers in TRICARE Select and gives Select beneficiaries access to no-cost preventive services from network providers. To meet this goal, the Department of Defense (DoD) must establish mechanisms for monitoring compliance with access standards.

2.  Procedures for the Collection of Information

Sampling Procedure

TSS-P Universe. Through the 2021 iteration, TSS-P was a survey of physicians (MD or DO). However, there are approximately 325,000 nurse practitioners and 150,000 physician assistants in the United States who provide care to patients.¹ TRICARE reimburses Nurse Practitioners (NP) and Physicians Assistants (PA) at 85 percent of the rate for physician reimbursement. Prior to 2021, the TSS-P used as the sampling frame the American Medical Association (AMA) Master File, which includes only physicians. The transition to the National Plan and Provider Enumeration System (NPPES) includes physicians as well as NPs and PAs who are able to bill Medicare or Medicaid. The FY 2022 survey will expand the universe to include NPs and PAs who are providing care in the eligible specialties. Based on preliminary review of NPPES, about 16 percent of the provider universe are NPs or PAs.

Sample Allocation by Provider Type. For the 2021 design, the fielded sample sizes for mental health providers and physicians were the same, however due to a lower response rate for mental health providers (see table below), the respondent sample size of mental health providers is much smaller than for the physicians. As a result, the 2021 sample has greater precision for physicians than mental health providers. The FY 2022 sample of 50,000 providers will be allocated such that the expected number of responding mental health providers is equal to the expected number of responding physicians. The benefits of this approach are that it will increase the precision level but also will provide similar levels of precision for analysis of both physicians and mental health providers. This change will facilitate more accurate comparisons between the two types of providers.

The response rate in the last iteration of the survey (FY 2021) was 12.6%. Response rates by Provider Type are in the table below.

Response rates by Provider Specialty are provided below.

¹ Mental/Behavioral Health providers

² Physicians

Fielding

Contact information for sampled providers is transmitted to the vendor responsible for fielding the survey. Each sample member is assigned an internally generated ID number. Only that ID is used when the survey is fielded. Responses are recorded and the response data is incorporated into the analysis file using the internally generated ID and reports are prepared.

A multi-mode data collection method is used through a mailed survey with internet option and a telephone follow-up survey. The initial and follow-up survey includes a cover letter signed by a senior DHA principal investigator requesting the recipient’s participation and requesting a response by return mail, internet, or fax, as well as providing a toll-free number to call with any questions and a web address to take the survey via the internet. If providers’ responses to the mailing are not obtained, their offices are contacted by telephone. The telephone survey uses a standardized Computer Assisted Telephone Interview (CATI) protocol.

Mailed surveys are sent to the provider’s stated work address to the extent the work address is different from the home address and can be discerned. Telephone follow-up is to the work address as well, and, similarly, to the extent the work telephone is different from the home address and can be discerned. These surveys are designed to be answered by the billing manager or person responsible for the provider’s billing practices, to minimize the burden on the provider’s practice, and to obtain data the billing expert may be most knowledgeable about. If a recipient receives multiple surveys for multiple providers in the same office or practice group, the recipient is asked to complete a separate mail survey or answer to a separate scripted telephone survey for each provider.

The survey operations contractor administers the telephone survey. The vendor uses standard telephone survey research methodology in administering the telephone questionnaires to include documentation of interviewer training, valid retrievable call records, and a log of interview sessions. A computerized telephone matching service (if needed) and Directory Assistance are used to track current telephone numbers. To optimize the chances of locating respondents and enlisting cooperation, calls are made at different times of the day, on different days of the week, but calls are made only during normal business hours. Calls are not made during weekend or evening hours.

The survey is fielded only to providers with specialties reimbursed by TRICARE, and only to providers who offer care in an office-based practice. Information from the frame is not always sufficient to determine eligibility. Therefore, procedures for determining eligibility are incorporated in fielding and subsequent data processing methods, as described below.

TRICARE reimburses mental health providers of the following types:

• Psychiatrists (or other physicians)

• Clinical psychologists

• Certified psychiatric nurse specialists

• Clinical social workers

• Certified marriage and family therapists

• Pastoral counselors

• Mental health counselors

A respondent is counted as part of the final sample if they are eligible for the survey and a respondent to the questionnaire.

Blank returns are removed, non-respondents, and any respondents found to be ineligible for the survey from the database. In addition, among eligible respondents with a non-blank questionnaire, we included only questionnaires that were “complete” in the database.²

To determine if a questionnaire is “complete”, we chose 3 key questions plus 2 supplemental questions. We accept questionnaires as complete questionnaires if all 3 of these key items plus at least one of the supplemental questions had valid answers. Otherwise, we considered the questionnaire incomplete. These key survey variables and their valid answers are:

• PROVIDE (Question 1): Is [Insert Provider Name], [Credentials] in a practice providing treatment to patients?

• Respondents must answer “Yes” to have a valid answer for this question.

• We treated respondents who answered “No” as ineligible respondents.

• AWARE (Question 3): Are you aware of TRICARE Select (formerly known as TRICARE Standard or Extra)?

• Respondents must answer either “Yes” or “No” to have a valid answer for this question.

• NEWTRI (Question 7): As of today, is [Insert Provider Name], [Credentials] accepting new TRICARE Select patients?

• Respondents must answer either “Yes”, “No”, or “Don’t Know” to have a valid answer for this question.

The supplemental survey variables and their valid answers are:

• NEWTRI1-NEWTRI12 (Question 8): Why is [Insert Provider Name], [Credentials] not accepting new TRICARE Select patients? MARK ALL THAT APPLY

• Respondents must have values of “Yes”, “No”, or valid skips for all values of Question 8 to have a valid answer for this question.

• NEWMED1-NEWMED10 (Question 10): Why is [Insert Provider Name], [Credentials] not accepting new Medicare patients? MARK ALL THAT APPLY

• Respondents must have values of “Yes”, “No”, or valid skips for all values of Question 10 to have a valid answer for this question.

Dispositions are assigned and verbatim responses are coded by the survey administrator to facilitate analysis. The coded response data and the original responses are both returned to Mathematica, where they are reviewed and incorporated into a file for subsequent processing and analysis.

Weighting

Sampling weights are equivalent to the reciprocal of the probability of each respondent’s selection into the sample. Sampling weights are further adjusted for non-response within the classes formed based on the percentiles of the propensity scores from the propensity model. Finally, we post stratified the non-response adjusted weights to the frame totals to obtain weighted totals equal to the population totals within each stratum and trimmed some extreme weights to lessen the effect of extreme weights on variance inflation (thereby reducing the mean square error). Chapter 3 contains the weighted and unweighted frequencies for each variable.

Calculation of variance estimates in the TSS requires a design-based variance estimation technique that is available in most statistical software packages for analysis from a complex survey data, such as SAS/STAT® version 8 or higher, and STATA®. This technique requires sample design information, including the analysis weight and stratification information. As an alternative, a replication technique such as the Jackknife method can be used to calculate variance estimates. In the TSS, a series of 60 jackknife replicate weights are calculated and attached to each record in the database. These weights are computed by systematically dropping a one-sixtieth portion of each stratum and recomputing the weights then for the remaining 59/60ths of each stratum. We processed the replicate weights through all of the weighting steps so that all components of the final variance are accurately represented (effects of non-response in particular). The sum of squares from the jackknife replicate estimators generated using these weights will allow for a consistent variance estimator for both linear and nonlinear statistics (with coefficient 59/60).

Reporting and Analysis

An annual report as well as accompanying briefing slides will be produced from the results of this survey. This annual report provides results from the accompanying TSS-Beneficiary survey as well.

TSS-Provider results from the last report (2021) are shared below. Provider results are reported by physician and non-physician behavioral health providers. Within physicians, results are split out by primary care physicians (PCPs), specialists, and psychiatrists.

FY 2021 TSS-Provider Report Summary: Methods

Provider Sample The provider survey included a sample of 50,000 providers using strata of provider type (primary care physicians, specialist, and mental health providers) and Prime Service Areas (PSAs) and non-Prime Service Areas (Non-PSA). Approximately 30% of surveys were returned, but after accounting for ineligible responses, 6,313 eligible completed surveys were retained for analysis.

Provider Fielding The provider survey was fielded by DataStat by mail with phone follow-up. Respondents could reply by mail, phone, web, or fax. The 2021 survey was conducted from January 2021 to March 2021.

Weighting For both beneficiary and provider surveys, the initial sample was weighted with base weights, and the returned sample was weighted with non-response and post-stratification weights. A series of 60 jackknife replication weights were calculated. This greatly increases the accuracy of the sample data in estimating the population.

Analysis To account for the complex survey design, all analysis was performed using the SAS survey procedures, such as the surveyfreq and surveymeans procedures. These analyses incorporate the design and weighting of the survey data and are necessary for accurate population estimates. Beneficiary results are reported overall and by PSA-status. Provider results are reported overall, by provider type (Physician or Behavioral Health) and by physician type (primary care physician, specialist, and psychiatrist).

FY 2021 TSS-Provider Report Summary: Findings

TRICARE Acceptance

• 82% of physicians and 60% of behavioral health providers were aware of TRICARE Select.

• 80% of physicians and 42% of behavioral health accept new TRICARE Select patients.

• Within physicians, 79% of PCPs, 85% of specialists, and 74% of psychiatrists were aware of TRICARE Select.

• Within physicians, 75% of PCPs, 89% of specialists, and 49% of psychiatrists accept new TRICARE Select patients.

Reasons for not Accepting TRICARE

• The top reasons for not accepting TRICARE Select were “Other,” “Not aware of TRICARE Select,” and “Not accepting new patients.”

• Physicians were more likely to not accept TRICARE Select because they were not accepting new patients.

• Behavioral health providers were more likely to not accept TRICARE Select because of “Other”, they were not aware of it, they had problems being accepted, or they only took private insurance.

• Open text analysis revealed many behavioral health providers were not eligible to be credentialed or worked in facilities or positions that did not accept insurance, such as in schools, prisons, or as social workers.

• Some providers stopped accepting TRICARE Select because of non-payment of claims.

.

3.  Maximization of Response Rates, Non-response, and Reliability

The cover letter that accompanies each mailed survey is the primary method used to encourage participation in the survey effort. Both the cover letter and telephone script include information about the purpose of the survey and a brief description of how the information will be used by DHA and the importance of completing this short survey for the benefit of military beneficiaries. For offices with multiple selected physicians and mental health providers, the billing manager recipient will receive separate surveys for each requested physician, and will be asked to complete one survey for each. In addition, telephone interviewers are trained in interviewing techniques designed to minimize incidences of respondent refusals to participate in the survey. They ask respondents to answer separately for each physician in cases where multiple doctors are being surveyed in the same office.

Non-response analyses have indicated that membership in the TRICARE network is positively related to survey response. Thus, in order to ensure that the data can be generalized to the universe under study, an indicator of network membership is obtained by linking to the sample frame an indicator of network membership from membership lists provided by the TROs. If non-response adjusted sampling weights are employed, rates calculated from survey responses are representative of the population under study.

4.  Tests of Procedures

The sample and results of surveys fielded from 2012 to 2015 were evaluated by the Government Accountability Office (GAO) in 2017 and found to be statistically and methodologically sound.

Modified sampling techniques moving forward with the 2022 survey to encourage equal response rates from mental health providers and physicians should result in higher response rates overall.

5.  Statistical Consultation and Information Analysis

a. Provide names and telephone number of individual(s) consulted on statistical aspects of the design.

Melissa Gliner, PhD

Defense Health Agency, Analytics and Evaluation Division

703-681-3636

Chantell Frazier, PhD

Altarum Institute

202-776-5196

Christopher Duke, PhD

Altarum Institute

734-478-0922

Joseph Gasper, PhD

Westat

301-251-1500

b. Provide name and organization of person(s) who will actually collect and analyze the collected information.

Melissa Gliner, PhD

Defense Health Agency, Analytics and Evaluation Division

703-681-3636

Marielle Weindorf

DataStat

734-994-0540

Christopher Duke, PhD

Altarum Institute

734-478-0922

Laura Pinnock

Altarum Institute

734-302-4662

Joseph Gasper, PhD

Westat

301-251-1500

1 See https://www.aanp.org/about/all-about-nps/np-fact-sheet and https://www.nccpa.net/wp-content/uploads/2021/07/Statistical-Profile-of-Certified-PAs-2020.pdf

2 There were 30 respondents with an unknown eligibility status who also had complete questionnaires. We included these 30 respondents among the eligible respondents with complete questionnaires.

10