JOINT OUTPATIENT EXPERIENCE SURVEY (JOES)
SURVEY SUITE – 0720-JOES
SUPPORTING STATEMENT – PART B
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Description of the Activity
The Joint Outpatient Experience Survey (JOES), JOES-CHAPS Direct Care (DC), JOES-Dental, and JOES-Ambulatory employ rigorous stratified random sampling methodology to elicit generalizable insights across the diverse Military Health System (MHS) landscape. Sampling rates align with visit volumes at each Military Treatment Facility (MTF) to facilitate proportional representation. The overarching aim is meeting target annual response volumes per MTF for robust statistical analysis. Dynamically calibrated sampling protocols factor in facility-level response tendencies, increasing selection where response rates are lower. This balances representation and mitigates non-response bias.
Large MTFs garner proportional allocation to ensure ≥50 completes monthly. Small facilities group into a collective stratum with shared allocation based on their overall visit share, ensuring representation. This framework couples scalable automated sampling with nuanced adjustments to optimize inclusion and uphold statistical validity. Overall, JOES, JOES-CAHPS, JOES-Dental, and JOES-Ambulatory sampling rigor yields actionable insights to inform patient-centered improvements across the diverse Military Health System.
The following are the expected and realized response rates for the four surveys:
Survey |
Expected Response Rate |
Realized Response Rate Calendar Year 2024 as of 5/31/2024 |
JOES |
10% |
7% |
JOES-CAHPS DC |
17% |
14% |
JOES Dental |
4% |
2% |
JOES-Ambulatory |
4% |
2% |
The following is the estimated sample plan by facility:
Parent DMIS |
Facility Name |
Allocation % |
0001 |
AHC FOX-REDSTONE ARSENAL |
0.33% |
0003 |
AHC LYSTER-RUCKER |
0.51% |
0004 |
42nd MEDGRP-MAXWELL |
0.36% |
0005 |
ACH BASSETT-WAINWRIGHT |
0.49% |
0006 |
673rd MEDGRP-JBER ELMNDRF-RICH |
0.51% |
0008 |
AHC R W BLISS-HUACHUCA |
0.17% |
0009 |
56th MEDGRP-LUKE |
0.26% |
0010 |
355th MEDGRP-DAVIS-MONTHAN |
0.26% |
0013 |
19th MEDGRP-LITTLE ROCK AFB |
0.26% |
0014 |
60th MEDGRP-TRAVIS |
0.56% |
0015 |
9th MEDGRP-BEALE |
0.17% |
0018 |
30th MEDGRP-VANDENBERG |
0.17% |
0019 |
412th MEDGRP-EDWARDS |
0.17% |
0024 |
NH CAMP PENDLETON |
1.40% |
0028 |
NHC LEMOORE |
0.33% |
0029 |
NMC SAN DIEGO |
2.10% |
0030 |
NH TWENTYNINE PALMS |
0.33% |
0032 |
ACH EVANS-CARSON |
1.35% |
0033 |
10th MEDGRP-ACADEMY |
0.34% |
0036 |
436th MEDGRP-DOVER |
0.32% |
0038 |
NHC PENSACOLA |
1.49% |
0039 |
NH JACKSONVILLE |
2.06% |
0042 |
96th MEDGRP-EGLIN |
1.66% |
0043 |
325th MEDGRP-TYNDALL |
0.26% |
0045 |
6th MEDGRP-MACDILL |
1.07% |
0046 |
45th MEDGRP-PATRICK |
0.33% |
0047 |
AMC EISENHOWER-FT GORDON |
1.88% |
0048 |
ACH MARTIN-FT BENNING |
2.00% |
0049 |
ACH WINN-FT STEWART |
1.99% |
0050 |
23rd MEDGRP-MOODY |
0.33% |
0051 |
78th MEDGRP-ROBINS |
0.42% |
0052 |
AMC TRIPLER-SHAFTER |
1.64% |
0053 |
366th MEDGRP-MOUNTAIN HOME |
0.17% |
0055 |
375th MEDGRP-SCOTT |
0.67% |
0056 |
JAMES A LOVELL FED HEALTH CARE CENTER |
1.39% |
0057 |
ACH IRWIN-RILEY |
0.55% |
0058 |
AHC MUNSON-LEAVENWORTH |
0.26% |
0059 |
22nd MEDGRP-MCCONNELL |
0.17% |
0060 |
ACH BLANCHFIELD-FT CAMPBELL |
3.10% |
0061 |
AHC IRELAND-KNOX |
0.63% |
0062 |
2nd MEDGRP-BARKSDALE |
0.26% |
0064 |
ACH BAYNE-JONES-POLK |
0.45% |
0066 |
316th MEDGRP-MALCOLM GROW |
1.08% |
0067 |
WALTER REED NATIONAL MILITARY MEDICAL CNTR |
3.79% |
0068 |
NHC PATUXENT RIVER |
0.33% |
0069 |
FT MEADE MEDDAC |
1.68% |
0073 |
81st MEDGRP-KEESLER |
0.93% |
0074 |
14th MEDGRP-COLUMBUS |
0.17% |
0075 |
ACH LEONARD WOOD |
0.60% |
0076 |
509th MEDGRP-WHITEMAN |
0.17% |
0077 |
341st MEDGRP-MALMSTROM |
0.17% |
0078 |
55th MEDGRP-OFFUTT |
0.33% |
0079 |
99th MEDGRP-NELLIS |
0.68% |
0083 |
377th MEDGRP-KIRTLAND |
0.17% |
0084 |
49th MEDGRP-HOLLOMAN |
0.33% |
0085 |
27th SPECIAL OPS MEDGRP-CANNON |
0.17% |
0086 |
ACH KELLER-WEST POINT |
0.44% |
0089 |
AMC WOMACK-BRAGG |
2.86% |
0090 |
4th MEDGRP-SEYMOUR JOHNSON |
0.26% |
0091 |
NMC CAMP LEJEUNE |
1.68% |
0092 |
NHC CHERRY POINT |
0.37% |
0093 |
319th MEDGRP-GRAND FORKS |
0.17% |
0094 |
5th MEDGRP-MINOT |
0.17% |
0095 |
88th MEDGRP-WRIGHT-PATTERSON |
1.42% |
0096 |
72nd MEDGRP-TINKER |
0.33% |
0097 |
97th MEDGRP-ALTUS |
0.17% |
0098 |
AHC REYNOLDS-FT SILL |
0.76% |
0100 |
NHC NEW ENGLAND |
0.86% |
0101 |
20th MEDGRP-SHAW |
0.36% |
0103 |
NHC CHARLESTON |
0.35% |
0104 |
NH BEAUFORT |
0.71% |
0105 |
AHC MONCRIEF-JACKSON |
1.21% |
0106 |
28th MEDGRP-ELLSWORTH |
0.17% |
0108 |
AMC WILLIAM BEAUMONT-FT BLISS |
2.71% |
0109 |
AMC BAMC-FSH |
2.12% |
0110 |
AMC DARNALL-FT HOOD |
2.68% |
0112 |
7th MEDGRP-DYESS |
0.17% |
0113 |
82nd MEDGRP-SHEPPARD |
0.33% |
0114 |
47th MEDGRP-LAUGHLIN |
0.17% |
0117 |
59th MDW-WHASC-LACKLAND |
1.25% |
0118 |
NHC CORPUS CHRISTI |
0.17% |
0119 |
75th MEDGRP-HILL |
0.26% |
0120 |
633rd MEDGRP JBLE-LANGLEY |
0.94% |
0121 |
AHC MCDONALD-EUSTIS |
1.02% |
0122 |
AHC KENNER-LEE |
0.79% |
0123 |
FORT BELVOIR COMMUNITY HOSPITAL |
3.94% |
0124 |
NMC PORTSMOUTH |
5.15% |
0125 |
AMC MADIGAN-FT LEWIS |
2.14% |
0126 |
NH BREMERTON |
0.56% |
0127 |
NHC OAK HARBOR |
0.26% |
0128 |
92nd MEDGRP-FAIRCHILD |
0.17% |
0129 |
90th MEDGRP-FE WARREN |
0.17% |
0131 |
ACH WEED-IRWIN |
0.26% |
0203 |
354th MEDGRP-EIELSON |
0.17% |
0248 |
61st MED SQ-LOS ANGELES |
0.17% |
0252 |
21st MEDGRP-PETERSON |
0.26% |
0280 |
NHC HAWAII |
0.44% |
0287 |
15th MEDGRP-JBHP HCKM-PEARL HRBR |
0.26% |
0306 |
NHC ANNAPOLIS |
0.40% |
0310 |
66th MEDICAL SQUADRON-HANSCOM |
0.26% |
0326 |
87th MEDGRP-JB MCGUIRE-DIX-LAKEHURST |
0.61% |
0330 |
AHC GUTHRIE-DRUM |
1.28% |
0338 |
71st MEDGRP-VANCE |
0.17% |
0356 |
628th MEDGRP-JB-CHARLESTON |
0.33% |
0364 |
17th MEDGRP-GOODFELLOW |
0.17% |
0385 |
NHC QUANTICO |
0.64% |
0607 |
LANDSTUHL REGIONAL MEDCEN |
2.14% |
0609 |
BAVARIA MEDDAC-VILSECK |
1.37% |
0610 |
AHC BG CRAWFORD F SAMS-CAMP ZAMA |
0.17% |
0612 |
ACH BRIAN D ALLGOOD-PYEONGTAEK |
1.33% |
0615 |
NH GUANTANAMO BAY |
0.17% |
0617 |
NH NAPLES |
0.26% |
0618 |
NH ROTA |
0.33% |
0620 |
NH GUAM |
0.47% |
0621 |
NH OKINAWA |
1.32% |
0622 |
NH YOKOSUKA |
0.95% |
0624 |
NH SIGONELLA |
0.37% |
0633 |
48th MEDGRP-LAKENHEATH |
0.79% |
0635 |
39th MEDGRP-INCIRLIK |
0.17% |
0637 |
8th MEDGRP-KUNSAN |
0.17% |
0638 |
51st MEDGRP-OSAN |
0.33% |
0639 |
35th MEDGRP-MISAWA |
0.26% |
0640 |
374th MEDGRP-YOKOTA |
0.33% |
0802 |
36th MEDGRP-MARIANAS GUAM |
0.26% |
0804 |
18th MEDGRP-KADENA |
0.36% |
0805 |
52nd MEDGRP-SPANGDAHLEM |
0.33% |
0806 |
86th MEDGRP-RAMSTEIN |
0.47% |
0808 |
31st MEDGRP-AVIANO |
0.33% |
5537 |
AIR FORCE LINE MEDICAL UNITS |
0.17% |
7139 |
1st SPCL OPS MEDGRP-HURLBURT |
0.56% |
7200 |
460th MEDGRP-BUCKLEY |
0.17% |
The JOES-CHAPS Private Sector Care (PC) surveys beneficiaries who access care through a robust network of civilian providers, facilitated by TRICARE regional networks. This expansive private sector care landscape encompasses over 224 Prime Service Areas across two overarching regions – East and West.
TRICARE private sector care encounters originate from reimbursement claims submitted by providers after delivering services. A monthly file of closed claims provides the sampling frame, with encounters vetted using the same stringent business rules as the direct care component to uphold data integrity. The sampling methodology stratifies by region and dynamically calibrates selection rates to ensure representative inclusion across the national footprint. The East Region accounts for approximately 65% of the private sector care population while the West Region comprises the remaining 35%. This proportional allocation aligns sampling with beneficiary distribution and utilization patterns.
The JOES-Emergency Department survey employs a census methodology at Walter Reed National Military Medical Center, comprehensively sampling all eligible beneficiaries receiving outpatient emergency care. This total coverage approach provides granularity of feedback within this high-priority department.
The JOES Walk-in Contraceptive Clinic survey collects near real time insights using an innovative point of service modality. Via QR code, all patients can immediately provide feedback post-visit, offering unfiltered perspectives on their experience.
Together, these surveys exemplify targeted census and point-of-service data collection strategies that unlock uniquely detailed and timely insights. The JOES-Emergency Department census yields a comprehensive experiential profile of the entire beneficiary population seeking emergency services at Walter Reed National Military Medical Center. Meanwhile, the walk-in contraceptive clinic QR-based approach empowers rapid cycle learning by capturing near real time feedback.
2. Procedures for the Collection of Information
The JOES suite of surveys employs a sophisticated survey management system to enable real-time tracking of beneficiaries across all survey engagement touchpoints. This integrated data infrastructure consists of:
Secure encounter data files containing patient identifiers to facilitate targeted sampling while protecting privacy.
A sample disposition database recording major events and outcomes throughout the survey lifecycle for each beneficiary, providing a comprehensive engagement history. Disposition tables are hierarchically linked by unique IDs for dynamic calculation of current status.
Real-time individual-level monitoring of survey progression, patient identifiers, and contact information to inform tailored engagement and outreach.
Automated compilation of final member-level datasets at survey closeout, merging disposition history with response data from all collection modes.
Daily processing of mail survey returns and online responses continuously refreshes disposition flags, enabling ongoing tracking of duplications, undeliverable correspondence, and eligibility. The survey management system couples large-scale sampling frames with granular per-beneficiary tracking across time and channels.
3. Maximization of Response Rates, Non-response, and Reliability
The JOES suite of surveys apply carefully calibrated business rules to optimize response rates through targeted respondent engagement. Exclusion windows between survey instances mitigate participant burden and minimize opt-outs
For Primary Care encounters, a 90-day exclusion prevents mail re-contacts, while a 45-day exclusion applies for email and SMS. This balances broad reach with appropriate spacing.
Specialty Care mailings enforce a 75-day exclusion, with a 45-day exclusion for email and SMS. Readiness Specialties narrow this further to 60 and 35 days respectively, maximizing their unique participation.
A 6-month provider-level exclusion limits repetitive sampling of patient populations. This reduces fatigue and sustains engagement among high-frequency MTF users. Overall, adaptive protocols promote survey completion by thoughtfully managing respondent burden. Tailored re-contact intervals give patients time to respond while encouraging follow-up for initial non-responders. Strategic sampling fosters participation across varied beneficiary subgroups. Ultimately, intelligently calibrated contact rules bolster involvement to generate insights from as many Military Health System users as possible.
4. Tests of Procedures
No tests of procedures or methods are currently planned.
5. Statistical Consultation and Information Analysis
Names and telephone number of individuals consulted on statistical aspects of the design.
Dr Kimberly Aiyelawo, Defense Health Agency
703-681-8846
kimberley.a.aiyelawo.civ@health.mil
Dr Melissa Gliner, Defense Health Agency
703-681-0366
melissa.d.gliner.civ@health.mil
Dr Alan Roshwalb, Ipsos Public Affairs, LLC
202-420-2029
Timothy Amsbary, Ipsos Public Affairs, LLC
202-296-9197
Amar Patel, Ipsos Public Affairs, LLC
773-680-3814
Names and organization of persons who will actually collect and analyze the collected information.
Timothy Amsbary – Ipsos Public Affairs, LLC
Alan Roshwalb – Ipsos Public Affairs, LLC
Amar Patel – Ipsos Public Affairs, LLC
Dominick Hannah – Ipsos Public Affairs, LLC
Jazz Stephens – Ipsos Public Affairs, LLC
Alka Singh – Ipsos Public Affairs, LLC
Miko Filppula – Ipsos Public Affairs, LLC
Michael Guzy – Ipsos Public Affairs, LLC
Megan Grobert – Ipsos Public Affairs, LLC
Randall Goldammer – Ipsos Public Affairs, LLC
Jade Hyunh-Pring – Ipsos Public Affairs, LLC
Michael Buha – Ipsos Public Affairs, LLC
Sherin Mattappallil – Ipsos Public Affairs, LLC
Sarah Kahl – Ipsos Public Affairs, LLC
Jessica Bradley – Ipsos Public Affairs, LLC
Robert Riley – Ipsos Public Affairs, LLC
Rise Goldstein - Ipsos Public Affairs, LLC
Denise Ye - Ipsos Public Affairs, LLC
Ruth Young - Ipsos Public Affairs, LLC
Kathleen Santos - Ipsos Public Affairs, LLC
Abby Foy - Ipsos Public Affairs, LLC
Camille Hoy - Ipsos Public Affairs, LLC
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Patricia Toppings |
| File Modified | 0000-00-00 |
| File Created | 2024-11-20 |