MOVE Memo

MOVE! Weight Management Program for Veterans

Survey of Patient Experiences

Introduction

As the Veteran Health Administration’s (VHA) National Program Office responsible for policy and oversight regarding weight management treatment, we will use the information collected through this research study to evaluate and improve the MOVE! Weight Management Program for Veterans. The MOVE! Weight Management Program for Veterans is a behaviorally-based, supported self-management program governed by VHA Handbook 1101.1¹ and the effectiveness of such programs is largely based on patient involvement and engagement, which makes obtaining information about patient program experiences directly from patients critical. As of September 2009, nearly 204,000 patients have been seen with MOVE! at more than 1,047,000 visits at over 350 sites of care (Medical Centers, Independent Outpatient Clinics, Community Based Outpatient Clinics).² To date, no systematic investigation of patient experiences with MOVE! has been undertaken.

The primary purpose of the survey is to evaluate overall patient satisfaction with the MOVE! program. Based on the information we collect, the MOVE! Program will be further refined and improved for local implementation. Secondary research aims include determining dietary and physical activity behaviors and neighborhood characteristics for those who are receiving or who have received treatment. A number of VHA researches with subject matter expertise in these areas contributed to the development of the survey instrument, such that the information we learn from this survey can guide future research in these areas. This research study has been approved by the Durham Veterans Affairs Medical Center Institutional Review Board and has been granted a Waiver of Informed Consent and a Waiver of HIPPA Authorization.

Methods

Design:

This cross-sectional study will be conducted using a postal survey mailed to a sample of 5,000 eligible study subjects.

Study Population:

The Study Population (also known as Universe) is the collection of patients with at least 2 MOVE! Weight Management treatment-related visits in FY09 and/or FY10 to date as identified by Decision Support Service (DSS) identifiers (“stop codes”) 372 or 373, which are exclusively reserved for the use of MOVE! Weight Management related treatment clinic visits or telephone contacts. We will use the OPC treatment file (MDPPRD.MDP.SAS.SE09) located at the Austin Information Technology Center (AITC) to identify the universe of respondents. We will limit the potential universe by excluding the following subjects:

• Patients who received bariatric surgery within or outside VHA (i.e. fee basis) during FY09 or FY10 to date

• Employee-patients who received MOVE! related treatment identified with occupational health DSS identifiers

The remaining subjects within the universe will be assigned to the facility where they have received the majority of treatment (more than half their visits). Facilities included in this study are those located in the continental US, Alaska and Hawaii. Patients from Philippines and Puerto Rico or from CBOCs that are contracted will be excluded from the universe.

Sampling:

A two-stage, stratified cluster sampling design will be used to select eligible subjects from facilities. VA hospitals and non-contracted community-based outpatient clinic (CBOC) constitute the primary sampling units (PSU). VA Hospitals will be stratified based on their facility complexity criteria, 1a, 1b, 1c, 2 or 3, where 1a comprises the more complex facilities and 3 corresponds to the least complex facilities. This complexity model was developed by the VHA National Leadership Board to provide a consistent and systematic approach to classifying VHA facilities for the purposes of comparison.³ (Note: Orlando (VISN 8, Facility #675) and New Orleans (VISN 16, Facility #629) were not assigned a complexity level during the latest model refinement, thus we assigned them both to complexity level 3 given how other large outpatient independent clinics are assigned.) We classified CBOCs into four strata (small, medium, large, and very large) based on the FY 2009 total outpatient census at the CBOC.

The desired final sample size for this survey is 5,000 and was selected based on several factors including statistical considerations (precision and validity of estimates) and feasibility factors (resources required for survey deployment and analysis). The final sample size will be allocated as 60% (n=3,000) for VA Hospitals (strata 1a to 3) and 40% (n=2000) for CBOCs (strata “small” to “very large”). Previous studies suggest that a 66% response rate is a reasonable assumption for this study. This assumption is based on the current response rate for the VHA Survey of Healthcare Experiences of Patients (SHEP), a similarly designed and deployed VHA patient satisfaction survey. Based on this information, it is expected at least 3,300 (=5,000 x .66) will return a fully or partially completed survey.

Table 1. VA Hospitals and CBOCs by Stratum and FY09 Total MOVE Unique Patient Counts

Stratum	# PSUs*	FY09 Total MOVE Unique Patients	# PSUs with < 30 MOVE Patients	Sample Size	FY09 Female Total MOVE	Expected Sample Size of Females	Desired Sample Size of Females
1a	34	20,108	0	1,177	2,469	480	588
1b	15	7,470	0	437	796	58	218
1c	17	6,428	0	376	833	52	188
2	34	8,992	1	526	1,015	88	263
3	39	8,263	3	484	935	75	242
Subtotal	139	51,261	4	3,000	6,048	753	1,500
CBOC—Small	64	1,242	50	166	78	10	90
CBOC—Medium	107	4,165	55	600	391	167	303
CBOC—Large	49	4,283	11	624	414	183	312
CBOC—Very large	31	4,048	3	589	532	221	295
Subtotal	251	13,738	119	2,000	1,581	581	1,000
Total	390	64,999	123	5,000	7,629	1,334	2,500

*PSU=primary sampling unit

Given the observed range in the number of MOVE outpatients across PSU in FY09, each stratum will be divided in two sub-strata, PSUs with less than 30 MOVE patients and PSUs with 31 or more MOVE patients. All PSUs with less than 30 MOVE patients will be removed from the population frame as they are deemed “start up” sites.

A sample of PSUs will be selected at random from each stratum using probability proportional to the size (number of MOVE outpatients). Proportional allocation will be used to determine the size of the random sample that will be drawn from each stratum within the Facilities and CBOCs. In proportional allocation the more units the stratum contains, the larger the sample will be for this stratum. This allocation will result in a representative sample in the sense that it will have an outpatient distribution similar to the population’s distribution⁴. Table 1 shows the expected size of a sample of outpatients (both genders) and the expected number of females that will be drawn from each stratum using proportional allocation. The expected number of female outpatients (see “Expected Sample Size of Females” in Table 1) in the sample was estimated using the stratum gender distribution from 2009 data. Female patients will be oversampled to ensure sufficient statistical power to detect differences in outcomes and to provide estimates with enough precision for both genders.

Since information regarding the gender of patients is available in advance, we will stratify the listings of MOVE outpatients of the selected PSUs by gender, and within each gender (male and female) we will select a random sample of approximately the same number for both males and females. For example, if a selected PSU was assigned 36 sampling units, then we will select 18 females and 18 males at random from the list of MOVE patients attending the selected PSU.

If the sample allocated to a selected PSU is larger than the actual number of MOVE females and/or males, then we will include all the sampling units in the selected PSU and the remaining number of units to be sampled will be reallocated using the same procedure used before. For example, if the sample size for a selected PSU is 120 outpatients, then the number of females to be selected is about 60. If the number of female outpatients in the PSU is 40, then we will include the 40 female patients in the sample, and the remaining 20 females will be reallocated to the other strata using proportional allocation.

Two general rules of thumb frequently used when deciding the selection on the number of primary sampling units and the cluster sizes are: the more PSUs are selected in the sample the better, as both geographic representation, or spread, and overall reliability will be improved; the smaller the cluster size, the more reliable the estimates will be^5,6. Another consideration in determining the number of PSUs and respondents from PSUs selected is based on the level where comparisons are desired. If a research goal is to compare among subpopulations (e.g. strata or gender), then similar sample size is desired for each sub-population.

In order to have enough sample size at different levels to obtain reliable estimates when comparing estimates for detecting differences across strata, PSUs groups, or other subgroups (sub domains) of interest (e.g., between female and males within strata) a sample size of 30–40 participants per PSU is required, which results in 75 to 100 PSUs selected from the VA Hospitals and 50 to 67 PSUs selected from the CBOCs. The use of 30–40 as minimum number of samples per cluster is based on the Central Limit Theorem (CLT), which postulates that for sample size greater than 30 the sample mean will be approximately normally distributed for most distribution functions. This is relevant since the normality assumption is mostly used for testing hypothesis⁷.

Enrollment:

Subjects will be enrolled in the study using a modified Dillman approach⁸, which includes a pre-notification letter, personalized survey cover letter with pre-paid return envelope, and a thank you/reminder post-card in 4 weeks if no survey is returned (see attached). While the cover letter indicates that participation in the survey is voluntary, it is not feasible for us to obtain verbal or written informed consent from study participants due to the size of the study population (N=5,000) and the national scope of the survey, thus we are requesting a Waiver of Informed Consent and a Waiver of HIPAA Authorization (see waiver request included as part of the protocol amendment submission). Subjects who return a partially or fully completed survey will be considered enrolled responders. Subjects who are mailed a survey, but who do not return a survey will be considered enrolled non-responders. Subjects who return a survey but who mark the following item will be removed from the study population and considered not enrolled.

If you have not received MOVE! Weight Management treatment at a VA medical center or clinic and believe that you may have received this survey in error, please check this box and return the survey in the enclosed envelope.

Survey Instrument:

The survey instrument is a 26-item instrument developed in collaboration with subject matter experts in the areas of survey design, health behaviors including diet and physical activity, patient satisfaction, self-management, weight management, and health literacy. Approximately one-half of the survey items were adopted word-for-word from existing surveys and instruments. This includes the items assessing self-reported diet and physical activity behaviors, neighborhood characteristics, health-related quality of life, and demographic information. The use of existing items with established validity and reliability improves the accuracy with which we can measure these domains. We could not locate suitable existing items to measure patient satisfaction with weight management treatment, some weight management-related behaviors, and MOVE! specific items. Thus, we either designed items to assess these domains, or adapted items from other surveys.

The survey was pilot tested with 8 subjects currently receiving MOVE! treatment at the Durham VAMC as part of the pilot phase of this study. Very minor revisions to improve clarity and readability were made based on this pilot test.

Other Data Proposed for Use:

We propose using the following VHA data to be extracted from existing VA databases to obtain names, mailing addresses, and additional variables to further characterize subjects, assess the degree of non-response bias, and conduct linked analyses with the respondent’s survey data.

Variable	VA Corporate Dataset Name/Identifier	Comments
Scrambled SSN	MDPPRD.MDP.SAS.SE09	FY09, FY10
Age	MDPPRD.MDP.SAS.SE09	FY09, FY10
Gender	MDPPRD.MDP.SAS.SE09	FY09, FY10
Race/ethnicity	MDPPRD.MDP.SAS.SE09	FY09, FY10 May not be completely usable due to a large percentage of missing values.
Names and mailing address	S558TL1.NPC.MOVE.NCPPRGM.FILE1 (most current pt information from NPCD) S558TL1.NPC.MOVE.NCPPRGM.FILE2 (duplicate/additional pt information) S558TL1.NPC.MOVE.NCPPRGM.FILE3 (no information found in NPCD) S558TL1.NPC.MOVE.NCPPRGM.FILE4 (pt information from NPCD of individuals who are deceased)	Names and mailing address
Priority Score	RMTPRD.MED.SAS.ENROLLEE.ENONEPER	FY09, FY10
Comorbidity Score	RMTPRD.MED.DCG.RISK.SMART22.FY08	FY08, FY09
Geography (Urban/Rural)	MDPPRD.MDP.SAS.NED.MAIN.SEP08GEO.PSSG	FY08, FY09
Diabetes Diagnosis	MDPPRD.MDP.SAS.SE09	FY09, FY10
Number of MOVE Visits, including group, individual, or telephone visits.	VHA Service Support Center (VSSC) MOVE Cube encounters	FY09, FY10
Use of the weight loss medications orlistat and sibutramine	VHA Pharmacy Benefits Management	FY09, FY10
Height and Weight (Body Mass Index)	VHA Corporate Data Warehouse	FY09, FY10 for weight FY00-FY 10 for height

In addition to the above patient-level variables, we will also use the VSSC MOVE Cube and the VSSC Uniques cube to obtain the gender distribution for MOVE! treated individuals and for the overall VHA outpatient population in FY09 and FY10 to date.

Survey Collection and Management:

All survey collection and management activities will be handled by Synovate, Inc., the vendor awarded a contract to conduct these activities via solicitation #246-08-RQ-0229. The scope of work involved in this contract is outlined in the contract materials that are attached and are briefly summarized as the following:

• Survey set-up, which involves converting survey to a format and design suitable for use with their optical scanning system

• Production of survey forms and survey-related correspondence including: pre-notification letter and envelope, survey including cover letter and envelope, reply envelope, and reminder/thank you postcard

• Address cleaning and collating though code 1 mail streaming and national change of address (National Change Of Address (NCOA)) cleaning.

• Obtaining postage and mailing of all survey related correspondence

• Survey and whitemail processing

• Data collection using optical scanning or digital image capture techniques

• Data cleaning, quality control, and documentation.

Analyses:

The analytic plan includes descriptive statistics (frequencies, percentages, means, etc.) for each survey item. All analyses will be conducted using appropriate techniques (i.e., weighting) for a multi-stage, stratified cluster sampling survey design to ensure valid estimates.

Weighting class adjustment will be used to compensate for nonresponse. In weighting class adjustment the original sample (respondents and nonrespondents) is divided into mutually exclusive and nonoverlapping adjustment cells. The information or data used to make the assignments of survey participants to adjustment cells needs to be available for both respondents and nonrespondents. Because respondents and non-respondents are patients within the Veterans Health Administration (VHA), socio-demographic and health information is available through existing VHA administrative and clinical data sources.

Post-stratification adjustment will be used to make a further adjustment to sampling weights so that the final weighted sample distribution is as close as possible to that of the target population. This procedure adjusts for differential observed response rates among strata defined by important variables available at the population level. Since we are oversampling women, then the gender distribution of the sample might not resemble the gender distribution of the MOVE treated population. This introduces bias into any estimate one wants to obtain from the sample data because statistical procedures will give greater weight to women. In order to calculate a post-stratification weight, we will use auxiliary data set (e.g., gender distribution) to compare to the sample. We then compare the distribution of the sample with the auxiliary data (e.g., gender distribution). If the distributions are close enough, there is no need to calculate post-stratification weights. However, if they (the distributions) differ by more than a few percentage points (e.g., 2%), then we will need to calculate the post-stratification weights.

Bivariate and multivariate analyses

Because of the cross-sectional nature of the data being collected, we will limit bivariate and multivariate analyses to exploratory analyses to assess the relationship between selected patient characteristics (independent variables) and selected survey responses (dependent variables). As this survey is the first exploration and measurement of patient experiences and behaviors related to MOVE! weight management treatment, we do not have any pre-specified hypotheses that we will be testing. Rather, we will use bivariate (cross tabs/chi square, t-tests) and multivariate (logistic regression modeling) analyses to explore associations between responses on survey items and various patient characteristics, such age, gender, BMI, health status, and number of visits.

Dissemination, Notification, and Reporting of Results.

No individual subject results from this study will be shared, published, or disseminated. Findings from the survey will only be presented in aggregate formats including the following: manuscripts, poster presentations, oral presentations, national VHA conference calls, and briefings to senior VA leadership. The findings of the study will be used to inform policy and programs related to the MOVE! Weight Management Program for Veterans.

REFERENCES

1 Veterans Health Administration. VHA Handbook 1101.1 Managing Overweight/Obesity for Veterans Everywhere (MOVE!) Program. Available at: http://vaww1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1396 . Accessed 10/19/09.

2 VHA Support Service Center. MOVE! Visits Proclarity Data Cube. Available at: http://klfmenu.med.va.gov/products.asp. Accessed 10/19/09.

3 VHA National Leadership Board. Executive Decision Memorandum. FY08 Facility Complexity Level Model. Available at:http://nlb.vssc.med.va.gov/EDMs/Document%20Library/EDM%20FY08%20Facility%20Complexity%20Level%20Model.pdf. Accessed 10/19/09.

4 Lohr, S. (1999). Sampling: Design and analysis. Pacific Grove, CA Duxbury Press.

5 Cochran, W. G. (1977). Sampling techniques. 3rd ed., New York: John Wiley & Sons, Inc.

6 Kish, L. (1965). Survey sampling. New York: John Wiley & Sons, Inc.

7 Levy, P. S. & Lemeshow, S. (1999). Sampling of populations: Methods and applications. New York: John Wiley & Sons, Inc.

8 Dillman DA. 1995.  Survey Methodology.  Dec 1995, Vol 21, No. 2, pp 159-165.

7