CMS-10621 MIPS Data Validation Electronic Survey

Merit-Based Incentive Payment System (MIPS) Data Assessment, Accuracy and Improper Payments Identification Support Contract

Provided in Paperwork Reduction Act Package as Appendix H and a placeholder for MIPS Data Validation Survey, which will be developed and shared in a future MIPS PRA package.

Sponsored by:

U.S. Department of Health and Human Services,

Centers for Medicare & Medicaid Services

Public Burden Statement: According to the Paperwork Reduction Act of 1995, a federal agency may not conduct, and a person is not required to respond to, an information collection request unless it displays a currently valid OMB control number. The valid OMB control number for this information collection is [XXXX-XXXX]. The time required to complete this information collection is estimated to average 15 minutes per respondent, including the time to review instructions and complete and review the information collection. If you have comments concerning the accuracy of this burden estimate or any suggestions for reducing this burden, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, MD 21244-1850.

OMB No.: [xxxx-xxxx]

Expires: [3 yrs from OMB approval date]

Participation in this Survey

Your participation is requested on an important survey. As a CMS Web Interface/Registry that submits data to the Centers for Medicare & Medicaid Services (CMS) as part of the Merit-Based Incentive Payment System (MIPS) program, we require your assistance in evaluating features of this program. This survey consists of 43 CMS Web Interface questions and 44 Registry questions regarding how the CMS Web Interface and Registries collect, validate, and submit data that are reported through the MIPS program. The questions will guide you through a systematic assessment of your data handling practices and help you identify opportunities for improvement and more accurate future reporting. The survey includes both an online and an interview component. We estimate that participation can be accomplished in less than 2 hours, inclusive of minimal time for preparation and follow up.

This is a voluntary survey. It is designed to measure the (1) data handling by CMS Web Interface (or Registries) and CMS and its contractors, (2) clinical quality measures results accuracy, and (3) payment error. Information from the CMS Web Interface and Registries will help formulate MIPS program integrity features that could result in clearer information shared by CMS for future performance periods. We are genuinely interested in your candid observations of the way your program operates today. If you are willing to participate in this activity, please complete this survey by _____, 201x. If you have difficulty or questions when completing this survey, please contact ____________________ or xxx-xxx-xxxx.

The Questions in this Survey

This survey asks how your organization currently manages data errors and, inconsistencies that are due to data handling, program requirements or changes in measures or specifications. The questions are organized into six sections:

Section 1: Corporate Information asks you to identify demographic and organizational characteristics..

Section 2: Training asks what knowledge, skills, behavior or other learned capabilities are required of your staff or presented in structured training programs.

Section 3: Data handling asks you to describes your process for the transfer of data from one system to another, such as from the provider or the intermediary to CMS, or from the provider to the intermediary.

Section 4: Quality Assurance asks you to describe and comment on your quality assurance processes and your approach to improving quality.

Section 5: Feedback: This section was created for you to provide feedback on the program to CMS. Use this section to enter evaluative or corrective comments.

CMS Web Interface Survey

The questions for the CMS Web Interface Survey appear in Table 1.

CMS Web Interface Survey Questions
Item #	Question	Question Type
Corporate Information
	Name	Text input
	Practice or Company Name	Auto populate
	Address	Auto populate
	Telephone	Auto populate
	E-mail address	Auto populate
	CMS Web Interface name	Auto populate
	CMS Web Interface TIN	Auto populate with Last 4 digits of the TIN only
	Number of eligible clinicians?	Drop down list: 25-99 Eligible Clinicians >100 Eligible Clinicians
Training
	Do you have education and credentialing requirements for staff that perform chart abstraction?	Yes or No.
	How do you train staff to understand measures and perform chart abstraction?	Check all that apply: Vendor-supplied training. Group or classroom setting. One-on-one. Virtual class room. Individual online training. If selecting virtual classroom or individual online training, specify name of training program and URL. Other, specify __ Do not provide training.
	Do you have written processes and procedures for chart abstraction training?	Yes or No. If Yes, please upload a copy of your training policy and procedure document.
	During the training, how do you validate that the chart abstraction done by the trainee is accurate?	Check all that apply: Practice cases performed during training. Abstractor over-reads (i.e., re-review of abstracted data) in the training environment. What percentage of over-reads are performed (numeric input)? What is your organization’s acceptable error rate (numeric input)? How are records selected for over-reads (numeric input)? Other, specify ___ None
	Do you have written process and procedures for data conversion (e.g. converting data abstracted from paper medical record to CMS Web Interface) training?	Yes or No. If Yes, please upload a copy of your training policy and procedure document.
	Do you have written processes and procedures for data reporting (e.g. populating the CMS Web Interface) training?	Yes or No. If Yes, please upload a copy of your training policy and procedure document.
Data Handling
	Do you import the beneficiary list from the CMS Web Interface to assist with chart abstraction?	Yes or No. If yes, Do you convert the XML file to excel? Yes or No.
	Do you verify that the beneficiary assignment matches your information?	Yes or No.
	What are your methods of chart abstraction?	Check all that apply: Manual paper chart abstraction Automated EHR query Web portal Medical Record Abstraction Tool Database Other, specify
	How many information systems do you need to access for chart abstraction (e.g., EHRs, paper charts, lab systems, registries)?	Insert numeric input ___
	Do you use a tool to collect and store abstracted data prior to entering it into the XML file and/or CMS Web Interface? a) Electronic tool name (or vendor) b) Does the tool do validations and calculate performance? c) For NO, please select how you input abstracted data?	Yes or No. If Yes, continue to question 22a & 22b. If No, continue to: a)Tool name or vendor __________________. b)Yes or No c)Check all that apply Electronically import into XML file Manually key in data into XML file Manually input data into the Web Interface Other
	Do you exclude members for any reason other than what is in the technical specifications?	Yes or No. If Yes, explain by measure your reason for excluding members.
	Do you manually enter or upload data into the CMS Web Interface?	Drop down list Manually enter Upload data Both
	If you encounter errors uploading data to the CMS Web Interface, what type are they?	Input types of errors.
	Have you enabled the data validation Errors and Warning in the CMS Web Interface?	Yes or No
	Do you encounter data validation Errors and Warning in the CMS Web Interface?	Check all that apply: Populate missing values Correct reporting period Investigate inconsistencies between similar elements Other, please specify
	If you have not enabled the data validation Errors and Warning in the CMS Web Interface, then how do you resolve the error?	Please describe
	Are there measures that consistently have Errors and Warning messages in the CMS Web Interface?	Yes or No If yes, please describe the type of Error and Warnings messages associated with these measures
Quality Assurance
	Do you have written processes and procedures to ensure that accurate data is abstracted? If Yes, is the process followed consistently?	Yes or No. Yes or No. If Yes, please upload a copy of your policy and procedure document.
	Do you have written processes and procedures to ensure that accurate data is uploaded into the CMS Web Interface? If, Yes, is the process followed consistently?	Yes or No. Yes or No. If Yes, please upload a copy of your policy and procedure document.
	Do you have written processes and procedures to ensure that accurate data is reported to CMS? If, Yes, is the process followed consistently?	Yes or No. If Yes, please upload a copy of your policy and procedure document.
	Is there someone at your practice supporting quality assurance to ensure that the data is validated before transmission to CMS?	Yes or No. If Yes, what is that person’s position in your practice?
	Do you provide any oversight for chart abstraction done in the field?	Check all that apply: Abstractor over-reads (i.e., re-review of abstracted data) in the field. What percentage of over-reads are performed (numeric input)? What is your organization’s acceptable error rate (numeric input)? How are records selected for over-reads (numeric input)? Other, specify ___ None.
	Describe how you provide technical assistance and/or train staff on how to respond to questions that arise during chart abstraction.	Check all that apply: One-on-one consultation Ask a supervisor or super-user Online training manual Help line Other, specify ___ None
CMS Web Interface Feedback
	What do you perceive to be the biggest challenge in reporting accurate and complete data to CMS?	Fill in the blank, or upload document.
	Describe any process improvement activities used to increase your organization’s reporting accuracy.	Fill in the blank.
	Describe the issues you have observed with the feedback reports	Fill in the blank.
	Do you find the informal review process beneficial for resolving any issues? If no, describe the difficulties.	Fill in the blank.
	Describe the issues and challenges you have faced with the CMS Web Interface.	Fill in the blank.
	Which measures were the most problematic for your chart abstractors?	Check all that apply: List measures in drop down list.

Table 1: CMS Web Interface Survey Questions

Registry Survey

The questions for the Registry Survey appear in Table 2.

Item #		Question		Question Type
Registry Information
		Name		Text input
		Practice or Company Name		Auto populate
		Address		Auto populate
		Telephone		Auto populate
		E-mail address		Auto populate
		Registry name		Auto populate
		Registry TIN		Auto populate with Last 4 digits of the TIN only
		Number of eligible clinicians reporting for MIPS?		Numeric input.
Training Information
		How do you train staff to understand clinical measure specifications?		Check all that apply: Group or classroom setting. One-on-one. Virtual class room. Individual online training. If selecting virtual classroom or individual online training, specify name of training program and URL. Other, specify __ Do not train staff.
		How do you train staff to understand calculated results?		Please describe process or upload document
		How do you train staff to transfer data to XML		Please describe process or upload document
		Do you have written processes and procedures for training staff to support eligible clinicians’ who submit data to your registry?		Yes or No. If Yes, please upload a copy of your policy and procedure document.
		Do you have written process and procedures for calculating performance results using the MIPS measure specifications?		Yes or No. If Yes, please upload a copy of your training policy and procedure document.
		Do you have written processes and procedures for transferring results to XML?		Yes or No. If Yes, please upload a copy of your training policy and procedure document.
		What type of education do you provide to eligible clinicians on the clinical measure specifications?		Check all that apply: Group or classroom setting. One-on-one. Virtual class room. Individual online training. If selecting virtual classroom or individual online training, specify name of training program and URL. Other, specify __ None
Data Handling
		What type of tool do you use to collect data from eligible clinicians?		Check all that apply: CMS Web Interface XML Direct data query EHR Other, specify ___
		Do you use a tool to do validation checks and display performance for the eligible clinician to confirm that the information is accurate? if no, answer question 20.		Yes or No.
		How do you perform validation checks and display performance for the eligible clinician to confirm that the information is accurate?		Please describe your process
		Did your registry successfully pass use cases for all measures?		Yes or No. If No, please describe steps taken to address use case errors.
		Did your organization use the 2012 SEVT tool to test XML files?		Yes or No.
		Are members excluded for any reason other than what is in the technical specifications?		Yes or No. If Yes, explain by measure your reason for excluding members.
		How many different file formats do eligible clinicians use to submit data?		Insert numeric input ___
		What is the ETL process to aggregate patient level data into XML?		Fill in blank and upload data.
		Do you analyze and update the XML specification changes for each program year?		Yes or No. If yes, upload documentation that describes that process.
		Do you analyze and update the measure specification changes for each program year?		Yes or No. If yes, upload documentation that describes that process.
Quality Assurance
		Do you validate accurate data is collected from eligible clinicians? If Yes, is the process followed consistently?		Yes or No. Yes or No. If Yes, please upload a copy of your policy and procedure document.
		Do you validate that results are calculated correctly using the correct measure specifications? If Yes, is the process followed consistently?		Yes or No. If Yes, please upload a copy of your policy and procedure document.
		Do you validate that results are accurately transferred to XML? If Yes, is the process followed consistently?		Yes or No. If Yes, please upload a copy of your policy and procedure document.
		Do you validate to identify anomalies before you submit your data (e.g. numerator is greater than denominator)? If Yes, is the process followed consistently?		Yes or No. If Yes, please upload a copy of your policy and procedure document.
		Is there someone at your organization who analyzes XML changes?		Yes or No. If Yes, what is that person’s position in your practice?
		Is there someone at your organization supporting quality assurance to ensure that the data is validated before transmission to CMS?		Yes or No. If Yes, what is that person’s position in your practice?
		Do you provide any oversight for calculating and maintaining measure specifications and algorithms?		Yes or No If Yes, please describe that process.
		How do you provide technical assistance and/or train staff on how to respond to questions that arise on measure specifications, measure calculations, and converting results to XML format.		Check all that apply: One-on-one consultation Ask a supervisor or super-user Online training manual Help line Other, specify ___ None.
	Registry Feedback
			What do you perceive to be the biggest challenge in reporting accurate and complete data to CMS?		Fill in the blank, or upload document.
			Describe any process improvement activities used to increase your organization’s reporting accuracy.		Fill in the blank.
			Describe the issues you have observed with the feedback reports.		Fill in the blank.
			Do you find the informal review process beneficial for resolving any issues? If no, describe the difficulties.		Fill in the blank.
			Describe the issues and challenges you have faced transferring data to XML.		Fill in the blank.
			Which measures did eligible clinicians find the most difficult to understand?		Fill in the blank.
			Which measures did staff find the most difficult to program?		Fill in the blank.

Table 2: Registry Survey Questions

17