CMEC Memo to OMB for Frame Work and Cog Interviews

MEMORANDUM
TO:

Robert Sivinski
Office of Statistical and Science Policy
Office of Management and Budget

THROUGH:

Jeffrey H. Anderson
Director
Bureau of Justice Statistics

FROM:

Connor Brooks
Statistician, Law Enforcement Statistics Unit
Bureau of Justice Statistics

DATE:

September 7, 2018

SUBJECT:

BJS request for Generic Clearance to conduct: 1) outreach to
identify/confirm missing reporting units for the 2018 Census of
Medical Examiner and Coroner Offices (CMEC) and 2) cognitive
interviews to finalize CMEC survey instrument, under the generic
clearance agreement OMB Number 1121-0339

Introduction
In 2019, the Bureau of Justice Statistics (BJS) is planning to conduct the Census of Medical
Examiner and Coroner Offices (CMEC), which was last conducted in 2005. Thirteen years later,
BJS’s final report 1 remains the only comprehensive source of basic data about the U.S.
medicolegal death investigation system. BJS’s landmark report made clear that medical
examiners and coroners varied widely across all measures (e.g., jurisdiction size and type,
caseload, staffing, procedures performed, record retention, use of national databases, operation
and budget). The purpose of this iteration of the CMEC is to survey the universe of
approximately 2,150 medical examiners and coroners who conduct medicolegal death
1

Hickman, M. J., Hughes, K. A., Strom, K. J., & Ropero-Miller, J. D. (2007). Medical examiners and coroners’
offices, 2004. Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice, Washington, DC.

1

investigations in the United States with the goals of: 1) generating statistics that will help BJS
develop a detailed understanding the of U.S. medicolegal death investigation system; 2)
gathering information that will help address training, staffing, or jurisdictional coverage needs;
and 3) further developing an understanding of the relationship between law enforcement
agencies and medical examiner and coroner offices (MECs).
This generic clearance is to request approval to conduct three activities in preparation for the
2019 administration of the CMEC under BJS’s generic clearance agreement (OMB Number
1121-0339). First, a frame verification effort with telephone outreach is needed to confirm or
collect contact information for a portion of the MEC frame. Second, cognitive interviews need to
be conducted to test the survey instrument for clarity and interpretation of the questions and
answer options. More details about each of these requests follow.
Request for Frame Verification Telephone Outreach Effort
During the 2017 National Forensic Laboratory Information Management System (NFLIS)
Medical Examiners and Coroners Survey funded by the Drug Enforcement Administration
(DEA; OMB Clearance # 1117-0034, Expiration Date: May 31, 2019), 2,156 MECs were
enumerated by RTI International. During this effort, 268 MECs (12% of the total) were “absolute
nonresponders” in that the NFLIS study team was unable to complete verification calls,
prompting calls, or nonresponse follow-up calls, and never received a survey response. Thus,
these 268 reporting units constitute a batch of MECs for which it is not clear if the most current
and appropriate contact or address information is on record.
BJS has permission from DEA to use the NFLIS MEC frame for fielding the CMEC, but this
frame file will lack verified information for these 268 nonresponding units. BJS requests
clearance from OMB to collect or confirm contact information and addresses from these 268
units. As part of obtaining this information, the project team will conduct Internet searches and
search for these offices on professional lists (e.g., National Association of Medical Examiners
and the International Association of Coroners & Medical Examiners). It is anticipated that
through the Internet searches and list comparisons, we may discover approximately 50 new
agencies to add to the frame based on the NFLIS project team’s experience in post data
collection cleaning of the MEC universe. Thus, we anticipate about 325 reporting units will
require verification calling efforts. We aim to conduct this verification call effort in January and
February of 2019.
Appendix A provides the list of the proposed ten verification questions. The verification call
effort will involve project staff calling the MECs to confirm information currently on record with
respect to the MEC official name, the MEC office address, and the name and contact information
for the coroner or medical examiner. The verification call effort will also help identify a survey
point of contact given the survey topics and will request their contact information. The last three
questions are designed to ascertain study eligibility. For example, MECs that do not conduct
medicolegal death investigations, sign death certificates, or determine when autopsies should be
conducted are likely out of the scope for the 2018 CMEC and therefore should not be further
burdened. The results of these calls will be documented and uploaded into the case management
system to be used for all future communications with eligible MECs.
2

Burden Hours for the Frame Verification Telephone Outreach Effort
Table 1 provides a summary of the burden estimate for this effort. The project team will contact
up to 325 MECs via telephone. It is anticipated that respondents will need an average of 5
minutes to identify an appropriate person within the office to answer the questions if the
gatekeeper is unable to answer the questions. This estimate includes time necessary to call
respondents back if they indicate they are not available at the time, and we need to call back. For
gatekeepers who are able to answer the questions, the calls should last about 5 minutes.
Table 1. Burden Estimate for the Frame Verification Telephone Outreach Effort
Task

Average burden per
MEC

Identify appropriate respondent for
verification questions

5 minutes

Confirm information

5 minutes

Total estimated burden
hours
325 respondents x 5
minutes = 27.1 hours
325 respondents x 5
minutes = 27.1 hours

Total respondent burden for all respondents = 54.2 hours
Request for Cognitive Interviewing
BJS also requests clearance to conduct up to two rounds of cognitive testing of the 2018 CMEC
instrument. The project team will initially conduct 18 cognitive interviews over the telephone. In
prelude to this effort, the project team reviewed the 2004 CMEC instrument for completeness
and clarity. The project team also conducted a data quality assessment with the publicly available
2004 CMEC dataset to identify potentially problematic questions based on item response.
Questions with high item missingness were identified as candidates for deletion or revision. In
addition, an expert panel provided direction on survey revisions and provided input to make the
questionnaire content relevant to the 2018 needs and demands of the MECs. All of those efforts
have resulted in a draft 2018 CMEC instrument that is ready for cognitive testing.
The cognitive interviewing plan is designed to assess the survey instrument in terms of general
understanding, question and response wording, and survey design, all of which will help
minimize survey burden in the long run. The goal of this effort is to understand how well the
questions work when administered to a purposive subset of the survey’s target population, which
will include an even mix of coroners and medical examiners with representation across large and
small jurisdictions, and across geographical regions, including jurisdictions that border tribal
areas.
BJS and the project team will purposively select nine coroners and nine medical examiner offices
that are diverse across a set of primary characteristics, including 1) type of office: medical
examiner vs. coroner, 2) jurisdiction size: less than 25k population served, 25k to <250k, and
250k or more based on the most recent Census population estimate, and 3) location of office
based on Census region: West, Midwest, Northeast, and South. For medical examiners, given the
wide variety of organizational hierarchies represented by this population, the next characteristic
3

of interest includes whether the office serves state, regional, or county level jurisdictions. For
coroner offices, where possible, the project team will identify a diverse blend of coroners that fit
the above primary characteristics, but may represent the various professions of coroners, such as
morticians, funeral parlor directors, forensic nurses, and medical doctors who are not forensic
pathologists (e.g., general practice doctors who serve as the county coroner).
Each of these characteristics is important to be represented among the cognitive testing panel
because they reflect the diverse perspectives needed to ensure that the survey will resonate with
and be understood by a diverse field and capture the intended data. If the project team encounters
any refusals among the initial purposive sample, a substitute cognitive interview respondent will
be identified that fits the general profile of the refusal across the primary medical examiner or
coroner characteristics (e.g., a refusal coroner from the Midwest representing a medium-sized
city/county would be replaced by a respondent with the same characteristics).
Using the texts included in Appendix B, the project team will contact the prospective MEC
cognitive interview respondents to invite them to participate in 120-minute telephone cognitive
interviews. Most of the invitations will be delivered via email (see email template in Appendix
B-1) but there may be a handful of participants for whom we will need to send invitations
through the postal service if email addresses are not available (see letter template in Appendix
B-2). Once these invitations are sent, the project team will follow up by phone to confirm
willingness to participate and schedule the cognitive interview (see telephone script in Appendix
B-3). Upon receiving confirmation of the office’s willingness to participate in the cognitive
testing, we will mail or email them a packet including the consent form (Appendix C-1) and the
CMEC questionnaire (Appendix D). Because we are interested in how respondents understand
and interpret questions rather than the actual answers to the questions, respondents will not have
to complete the questionnaire and return it.
Participation in the cognitive test will be completely voluntary. All participants will be 18 years
of age or older. Prior to the scheduled interview, the participants will be provided information
that describes the interview, why they were chosen, what will happen during the discussion, the
risks and benefits of participation, and details ensuring confidentiality (see Appendix C-1). At
the time of the scheduled interview, staff will review these materials, including the consent form
with the respondents. They will be given the option to refuse to answer a question at any time
during the interview. Since these are telephone interviews, the interviewer will read the consent
form to the participant and obtain verbal consent. After receiving verbal consent, the interviewer
will check and sign the hard copy informed consent to have on record, certifying they have
obtained the participant’s permission to continue with the interview (see Appendix C-2).
Interviews will only be conducted if consent is obtained. If a participant refuses, they will be
thanked for their time and consideration.
If the participant consents to audio recording, the interviews will be recorded to allow for
accurate capture of responses and account of answers during the analysis stage. Audio recordings
will be saved on the project team’s secured project shared drive and will be deleted once the
survey is finalized. Audio recordings will not be sent electronically or transferred outside of RTI
to reduce the risk of an unauthorized party intercepting the recordings. A thank you letter will be

4

sent to participants within 48 hours of the completion of the cognitive interview (see Appendix
B-4).
The survey topics in the questionnaire include:
-

Section A: Administrative including type of office, jurisdiction served, and
accreditation
Section B: Budget and capital resources including annual budget, budget categories,
and funding sources
Section C: Workload (death investigations) including number of referrals, and
accepted cases
Section D: Specialized death investigations including procedures for unidentified
remains, infant death, toxicology, and opioids
Section E: Records and evidence retention including computer records systems and
archiving
Section F: Resources and Operations including internet access, access to databases,
training resources, and technologies

The testing protocol and example probes are presented in Appendix E. The respondents will see
only the formatted survey and not the probing questions. Each cognitive interview, including
probes, will take a maximum of 120 minutes. The cognitive testing protocol will introduce the
participants to BJS’s 2018 CMEC goals (see Appendix F for the study show card), and guide
participants through the questionnaire.
As respondents answer the questions, the interviewers will ask participants to note any aspects of
the instruments that are unclear, any questions or topics omitted, or any answer choices or
response categories missing or insufficient, or questions that should be struck or revised.
Interviewers will also ask questions to check respondents’ understanding of the questions.
During this cognitive testing phase, we will instruct participants to answer only those questions
which they can readily answer; participants are not expected to conduct research to answer any
questions. For those questions that are not readily answered without researching information, we
will ask participants how they would find the information necessary to answer the question and
who would be the person best suited to answer the question. To gauge burden, we will also ask
participants how long they think it would take them to complete the form on their own, including
any time necessary for research or involving other staff members.
Participants will not receive any compensation for the interview, but will be thanked for their
time and insights. The project team will review the feedback from the cognitive interviews and
revise the survey instruments as necessary.
The full 2018 draft CMEC instrument is provided in Appendix D. Cognitive interviews will take
place in September and October 2018. All information gathered from the CMEC cognitive
testing efforts will be integrated into the full information clearance package that is expected to be
submitted to OMB in December 2018.

5

If respondents’ feedback consistently indicate that certain questions are problematic, BJS and
RTI plan to conduct an additional round of cognitive testing with the revised questions with up to
12 respondents. Selection of these offices will follow the same procedure as the first round of
cognitive interviewing.
Burden Hours for the Cognitive Interviews
The burden hour estimates are provided in the following table.
Table 2. Burden Estimate for the First Round of Cognitive Interviews
Task
Initial contact and scheduling
Complete cognitive interview
including questionnaire completion

Average burden per
MEC
10 minutes
120 minutes

Total estimated burden
hours
18 respondents x 10
minutes = 3 hours
18 respondents x 120
minutes = 36 hours

Total respondent burden for all respondents = 39 hours
Table 3. Burden Estimate for the Second Round of Cognitive Interviews (if needed)
Task
Initial contact and scheduling
Complete cognitive interview
including questionnaire completion

Average burden per
MEC
10 minutes
120 minutes

Total estimated burden
hours
12 respondents x 10
minutes = 2 hours
12 respondents x 120
minutes = 24 hours

Total respondent burden for all respondents = 26 hours
Institutional Review Board
The project team has obtained approval from RTI’s IRB to ensure the testing protocols are compliant
with informed consent and data confidentiality standards (Appendix G).

Contact Information
Questions regarding any aspect of this project can be directed to:
Connor Brooks
CMEC Program Manager
Bureau of Justice Statistics
Office of Justice Programs
U.S. Department of Justice
810 7th Street NW
Washington, DC 20531
6

Office Phone: 202-514-8633
E-Mail: Connor.Brooks@usdoj.gov

Appendix A:
Appendix B:
Appendix C:
Appendix D:
Appendix E:
Appendix F:
Appendix G:

2018 Census of Medical Examiner and Coroner Offices Verification Call
Script
2018 Census of Medical Examiner and Coroner Offices Cognitive Testing
Recruitment Materials
2018 Census of Medical Examiner and Coroner Offices Cognitive Testing
Informed Consent
2018 Census of Medical Examiner and Coroner Offices Instrument
2018 Census of Medical Examiner and Coroner Offices Cognitive
Interviewing Protocol
BJS’s 2018 Census of Medical Examiner and Coroner Offices Goals
Showcard
IRB Submission Confirmation

7