Mini Supporting Statement for the HAN Risk Survey

Mini Supporting Statement:

Web-Based Survey to Assess Preferences When Communicating Cancer Risk Estimates

OMB # 0925-0046-09

May 14, 2008

Under Generic Study Titled:

Pretesting/Formative Research for NCI Communications Messages

(OMB No. 0925-0046, Expiration Date 1/31/2010)

Submitted By:

Nina Goodman, MHS

Senior Analyst

Office of Market Research and Evaluation

Office of Communications and Education

National Cancer Institute

6116 Executive Blvd., Suite 400

Rockville, MD  20892-8351

301-435-7789

goodmann@mail.nih.gov

Table of Contents

A. JUSTIFICATION

A.1 Circumstances Making the Collection of Information Necessary 1

A.2 Purpose and Use of the Information Collection 1

A.3 Use of Improved Information Technology and Burden Reduction 2

A.4 Efforts to Identify Duplication and Use of Similar Information 2

A.5 Impact on Small Businesses or Other Small Entities 3

A.6 Consequences of Collecting the Information Less Frequently 3

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 3

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency 3

A.9 Explanation of Any Payment or Gift to Respondents 4

A.10 Assurance of Confidentiality Provided to Respondents 4

A.11 Justification for Sensitive Questions 4

A.12 Estimates of Annualized Burden Hours and Costs 4

A.13 Estimate of Other Total Annual Cost Burden to Respondents

and Record Keepers 5

A.14 Annualized Cost to the Federal Government 5

A.15 Explanation for Program Changes or Adjustments 5

A.16 Plans for Tabulation and Publication and Project Time Schedule 5

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 6

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 6

B. Collections of information employing statistical Methods

B.1 Respondent Universe and Sampling Methods 7

B.2. Procedures for the Collection of Information 7

B.3 Methods to Maximize Response Rates and Deal with Non-response 8

B.4 Test of Procedures or Methods to be Undertaken 8

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 10

A. Justification

A.1 Circumstances Making the Collection of Information Necessary

In order to carry out NCI’s legislative mandate to educate and disseminate information about cancer prevention, detection, diagnosis, and treatment to a wide variety of audiences and organizations (e.g., cancer patients, their families, the general public, health providers, the media, voluntary groups, scientific and medical organizations), it is beneficial for NCI to pretest their communications strategies, concepts, and messages while they are under development.  NCI is currently developing a new online colorectal risk assessment tool for the public. This tool will use risk factor information to calculate an individual’s risk of developing colon cancer, and can be used to inform laypersons about their colon cancer risk via a website.  As part of the pre-test process, and in order to identify the best format and approach to use in this new tool,

NCI proposes conducting a web-based survey with members of the general public. This survey will inform the designers of the tool on how to best communicate uncertainty and increase understanding of individuals’ cancer risk estimates. By conducting formative research on this tool, this study will provide insight into the comprehensibility and usability, while hopefully minimizing confusion, stress, and an inappropriate change in screening behaviors. Current authorization for NCI’s education and information dissemination activities is contained in Section 410 of the Public Health Service Act (42 USC § 285a-2).   

A.2 Purposes and Use of the Information Collection

The objective of this research is to collect formative data via a web-based survey to inform NCI’s development of a new tool, the Colorectal Cancer Risk Assessment Tool (CCRAT), that will help predict an individual’s risk of developing colon cancer. The aim of the current project is to pretest alternative methods of communicating the individualized cancer risk estimates produced by the CCRAT, in order to inform the design of the tool for the general public that is as usable and informative as possible, and that minimizes the likelihood that the public will misinterpret the risk calculations produced by the tool.

A.3 Use of Improved Information Technology and Burden Reduction

As appropriate, automated information technology will be used to collect and process information for this survey to reduce the burden on the public. The survey will be conducted on a dedicated, secure, and encrypted Web site.  People outside of the site will not receive the survey material.  Responses will be collected online. 

A.4 Efforts to Identify Duplication and Use of Similar Information

As this risk tool is in the process of being developed, NCI reviewed existing litera­ture and data bases, including pretesting reports on existing messages and materials, and consulted with outside experts to evaluate available informa­tion on similar tools with comparable audienc­es.  Although other risk models have been developed by NCI, for example for breast cancer (www.cancer.gov/bcrisktool/), the tool is not comparable since different cancers pose different research questions.  Furthermore, there is no existing data to guide the development of the CCRAT tool as comparable risk tools have not been fully evaluated in terms of the methods used to present numerical information and people’s responses to these methods. Since the cancer field is so diverse and complex and there are variations between tools, new data collection instruments are generally prepared to ensure that the questions asked of participants pertain explicitly to the functionality of the innovative tool being presented to them. 

A.5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this survey.

A.6 Consequences of Collecting the Information Less Frequently

Participation will be voluntary and users will be asked to complete this survey only once. The survey is considered appropriate to measure the impact of changes that take place on the Web site.  Without this survey, information will not be able to be adapted as effectively as possible to meet customer needs in the development of this tool.  The survey results will be administered, analyzed, and interpreted as needed by the National Cancer Institute.

A7.  Special Circumstances Related to the Guidelines of 5 CFR 1320.5

This survey will be implemented in a manner that fully complies with 5 C.F.R. 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

A 60‑day Federal Register notice soliciting comments on the “Pre-testing of NCI Communication Messages” was published on August 14, 2006, Volume 71, Number 156, Page 46486.  In response to the notice, there were no public comments.  The 30-Day Federal Register notice was published on October 23, 2006, Volume 71, Number 204, Page 62114.  In response to the notice, there were no public comments.   

In 2008, NCI has been consulting with Tom Lehman, Senior Communications Research Officer, Center for Social Marketing and Behavior Change at the Academy for Educational Development (AED), (phone: 202-884-8863, email: tlehman@aed.org) in order to assist with the design of the survey instrument.  During consultation, there have been no major problems that could not be resolved. 

A.9. Explanation of any Payment or Gift to Respondents

No payment or gift will be provided to survey participants.

A.10. Assurance of Confidentiality Provided to Respondents

Individual respondents will not be identified and participation will be strictly voluntary. Names are not recorded, nor are personal identifying data maintained in the database.  Respondents will be assured that neither their participation/non-participation nor any responses to items will have any effect on their eligibility for, or receipt of, services.

A.11 Justification for Sensitive Questions

No questions will be asked of a personal or sensitive nature.  Participants may choose to withdraw from the survey at any time.

A.12 Estimates of Annualized Burden Hours and Costs

There will be a maximum of 200 respondents with an average, pilot-tested response time of 12 minutes to complete the survey, which culminates in a total annual hour burden of 40 hours (Table A.12-1).

Table A.12-1  Estimates of Hour Burden
Types of Respondents	Number of Respondents	Frequency of Response	Average Response Time	Annual Hour Burden
General Public	200	1	12/60	40

The annualized cost to the respondents is estimated to be at $680.00 (Table A.12-2).

Table A.12-2  Annualized Cost to Respondents
Types of Respondents	Number of Respondents	Frequency of Response	Hourly Wage Rate	Respondent Cost
General Public	200	1	$17.00	$680.00

A.13. Estimate of Other Total Annual Cost Burden to Respondents and

Record Keepers

There are no capital costs, operating costs, and/or maintenance costs to report.

A.14. Estimates of Costs to the Federal Government

This current survey will have minimal impact on the total original cost that was estimated in the full supporting statement.

A.15. Explanation for Program Changes or Adjustments

This is a new survey under the existing generic study titled “Pretesting of NCI Communication Messages” (OMB #0925-0046).  This represents the ninth sub-survey in this generic study and, once approved, will be indicated by 0925-0046-09.

A.16. Plans for Tabulation and Publication and Project Time Schedule

If OMB approval is secured, the survey will be administered within 2-4 weeks.  We anticipate that it will take no more than 1 month to collect information from a maximum of 200 participants.  Results for this survey will be tabulated within 2-4 weeks after the completion of the survey.  Results of selected findings may be published in refereed journals and other publications within a timely fashion in order to contribute to the academic science literature.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

This survey will display the OMB expiration date.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

These surveys will comply with the requirements in 5 CFR 1320.9.  No exceptions to certification are requested.

B. Collections of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods

For this formative research, NCI will be conducting a web-based survey with members of the general public, ages 40 and older, with no family history of colon cancer; as these characteristics define the potential target population for the CCRAT. Potential participants will include both men and women residing in geographic regions across the United States, and representing all racial, ethnic, and educational backgrounds.

Because the purpose of the current survey is to collect information for use in formative pretesting of the CCRAT, the sampling methods will focus on obtaining, with the greatest efficiency possible, a sample of respondents that is diverse though not necessarily nationally-representative, and that has demonstrated proficiency in using the web. To this end, the survey will be conducted among members of a profesionally-managed web-based survey panel. Panel members have experience conducting online surveys, have already indicated an expressed interest in completing web-based surveys, and have explicitly agreed to participate in surveys presented to them about issues both unrelated and related to health. Panel members eligible to participate in the current survey will be contacted through an e-mail invitation from the panel managers which will include a secure, non-identifiable link to the web-based survey. Recruitment will continue until the target sample size for completed surveys is reached.

B.2. Procedures for the Collection of Information

Participants will be asked to answer a brief, self-administered, interactive web-based survey which shows different simulated outputs from the hypothetical colon cancer risk prediction tool. Participants will see one of four alternative outputs which vary according to 1) the message formats used to communicate uncertainty about the accuracy of individualized cancer risk estimates, and 2) the type of information given to enhance understanding of this uncertainty and of the meaning of risk estimates. The surveys will examine participants’ reactions to these alternative outputs from the hypothetical colon cancer risk prediction tool. NCI will use the results of the survey to inform the development of the CCRAT tool.

B.3 Methods to Maximize Response Rates and Deal with Non-response

For a full discussion about respondent recruitment pertaining to this generic sub-study, refer to Section B.1. For the survey, potential participants have previously “opted-in” to be members of a survey panel, as explained earlier; this will facilitate successful and efficient recruitment of participants, but also makes it difficult to obtain a sample representative of the broader population and to calculate true response rates. However, because this sub-study is formative in nature – to assess which is the best way to present risk data on a future tool – this recruitment strategy will suffice to achieve the primary goal of obtaining an adequate number of participants to allow meaningful comparisons of their reactions to the alternative outputs of the CCRAT. Non-response will be dealt with by continued e-mail recruitment of potential participants until the target sample size is reached.

B.4 Test of Procedures or Methods to be Undertaken

            The formative testing methodologies planned for this project represent the standard state-of-the-art approaches adapted from marketing and com­munica­tions research.  For this formative research , a self-administered web-based survey will be used, as it is a methodology frequently used by NCI to pretest drafts of NCI concepts and materi­als that is both reliable and efficient.  This survey will be accessed on-line at a designated Internet location.

The objective of the survey is to help NCI to determine how best to present individualized cancer risk information to the public, so that this information is clearly understood, comprehensive, useful, and relevant. Specifically, this formative research will focus on the question of how individual colon cancer risk estimates from the CCRAT should be presented to laypersons. Past research, including our own previous qualitative studies, indicate that message formats can dramatically affect how people understand their risk calculations as well as their attitudes and perceptions of risk. The proposed research in this OMB package is the first formative research effort to explore this issue on a larger scale (i.e., investigated with statistically large enough samples to identify message formats’ impact on respondents perception of risk).

This survey will focus on evaluating two distinct types of information that have been shown to be of potential importance in people’s responses to risk estimates, and that might be communicated by the CCRAT: 1) information on the uncertainty of a risk estimate as communicated by a confidence interval, or range of values likely to include the actual risk (e.g., “5 to 13%”); and 2) information on how a risk estimate compares to that of an average person (e.g., “higher than average”). Gaining insight into how people understand and respond to these two types of information in this context, will directly inform how the CCRAT output display will present risk estimate information to online users.

It would be impractical, and overly burdensome given the complexity of information, to pretest all possible combinations of this information in the same group of individual participants. To minimize response burden and reliably assess participants’ responses, participants will therefore be randomly assigned to one of four groups (n=50/group), which will differ according to whether or not these two types of information—on both statistical uncertainty (yes/no) and comparative risk (yes/no)—are provided to the participant. More simply put, each of the four groups will be shown a different option for how the CCRAT output display might communicate information about an individual’s colon cancer risk. The varied options are as follows:

• Group 1: No information on statistical uncertainty / No information on comparative risk

• Group 2: Information on statistical uncertainty / No information on comparative risk

• Group 3: No information on statistical uncertainty / Information on comparative risk

• Group 4: Information on statistical uncertainty / Information on comparative risk

The total sample size of 200 (50 respondents/group) represents the minimum number of participants required for reliable inferences to be drawn to guide NCI’s development of the CCRAT for public use.

B.5 Individuals Consulted on Statistical Aspects and Individuals

Collecting and/or Analyzing Data

     

Responsibility for collecting and analyzing information obtained from this survey will rest with Paul Han, MD, MA, MPH, Division of Cancer Control and Population Sciences (DCCPS), National Cancer Institute (NCI) and Nina Goodman, MHS, Office of Communications and Education (OCE), NCI. All data collec­tion and analysis will be performed in compliance with OMB, Privacy Act, and Protection of Human Subjects requirements.

2