#15_Justif Memo_Accrual CT_CTEP

Date: May 2, 2011

To: Office of Management and Budget (OMB)

Through: Mary Forbes, Report Clearance Officer, HHS

Seleda Perryman, Program Officer, OPERA, NIH

Vivian Horovitch-Kelley, PRA OMB Project Clearance Liaison, OMAA, NCI

From: Holly A. Massett, PhD

Associate Director, Office of Market Research and Evaluation

Office of Communications and Education, National Cancer Institute (NCI/NIH)

Nina Goodman, Project Officer

Office of Communications and Education (OCE),

National Cancer Institute (NCI)/NIH

Subject: Generic Sub-study, Rapid Feedback Tool to Identify Accrual Problems with Active NCI clinical Trials under “Formative Research, Pretesting, and Customer Satisfaction of NCI’s Office of Communications and Education,”

(OMB No. 0925-0046-15, Expiry Date 02/28/2013).

Background/Need and Use of Information

This information collection request described in this memo supports the National Cancer Institute (NCI) Clinical Investigations Branch of the Cancer Therapy and Evaluation Program (CTEP) in the Division of Cancer Treatment and Diagnosis (DCTD) and is a collaborative effort between CTEP and the NCI Office of Communications and Education (OCE). Low accrual to oncology clinical trials persists, with over 40% of NCI-sponsored trials failing to achieve minimum accrual goals. As of 2004, NCI has implemented an institute-wide effort to improve the operational efficiency of its clinical trial enterprise system. Recently, efforts have focused on improving the “running” of trials regardless of their ability to accrue, with many trials remaining open despite abysmal accrual rates. NCI is interested in identifying problems with these trials to determine if they are reparable or if they should be terminated. Getting the expert opinions of researchers in the oncology field (both oncologists and their staff) who accrue over 60% of patients to NCI’s trials is critical; yet, NCI has few mechanisms to seek their feedback and remain mostly connected to the academic researchers who develop the trial concepts but rarely accrue.

OCE has recently developed two brief online feedback tools to explore why a trial is poorly accruing. Each tool is a template survey that can be tailored to a trial with one version sent to oncologists (to ask about the trial’s science) and the other to research staff (to ask about a trial’s feasibility). OCE would like to pilot these tools for one year with two trials a month (up to 24 trials) to assess the degree to which the tools aid CTEP in making decisions about whether to keep a low accruing trial open or to close it. Currently, no other mechanisms exist to solicit this type of feedback from oncology researchers in the field, so their perspective is not represented.

In order to assess if the pool of questions on the templates were appropriate, OCE tailored the tools to a trial that CTEP identified as low accruing. OCE then worked with one community program willing to send invitations out to 4 oncologists and 5 staff persons (total 9) who were willing to fill out the surveys and then participate in a debriefing session afterwards. The tools were further refined based on this feedback. One overriding comment from these 9 individuals was the excitement and support of such feedback tools for future trials. One oncologist noted: “It’s good that NCI is asking our opinion about these trials.” Additionally, it took only an average of five (5) minutes to complete each survey.

The request put forth in this memo is to determine if such opinions can be collected systematically in an efficient and rapid manner to better guide NCI on its trial portfolio decisions. It is understood that the data are qualitative in nature and designed to guide decisions, but not be the sole or primary determinant of such decisions.

Participants

NCI’s Cancer Trials Support Unit (CTSU) tracks all cooperative group sites that receive IRB approval to open each of its trials. For each site, a primary Principal Investigator is listed as well as his/her contact information; the PI typically represents 8-10 sub-sites (i.e., the primary site is the parent site to several satellite sites). The number of PIs listed for each trial ranges from 50 into the hundreds. For each low-accruing trial selected to pilot these two tools, OCE will randomly select up to 50 primary PIs of those that opened the trial. Each PI will be sent an invitational email with the oncology survey link and the staff survey link and asked to send them to an oncologist and research staff member on their team (across their satellite sites) who they believe can best respond to the trial’s accrual concerns. Due to no known similar tools used for such feedback among oncology teams, it is unclear the response rate to be expected. That said, there is evidence that the field is highly motivated to provide accrual feedback to NCI. Additionally, the brevity and anonymity of the survey supports the assumption of a 75% response rate, amounting to 38 completions for each tool, or a total of 76 participants for each trial. Over the course of one year, and assuming two trials a month, the assumption is 1,824 participants completing the surveys (i.e., 152 burden hours total).

Given the nature of the online survey method and the anonymity of the responses (no identifiable information will be collected), invitees will first see a brief description of the project on the first computer page. The following page will inform them of their human subjects and privacy rights—they will be prompted to continue (clicking on the ‘next’ button implies their consent) or decline (and then click the ‘opt out’ button). Upon completing the questions, participants will be thanked for their feedback and reminded that their responses are anonymous. At that point, they can choose once again to either submit their responses or opt out and have their information deleted.

See Attachment 15A for a copy of the oncology feedback survey tool; and Attachment 15B for a copy of the staff feedback survey tool.

Methodology and Research Instrument

For each trial the process will be: 1) NCI identifies a trial as low accruing based on their established guidelines; 2) the trial name is inserted into the tools; 3) an email is sent by NCI to those PIs randomly selected among those who opened the trial inviting them to send the survey links to the oncologist and staff member on his/her team who are best able to speak to the trial’s accrual concerns; 4) invitees can click on the link to complete and submit responses anonymously about the trial; 4) a reminder is sent out after one week to PIs, with the tool shutdown after two weeks; 5) data are compiled from an excel sheet print out and put into a standard report to give to the PI and NCI trial coordinator. NCI can use the report as part of their decision-making to run the trial. OCE will keep records of the trials over the year and determine the impact of the findings overall. OCE will debrief with NCI CTEP individuals representing the 24 trials to identify the degree that the findings were used in decision-making to either amend or close the trials.

Questions on the oncology tool include: their type of oncology practice; reasons for opening the trial; level of difficulty in opening the trial; scientific interest in the trial (initially and currently); general assessment of number of patients accrued compared to those eligible at their site; degree of competition for patient population; top reasons for low accrual; and recommended changes to the trial to improve accrual. Two optional questions exist if applicable: drug availability and support from different practices. Questions on the staff tool include: specific number of patients accrued compared to those eligible at their site; number of screening failures and why; level of difficulty consenting patients and why; top reasons patients decline enrollment; challenges to conducting the trial; top reasons for low accrual; and recommended changes to improve trial accrual. These standard template questions will be “tailored” for each trial: tailoring consists of inserting the trial name, cancer type and practice type into the survey when applicable. It is expected that all questions will be used for each trial, except those optional questions noted above for the oncology tool.

The 12-month pilot project is expected to begin the month after OMB approval is awarded. For each trial, the close-ended responses will be tabulated and summarized in table format (counts/amounts) on a question-by-question basis. Crosstabs will be included using “organization type” and “accrual rates” (low/high) as the categorical variables. Qualitative responses will be analyzed using a general inductive approach that focuses on condensing raw textual data into brief ‘chunks,’ establishing clear links between research objectives and the data, and developing a framework to describe what the data indicate. Conclusions and recommendations for each trial will be provided to CTEP and the PI in conjunction with a summary of the findings (as detailed above). In addition, at the end of the one-year pilot OCE will compile a summary document detailing how the tools were used over the year (e.g., number of trials), the degree of impact the findings had on trial decisions, and lessons learned. A recommendation will be made as to whether or not the process should be expanded permanently.

It is anticipated that the summary findings assessing the tools’ functions, process, use and impact on NCI decision-making for low accruing trials will be submitted for publication in a journal such as the Journal of Clinical Oncology, which is expected to be read by individuals in the community oncology research field who oversee the opening and closing of trials at their sites. It is understood that the summary information collected and compiled over the year will be useful to institutions and organizations that also struggle with accrual to its clinical trials. The publication will include specific discussion of the tools’ limitations, the qualitative nature of the data, and other related lessons learned from the pilot process.

Other Considerations

• A request for Office of Human Subjects Research exemption was submitted on May 2, 2011 and we are awaiting approval.

• No PII from respondents will be collected at all from respondents who complete these feedback surveys. Furthermore, no email addresses will be collected or stored. There will be a database behind each survey web-link, but this database is not designed to allow NCI staff, or others connected to the project, the ability to access or search, and it will not have any information stored that identifies the people in any way what so ever.

Burden

A total of 24 oncology surveys and 24 staff surveys will be administered over the 12 months (assumes two trials a month; total = 48 completed surveys). Each survey should take each of the participants approximately 5 minutes (0.08 hours) to complete. It is expected that 38 persons will complete each survey for a total of 1,824 completions (38 persons x 48 surveys). We therefore expect the total respondent burden for this proposed effort to be 152 hours. There have been 13 previous sub-studies approved by OMB under Generic submission OMB No. 0925-0046, totaling 1,876 burden hours requested to date. Approval by OMB of this sub-study would bring the total burden hour requested to date to approximately 2028, which is 29% of the total burden hours allowed (7050). Estimated cost to the Federal Government is $40,000 (contract submitted by vendor to tailor the 24 surveys, submit findings table, enter data into formatted report).

Estimates of Burden Hours
Types of Respondents	Instrument	Number of Respondents	Frequency of Response	Average Time Per Response (Hours)	Total Hour Burden
Oncologists and Research Staff for NCI-Sponsored Trials	Oncology Feedback Survey (Attachment 15A)	912	1	5/60 (0.08)	76
	Staff Feedback Survey (Attachment 15B)	912	1	5/60 (0.08)	76
Total					152

List of Attachments (separate file)

15A: Oncology Feedback Survey

15B. Staff Member Feedback Survey

3