memo 6-15-07

1. Where is the survey instrument, letter, and pre-notification letter for the previous NIH employees (those who chose to leave the NIH)? These do not seem to have been uploaded. Let me know if I've overlooked it. Otherwise, I will have to return this ICR as "improperly submitted."

The instrument and letter for the former employees mail survey are in the OnTRAC system, so this may be an issue with ICRAS. Can the project clearance office check this?

2. If the Privacy Act does not apply to this ICR, does NIH have some other statutory authority to provide assurances of confidientiality? If yes, please include the citation. If not, please remove all uses of the word "confidential" and replace it with "private to the extent permitted by law."

We have consulted with NIH legal advisors who have advised that protections should be provided for in the terms of the contract in that no personally identified information will be provided by the contractor to NIH, and that possible language to use is "confidential to the extent permitted by law." This change is reflected in the revised instruments.

3. Please clarify if the regulations that implemented these 'new ethics rules" have been finalized (i.e. has a final rule been published)?

Yes. The final rule was published in the August 31, 2005 Federal Register (Volume 70 Number 168, page 51559 to 51574).

4. Does NIH plan to revise the rule as a result of this study? Will reports to Congress be made as a result of this study?

Since the rule belongs to HHS and OGE, NIH can only suggest changes. Thus, if the survey data supports a rule change, we would recommend one to HHS and OGE. At this point, we have no intention of formally reporting the results to Congress.

5. Please provide more information on the actual analysis of the data and how NIH will analyze non-response bias

The analysis techniques used will include primarily descriptive statistics (e.g., means, frequencies, segmentation) and multivariate regression or correlation to determine the impact of several attributes on perceptions of NIH. In addition, there will be considerable qualitative analysis on the open-ended responses.

NIH and Macro chose the methodology and survey process to minimize non-response. For example, for the mail surveys, we follow a Dillman approach with specifically timed advance letters, mailings, and reminders. For the telephone survey, we meticulously manage the callbacks, the incremental addition of new sample, refusals, etc. We believe that these methods will result in a response rate of at least 80%. If the response rate falls below 80%, NIH will first attempt to take corrective action through fortifying the methodology (e.g., adding callbacks, refusal conversion, extra mailings). At the conclusion of the study, NIH and Macro will do the following: A) Validate face validity by comparing respondent demographics or other characteristics to known percentages from the original file provided by NIH to Macro. B) As a proxy for non-respondents, NIH and Macro will analyze the responses of late responders to the survey and compare them to early responders.

6. What does it mean when NIH says "... the brief [paper version of the] survey contains only essential questions" (B3)? Does this mean the paper based version of the survey is an abbreviated version of the CATI survey?

No, the paper survey includes all questions attached to the supporting statement. This statement was intending to convey that all surveys are kept brief so as not to be burdensome to the respondent.

7. What doe sit mean when NIH says "respondents will be notified by mail prior to receiving the [paper-based] survey" (B3)? Earlier, the supporting statement said that all

Former staff members will be sent a notification letter, letting them know that they will be receiving a paper survey. Potential applicants will not be notified prior to the survey, but will simply receive a CATI call. We apologize if this process was not clearly explained in the Supporting Statement.

8. On the survey of potential employees, respondents should be offered the option to say "don't know"

NIH and Macro will make this change throughout both survey instruments, and revised instruments are provided.

9. The burden estimate in ROCIS/ICRAS does not match what the PRA blurb says (10minutes vs 15 minutes) on the instrument

The Supporting Statement and instruments say 15 minutes for potential applicants and 10 minutes for former employees, so this may be an issue with ICRAS. Can the project clearance office check this?

10. on question F1, what does "rotate b to f only" mean?

This means that the CATI program will be programmed to automatically rotate questions F1b through F1f for each respondent. This will reduce the potential for order bias. We intentionally did not include question F1a in the rotation as it is an overall question that seeks a gut-level response before any prompts, therefore we would like this questions asked first on all surveys.