0920_0768_2010_NPIN_Supporting_Statement A

Outcomes Data Collection of the National Prevention Information Network (OMB Control No. 0920-0768)

Contact Information:

F. E. Harrison, MBA, PMP

National Prevention Information Network Services (NPIN) Project Officer

National Center for HIV, Viral Hepatitis, STD, and TB Prevention/CDC

1600 Clifton Rd NE, MS E-07

Atlanta, GA 30333

404-639-6095

404-639-8910(fax)

­­­ckv3@cdc.gov

January 7, 2011

Outcome Evaluation of the National Prevention Information Network

OMB No. 0920-0768

Table of Contents

A. Justification

A1. Circumstances Making the Collection of Information Necessary

A2. Purpose and Use of the Information Collection

A3. Use of Improved Information Technology and Burden Reduction

A4. Efforts to Identify Duplication and Use of Similar Information

A5. Impact on Small Businesses or Other Small Entities

A6. Consequences of Collecting the Information Less frequently

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A9. Explanation of Any Payment or Gift to Respondents

A10. Assurance of Confidentiality Provided to Respondents

A11. Justification for Sensitive Questions

A12. Estimates of Annualized Burden Hours and Costs

A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

A14. Annualized Cost to the Government

A15. Explanation for Program Changes or Adjustments

A16. Plans for Tabulation and Publication and Project Time Schedule

A17. Reason(s) Display of OMB Expiration Date is Inappropriate

A18. Exceptions to Certification for Paperwork Reduction Act

Submissions

B. Collection of Information Employing Statistical Methods

B1. Respondent Universe and Sampling Methods

B2. Procedures for the Collection of Information

B3. Methods to Maximize Response Rates and Deal with

Nonresponse

B4. Tests of Procedures or Methods to be Undertaken

B5. Individuals Consulted on Statistical Aspects and Individuals

Collecting and/or Analyzing Data

LIST OF ATTACHMENTS (as separate documents)

Attachment 1-A

Public Health Service Act (42 United States Code 241) Section 301

Attachment 1-B

Government Performance and Results Act of 1993 (Public Law No. 103-62) (PDF)

Attachment 1-C

Executive Order 12862 (PDF)

Attachment 2-A

60-Day Federal Register Notice

Attachment 2-B

60-day Federal Register Notice (PDF)

Attachment 2-C

30-Day Federal Register Notice

Attachment 3-A

Introductory Screen for NPIN Web site User Survey

Attachment 3-B

NPIN Web site User Survey

Attachment 3-C

Introductory Email for NPIN Products and Services User Survey

Attachment 3-D

NPIN Products and Services User Survey

Attachment 3-E

Email Reminder for NPIN Products and Services User Survey

Outcome Evaluation of the National Prevention Information Network (0920-0768)

Request for Revision

A. Justification

A.1. Circumstances Making the Collection of Information Necessary

This is a revision of the OMB approved collection (OMB No.0920-0768: expiry 3/31/2011). This request is seeking approval for data collection for 3 years.

Based on our actual experience of the NPIN Web site user surveys in the past three years, 500 completed surveys can be expected in a three to six months period and it has served our information needs quite well. So now the estimated annual burden has been substantially reduced to 1000 respondents and 342 hours annually.

The background information and need for this data collection are presented in Section A.1.1. The legislative authorization for this data collection is cited in Section A.1.2.

A.1.1 Background

The CDC NPIN serves as the U.S. reference, referral, and distribution service for information on the human immunodeficiency virus (HIV), sexually transmitted diseases (STDs), tuberculosis (TB), and viral hepatitis. Established in 1988, NPIN is an outgrowth of CDC health communications and information dissemination programs and the CDC National AIDS Clearinghouse (NAC). NPIN is a key member of the network of government agencies, community organizations, businesses, health professionals, educators and human services providers that educate the American public about the grave threat to public health posed by HIV/AIDS, STDs, TB, and viral hepatitis, and provides services for persons who are living with, or at risk for, these conditions. NPIN plays a key role in supporting the mission of the CDC National Center for HIV, Viral Hepatitis, STD, and TB prevention (NCHHSTP) in the prevention and control of HIV/AIDS, STDS, TB, and viral hepatitis by linking Americans to prevention, education, testing, and healthcare services. Its main function is to facilitate communication among stakeholders by researching, collecting, organizing, and disseminating the latest information and materials produced by CDC.

The NPIN Web site, and its other products and services are the primary channels used by the CDC to provide information concerning prevention, treatment, and care of HIV, STD, TB, and viral hepatitis to its prevention partners, primarily community-based organizations, State and local health departments, and health professionals working in the areas of HIV/AIDS, viral hepatitis, STD, and TB prevention; stakeholders, and other constituents. The NPIN Web site includes several searchable databases that can be used to locate information about testing centers, funding opportunities, upcoming conferences, and educational materials.

This revision will improve implementation of current best practices in Web evaluation by asking questions about visitor’s top tasks on the NPIN site, and allow evaluation through the Products and Services survey of new NPIN services, including the dissemination of information via social media.

The CDC is authorized to collect this data under the following legislation:

• Public Health Service Act (42 United States code 241)

• Executive Order 12862

• Government Performance and Results Act of 1993 (GPRA) (Public Law 103-62)

The pertinent sections of these laws and Executive Order can be found in Attachment A.

The proposed data collection will be used to ascertain customer satisfaction with NPIN in terms of services and products, and will meet the requirements under Section 301 of the Public Health Service Act (42 United States code 241), Executive Order 12862, and GPRA. This data collection falls within the larger CDC NPIN service contract.

GPRA intends to “improve Federal program effectiveness and public accountability by promoting a new focus on results, service quality, and customer satisfaction” (Section 2.b.3). In order to fulfill this responsibility, NPIN must collect data from its user groups to better understand their needs and desires and in turn, respond accordingly.

A.1.2 Privacy Impact Assessment

Neither individual information nor “Information in Identifiable Form” (IIF) will be collected.

A.1.3 Overview of the Data Collection System

NPIN collects feedback from users of the NPIN Web site and users of NPIN products and services (OMB No.0920-0768 expiry 3/31/2011). The NPIN Web site User Survey was made available online through the Key Survey system. User feedback is completely voluntary. NPIN notified Web site users via a “blast” email on the availability of the user feedback survey online through the Prevention News Update publication. NPIN Web site users have also been able to access the link to the online form listed on the NPIN Web site and will have access to the survey through NPIN’s social networks (e.g., Twitter).

Screenshots of the two surveys are enclosed below:

A.1.4 Items of Information to be Collected

The following items of information will continue to be collected in the next three years

• NPIN Web site users’ experiences with the NPIN Web site

• Relevance to users’ work

• Areas in which NPIN Web site information is used

• User suggestions to improve the NPIN Web site and how to better meet user needs

• Information users would like NPIN to add on its site

• The kind of organization and population they serve

• User satisfaction with NPIN products and services

• User experience with NPIN customer service

• Future use of NPIN products and services

A.1.5 Identification of Websites(s) and Website Content

Directed at Children Under 13 Years of Age

NPIN provides a link to access the surveys on the NPIN Web site and this web site is NOT directed at children under 13 years of age.

The NPIN Web site provides products and services to prevention partners and the public on the prevention of HIV/AIDS, Other STDs, TB and viral hepatitis. It helps to connect partners in public health to resources that support prevention programs, including downloadable materials, training, funding opportunities, and evidenced-based interventions. More details can be found at www.cdcnpin.org A screenshot is enclosed below:

A.2. Purpose and Use of Information Collection

The primary purposes of the proposed data collection are to assess NPIN users’ satisfaction with the NPIN Web site, products, and services; determine the extent to which the users’ needs are being met; identify how the Web site, products, and services can be enhanced to meet the needs of the user; and identify new needs of the users in an ever-changing environment. Specifically, the evaluation will examine (1) items most frequently used, (2) perceived quality, (3) user expectations, satisfaction, and trust, (4) frequency of use, and (5) other sources of information used related to the treatment and prevention of HIV/AIDS, STDs, TB, and viral hepatitis.

Feedback from the Web site survey has lead NPIN to undertake usability testing to clarify partner needs and to ensure the site is providing partners with resources to strengthen their local prevention efforts. NPIN will be redesigning the site to better clarify NPIN’s role in connecting HIV/AIDS, viral hepatitis, STD, and TB prevention partners and to highlight NPIN’s unique resources to support prevention workers. These unique resources include the NPIN databases, and web communities.

Other features added to the site based on partner feedback include:

• Partner Forum, with a monthly Featured Partner profile and interactive web communities to address partner requests for program ideas and opportunities to connect with others working in the field

• Web Tools section with widgets, buttons, eCards and banners partners can use on their Web sites

• Updated electronic mailing list subscription process so users can set all their preferences on one page

Collection of information to date on the NPIN Web site has been more difficult than anticipated, possibly due to placement of the survey link and length of the survey. To address these issues, NPIN plans to simplify the survey and to place the survey link more prominently on the site.

There is a continued need for NPIN to collect data because Web best practices and customer needs are constantly evolving, and evaluation is critical to developing new services and features to meet the changing standards of the Web and developing needs of our user base.

The evaluation will be accomplished by survey data collection from two groups—users of the NPIN Web site and users of NPIN products and services (see Appendices E and F). Respondents for each survey will include representatives from government agencies, community-based organizations, advocacy organizations, various other organizations involved in the prevention and/or treatment of HIV/AIDS, STDs, TB, and/or viral hepatitis, and the general public. The NPIN Web site user survey will be conducted on an ongoing basis via the Web site and a blast e-mail reminder will be sent out annually. The NPIN products and services user survey will be conducted on a bi-annual basis with a blast email sent out every 6 months. When appropriate, NPIN will distribute the survey at conferences and via social networks. Some of the NPIN Web site user surveys and the NPIN products and services surveys will be conducted over the phone as needed, which will be kept to an absolute minimum.

The information collected from the surveys is not intended to provide statistical data for publication. The purpose of this activity is solely to obtain user feedback that will help identify opportunities to improve the services and products provided to the public by NPIN and to ultimately allow NPIN to fulfill its mission.

Collecting the information described in this package allows NPIN to:

• Acquire accurate, up-to-date information from users of the NPIN Web site, and other products and services on a regular basis and in a timely manner

• Identify the service needs of NPIN users and implement new features to meet those needs

• Identify the strengths and weaknesses of the NPIN Web site, and others products and services

• Collect data using a consistent format

• Comply with requirements under the Public Health Service Act, Executive Order 12862, and GPRA

• Provide the highest quality products and services to NPIN users

Without this information collection, CDC will be hampered in successfully carrying out its mission of providing quality products and services to populations served. Failure to continue with our data collection effort would compromise efforts to meet the legislative requirement of being as responsive as possible to the public who consistently seek information about the prevention and treatment of HIV/AIDS, STDS, TB, and viral hepatitis. Moreover, it would diminish NPIN’s value to the public in terms of usability and credibility as a comprehensive Federal information and education resource.

In addition, failure to continue our data collection effort would:

• Inhibit CDC’s ability to identify opportunities to improve upon the products and services provided to the NPIN user community

• Reduce CDC’s ability to plan for future products, services, and support systems to better serve the NPIN user community

The proposed continued information collection will enable NPIN to collect customer satisfaction data in an efficient manner, assess the ever-growing needs, and track user satisfaction and changes in what Web site features, products, and services are used most and least frequently. There are no legal obstacles to reduce the burden.

If NPIN does not collect this information, any changes to the Web site or to NPIN products and services will not be grounded in actual user needs and will be speculative.

A.2.1 Privacy Impact Assessment

No “Information in Identifiable Form” (IIF) will be collected.

A.3. Use of Improved Information Technology and Burden Reduction

Improved information technology will be used to reduce the burden on the public and to comply with requirements of the Government Paperwork Elimination Act (GPEA). The proposed surveys will be conducted using the Internet since the vast majority of NPIN users have web access. A limited number of surveys may be distributed at conferences when appropriate, but the majority of the surveys will be completed electronically. This survey approach will reduce survey administration costs, automate data collection procedures, and facilitate data analysis.

Emails and social networking sites will be used to introduce and distribute a link to the NPIN Products and Services User Survey instrument for a sample of NPIN users (see Attachment F). We estimate that 95% of our responses to the NPIN Products and Services User Survey will be collected electronically. For those who do not have web access or choose to complete the survey by phone, a telephone version of the survey is available to be administered by NPIN staff over the phone (see Attachment I). It is estimated that less than 5% of respondents will complete the NPIN Products and Services Survey by telephone. Telephone respondents will not be sent a copy of the survey in advance.

A.4. Efforts to Identify Duplication and Use of Similar Information

Through extensive literature search, database searches, and consultation with other Public Health Service agencies, NPIN has determined that it provides unique products and services. The proposed data collection effort does not duplicate any other survey conducted by CDC or other Federal agencies.

A.5. Impact on Small Businesses or Other Small entities

Some of the organizations that use the NPIN Web site, products and services are small entities. The number of questions on the surveys is kept to a minimum and the use of electronic means will also reduce the burden of the data collection. Further, the surveys have been designed using mostly closed-ended questions that are grouped into categories for ease of response and have been pre-tested to ensure minimal burden. Respondents to the NPIN Products and Services User Survey will also be offered the option of completing the survey over the telephone, which will be conducted by call center staff. If there is no response from the organizations that have been contacted, a follow-up email will be sent to invite them to complete and submit their survey either online or by phone. All respondents to the NPIN Web site survey will complete and submit the survey online through the use of the KeySurvey system.

A.6. Consequences of Collecting the Information Less Frequently

The project proposes to continue data collection on a bi-annual basis for the NPIN Products and Services User Survey and on an annual basis for the NPIN Web site User survey. (An email reminder process for the Web site survey will be implemented annually, but the survey in the KeySurvey system will stay active online). As NPIN develops new products and services, and makes enhancements to the Web site on a rolling basis, it is critical to obtain feedback from the user every 6 to 12 months in order to respond effectively to negative reactions or perceptions in a timely manner. If this information is collected less frequently, NPIN’s ability to meet the ever-changing needs of the target population will hampered.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

The current data collection fully complies with the regulation 5 CFR 1320.5. and does not contain any special circumstances.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.8.A. A 60-day Federal Register Notice was published on 10/22/2010 for the 2010 publication (Vol. 75, No. 204, Page 65356-65357). (See Attachment 2-B: 60-day Federal Register Notice in PDF). No comments have been received from the public.

A.8.B. On an ongoing basis, the NPIN team maintains regular communication with expert resources to share information and discuss strategies to ensure that NPIN resources are maximized and implemented effectively. Collaboration for this data collection occurs across the four main divisions of NCHHSTP—Division of HIV/AIDS Prevention (DHAP), Division of Tuberculosis Elimination, Division of STD Prevention, and Division of Viral Hepatitis. These ongoing consultations help to assure that mutual and related data needs are being met and that data collections are not duplicated.

Contact information of consultations:

DHAP

Chris Cagle

Associate Director for Policy and Planning

CDC/NCHHSTP/DHPSE/OD

8 Corporate Boulevard

Atlanta, GA 30329

Ph: 404-639-8156

Email: MCagle@cdc.gov

Division of Tuberculosis Elimination

Amera Khan

Health Education Specialist

CDC/NCHHSTP/DTE/CEBSB

8 Corporate Boulevard

Atlanta, GA 30329

Ph: 404-639-6428

Email: ARKhan@cdc.gov

Division of STD Prevention

Mary McFarlane

CDC/NCHHSTP/DSTDP/OD

8 Corporate Boulevard

Atlanta, GA 30329

Ph: 404-639-8309

Email: xzm3@cdc.gov

Division of Viral Hepatitis

Cynthia Jorgensen

Health Scientist (Team Leader)

CDC/NCHHSTP/DVH/PB

8 Corporate Boulevard

Atlanta, GA 30329

Ph: 404-718-8534

Email: CJorgensen@cdc.gov

Division of HIV/AIDS Prevention (Prevention Communications Branch)

Erin Connelly

Team Lead, Technical Information and Communication Branch

CDC/NCHHSTP/DHAP/TICB

8 Corporate Boulevard

Atlanta, GA 30329

Ph: 404-639-3065

Email: efd5@cdc.gov

A.9. Explanation of Any Payment or Gift to Respondents

No incentive, remuneration, or gifts will be provided to participants of this data collection.

A.10. Assurance of Confidentiality Provided to Respondents

Email addresses and phone numbers for the NPIN Products and Services User Survey will be obtained from the NPIN mailing lists which provide a comprehensive listing of national, state, and local organizations that provide HIV/AIDS, STD, TB, and viral Hepatitis prevention, education, and treatment services. While this contact email will be used to invite participation for the NPIN Products and Services User Survey either by email or phone, NO respondent identifying information will be collected in the survey.

Similarly, the NPIN Web site User Survey as currently implemented does not collect or obtain any personal information from the survey participant. Each survey contains only a computer-generated unique participant identifier that is NOT linked to any individual identifying information.

The intended use of the survey responses is expressly stated in each survey’s introductory email. The disclosure of any and all information is voluntary and all survey responses have been and will be analyzed in the aggregate.

Danya International, Inc., is the data collection contractor. Access to all databases is password protected and secured. Responses from surveys completed by telephone will be directly entered into a secured, password protected database. All data will be accessible only to staff directly involved in the project. Servers are protected from unauthorized physical access by separate key lock to the network room. All data will be stored for a period of three years and then destroyed.

A.10.1 Privacy Impact Assessment

First, no “Information in Identifiable Form” (IIF) will be collected. Moreover, Danya International, Inc. will ensure the highest level of privacy and confidentiality by using both electronic and physical means. Danya employs a stateful-inspection packet filtering firewall to protect their network perimeter and data contained within it from sources outside of the network. Internal security is controlled using Windows NT share and file level security, and Novell NetWare NDS security. All data are password protected and secured on file servers within a locked server room. Servers are protected from unauthorized physical access by separate key lock to the network room. The contractor backs up virtual data to its DLT (Digital Liner Tape) on a nightly basis, Monday-Friday. Daily tapes are stored on-site in a locked network room, and weekly and monthly tapes are stored off-site in a locked vault at a tape storage facility.

A.11. Justification for Sensitive Questions

There are no sensitive questions in the surveys.

A.12. Estimates of Annualized Burden Hours and Costs

The average time burden per response for the NPIN Web site User Survey is estimated to be 15 minutes. Average time required to complete the NPIN Products and Services User Survey is estimated to be 13 minutes. The NPIN Web site User Survey is comprised of 24 questions whereas the Products and Services User Survey is comprised of 18 questions. The results of the pre-test were positive with respondents finding the surveys easy to complete, unambiguous, and free of undue burden.

A.12. A. Estimated Annualized Burden Hours

The Web site User Survey will be offered to all users of the NPIN Web site. However, the expected number of respondents will be less than 500 based on our experiences in the past two years.

The burden estimate for the data collection of each survey is based on pre-test results. The estimated annual burden to respondents is shown in Table A.12-A.

Table A.12-A. Estimated Annualized Burden Hours

Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
NPIN Web site User Survey	500	1	15/60	125
NPIN Products and Services User Survey	500	2	13/60	217
Total	1000			342

A.12.B. Estimated Annualized Burden Costs

Various types of organizations and staff types use the NPIN Web site, products and services. To assess the annualized costs to the respondent of this data collection, an average hourly wage rate was calculated for each type of organization that is likely to have used NPIN products and services using the latest government statistics from U.S. Department of Labor (Table A.12-B).

To ensure that a representative sample of products and services users is obtained to complete the Products and Services User Survey, a stratified sampling methodology will be applied in which a random sample of users within each organization type will be selected to participate in the survey. Respondents include all types of NPIN Users including social service organizations, health services organizations, community-based organizations, associations, foundations, faith-based organizations, libraries, news/media organizations, businesses, general public, international agencies, correctional facilities, and educational institutions. The table below reflects burden estimates assuming 500 respondents are estimated to complete the Products and Services User Survey (95% online and 5% by phone). There is no screening process for the survey and no additional burden will be incurred for telephone response.

Table A.12-B. Estimated Annualized Burden Costs

Form Name	Total Burden (in hours)	Average Hourly Wage Rate*	Respondent Cost
NPIN Web site User Survey	125	$20.90	$2,612.50
NPIN Products and Services User Survey	217	$20.90	$4,535.30
Total	342	$20.90	$7,147.80

* Source: The latest government statistics from U.S. Department of Labor, Bureau of Labor Statistics (for all occupations), May 2008 National Occupational Employment and Wage Estimates. http://www.bls.gov/oes/current/oes_nat.htm

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no reporting or recordkeeping “non-hour cost” burdens associated with this proposed information collection.

A.14. Annualized Cost to the Government

The costs of these surveys to the Government are absorbed within the larger CDC National Prevention Information Network service contract. The services provided under this contract include basic process evaluation and data collection on an ad hoc basis. The following chart summarizes the components of the annualized costs of the survey under the service contract:

Component	Cost
Labor – Research Support	$26,952
G & A	$3,321
Other Costs (telephone, postage, supplies, computer/network support)	$2,000
Federal Personnel Costs (GS-13 level)	$960
Total estimated annual costs	$33,233

The total cost for the project is $99,699.

A.15. Explanation for Program Changes or Adjustments

Based on our actual experience of the NPIN Web site user surveys in the past three years, 500 completed surveys can be expected in a three to six months period and it has served our information needs quite well. So now the estimated annual burden has been substantially reduced to 1000 respondents and 342 hours annually.

For the NPIN Web Site User Survey, the number of questions has remained at 24. The first section has been revised, so “Your Impressions of the Web site” has become “Your Experiences with the NPIN Web site.” Questions have been added to reflect current Web practice to identify user’s top tasks and ease of completion. Questions include: the frequency of them visiting the NPIN Web site, how they came to NPIN site, the tasks users came to the NPIN site to accomplish, whether they were able to accomplish their tasks, and how they are planning to use the information obtained. Questions to rate the NPIN site and ease of navigation were retained.

In the “Relevance to Your Work” section, a question was added asking users what disease topics they look for on the NPIN Web site.

In the “Use of the NPIN Web Site” section, a question was included asking users to identify one thing they would most like to change or fix.

In the “How to Better Meet Your Needs” section and the “Background Information” section the questions have remained the same with only slight wording improvement and new subcategories like “Twitter”, “Facebook” and “LinkedIn” added.

For the NPIN Products and Services User Survey, the number of questions has been reduced from 27 to 18, which will substantially reduce the burden on the respondents.

The questions in “Your Expectations and Satisfaction” section, have remained unchanged.

In the “NPIN Customer Service” section, the three questions regarding ordering materials have been removed as materials are now available from NPIN solely via download.

In the “Use of NPIN Products/Services” section, one question has been added to identify whether users prefer Live-Chat or the 800 number.

In the “Relevance to Your Work” section, some new categories (such a Twitter) have been added to the NPIN products/services and one question regarding whether the information provided by NPIN was current has been removed.

In the “Future Use of NPIN Products and Services” section, one question regarding ordering materials has been removed, as it is no longer applicable. The other two questions have remained the same.

In “How to Better Meet Your Needs” section, one Likert scale question involving ordering materials has been removed.

A.16. Plans for Tabulation and Publication and Project Time Schedule

The NPIN data collection activities will continue under the current OMB approval until March 31, 2011. After the extension request has been approved, the NPIN team will start using the revised instruments. NPIN requests a three-year extension clearance.

This activity is essential to identify opportunities to improve the services and products provided to the public by NPIN and to ultimately allow NPIN to fulfill its mission.

Table A.16-A. Project Time Schedule for Revised New Surveys

Activity	Time Schedule
Complete programming of revised web-based surveys	30 days after OMB approval
Launch revised web-based surveys	30–90 days after OMB approval
Complete data collection	90–210 days after OMB approval
Analysis of survey findings	210–240 days after OMB approval

A.17. Reason(S) Display of OMB Expiration Date Is Inappropriate

None

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

None

36