NLM-WHR-Professionals_Focus_Group-Protocol-Rev_11-20-2012

Women’s Health Resources:

Usability Analysis and Audience Evaluation

Professionals Focus Group Protocol

November 20, 2012

Submitted to:

Laura Bartlett

Outreach and Special Populations Branch

National Library of Medicine

National Institutes of Health

Submitted by:

Ana Tellez

CommunicateHealth Inc.

26 Market Street, Suite 3

Northampton, MA 01060

413.582.0425

Table of Contents

Objectives 1

Participants 2

Protocol 3

Final Report 4

Appendix A: Senior and Junior Researcher Screeners 5

Appendix B: Advocacy Group Member Screener 8

Appendix C: Media Professional Screener 10

Appendix D: Professionals Focus Group Moderator’s Guide 12

Appendix E: Participant Questionnaire 17

Appendix F: Consent Form 19

Appendix G: Ranking Activity 20

Objectives

The National Library of Medicine, Specialized Information Services (NLM/SIS) is redesigning the Women’s Health Resources (WHR) website. This includes developing one or more research brief templates to communicate women’s health and sex differences research in a way that is engaging to specific target audiences.

In order to better understand the information needs and preferences of its users, CommunicateHealth will conduct focus groups with the following four target audiences:

• Senior Career Researchers

• Junior Career Researchers

• Advocacy Group Members

• Media and Communications Professionals

Specifically we want to learn:

1. How do the individual target audience groups define, understand, and use women’s health and sex differences research?

   1. What are the individual needs related to women’s health and sex differences research for each target audience group?

      1. What type of research do different target audiences seek out?

      2. How do target audience groups use information on women’s health and sex differences research?

   2. What are target audience groups’ attitudes and opinions related to women’s health and sex differences research?

      1. Is women’s health and sex differences research important to each target audience group? Why or why not?

      2. Is there a particular topic within women’s health that is most or least important to each target audience group?

2. What types of resources are the target audience groups using currently?

   1. What types of resources are they seeking and what are they finding?

   2. What do they like or dislike about these resources?

3. What do the target audience groups want out of the WHR website, research briefs, social media, and future mobile device offerings?

   1. What information, materials, tools, and messages are likely to help each target audience group utilize a WHR website and/or mobile application more effectively?

4. What information should be emphasized on research briefs for each target audience?

Participants

CommunicateHealth will conduct 11 focus groups with 50 participants total. Focus groups will be held remotely. Below is an overview of the focus group composition for each target audience.

1. Senior Career Researchers (15 participants)

• 20 years of research experience

• Mix of gender

• Mix of research area expertise

2. Junior Researchers (15 participants)

• Postdoctoral fellows and young/new faculty

• Mix of gender

• Mix of research area expertise

3. Advocacy Group Members (10 participants)

• Members of national health advocacy groups from a range of focus areas

• Mixture of stakeholders with and without formal biomedical or scientific training

4. Media and Communications Professionals (10 participants)

• Reporters and health writers

• Mix of national, local and college news writers

• Mix of bloggers and other non-traditional/emerging media outlets

• Mix of reporters from media outlets that target specific racial/ethnic audiences

• Mix of experience/knowledge in biomedical or health sciences

See Appendices A, B, and C for screeners and details on target demographics of participants.

Protocol

Each remote focus group session will last approximately 90 minutes and will include a moderator and note taker. The focus groups will be audio recorded to help with analyzing participant feedback and in writing the report. Focus groups will be conducted remotely. The moderator will follow a standard protocol for each session that includes the following (see Appendix D for Moderator’s Guide):

• Welcome, opening remarks, ground rules and introductions

• Explanation of the purpose of the focus group

• Consent form (see Appendix F)

• Warm-up discussion

• Discussion of Women’s Health Research

• Ranking activity (see Appendix G)

• Closing discussion

• Participant questionnaire (see Appendix E)

• Participant remuneration

  • Senior Career Researchers: $40

  • Junior Career Researchers: $40

  • Advocacy Group Members: $40

  • Media and Communications Professionals: $40

Stimulus Materials

To give participants something to react to and discuss, we will provide participants with a list of 25 sample resources found on the WHR website to help participants think about women’s health research.

Final Report

CommunicateHealth will analyze feedback from focus group participants to identify important themes, commonalities and preferences which will guide the redesign of the WHR website content and features. CommunicateHealth will deliver a topline report including user profiles for each target audience based on the findings and recommendations for design and content for research brief templates, the WHR Portal website, and a mobile site/app. Additionally, CommunicateHealth will include mock-ups of sample design recommendations in the topline report.

Appendix A: Senior and Junior Researcher Screeners

OVERVIEW

Our desired participants are senior career researchers with over 20 years of research experience and junior researchers who are currently postdoctoral fellows or have less than five years of experience as faculty at a research institution. Participants’ research focus areas will represent a mix of women’s health, sex differences, and areas related to women’s health. Participants will represent a mix of sex.

Senior Researchers: 20 years or more of research experience (18 participants)

Junior Researchers: Currently a postdoctoral fellow or having less than 5 years of experience as full-time faculty at a research institution (18 participants)

Research Focus Area: Women’s health, sex differences, or an area related to, but outside of, women’s health

Gender: Mixed

The following factors will not be targeted for the recruitment, but data on these factors will be collected from participants in a questionnaire administered at the end of the focus group:

• Frequency of Internet Use

• Preferred format of research resources (i.e. Web sites, PDFs, printed materials, mobile applications)

• Technology use (i.e., personal PDAs, tablets, mobile applications)

• Professional networking sites used

SCRIPT

Hello,

This is NAME from CommunicateHealth Inc. calling on behalf of the NIH’s National Library of Medicine. Thanks for expressing interest in participating in a research study to help the National Library of Medicine improve their Women’s Health Resources web portal. We are looking for researchers to provide feedback on resources they currently use in their research, and new resources they would like to see made available.

We would like to ask you a few questions to see if you qualify for the study. If you qualify, we will see if you are available to participate in a one-hour remote study (meaning you can do the study on your home or work computer). There is a $40 incentive to participate. Does this sound like something you would be interested in?

Great. Let's find out if you qualify. I have a few simple questions to ask. This should take less than 5 minutes. At some point, I may end the questions if I discover you don't qualify. This has nothing to do with you. We simply are looking for people who meet certain criteria.

1. How many years have you worked as a researcher?

_______ Am not a postdoctoral researcher (TERMINATE)

_______ 19 years or less [Continue to question #2]

_______ 20 years or more [Continue to question #3]

2. What best describes your current work experience?

_______ 6 years or more as faculty at a research institution (TERMINATE)

_______ 5 years or less as faculty at a research institution (CONTINUE)

_______ Completing a Postdoctoral Fellowship (CONTINUE)

3. I am now going to read a list of women’s health topics. As I read through the list, please indicate whether you have conducted research in any of the following areas¹: (Recruit a mix)

_______ AIDS and Pregnancy (CONTINUE)

_______ Breast Cancer (CONTINUE)

_______ Cervical Cancer (CONTINUE)

_______ Ovarian Cancer (CONTINUE)

_______ Uterine Cancer (CONTINUE)

_______ Vaginal Cancer (CONTINUE)

_______ Vulvar Cancer (CONTINUE)

_______ Osteoporosis (CONTINUE)

_______ Menopause (CONTINUE)

_______ Menopausal Hormone Therapy (CONTINUE)

_______ Postpartum Depression (CONTINUE)

_______ Military and Women (CONTINUE)

_______ Reproductive Health (CONTINUE)

_______ Other (please specify):

______________ (CONTINUE)

_______ None (CONTINUE)

4. I am now going to read a second list of health topics. As I read through the list, please indicate whether you have conducted research in any of the following areas: (Recruit a mix)

_______ Cancer other than those listed above (please specify): ___________ (CONTINUE)

_______ Diabetes (CONTINUE)

_______ Exercise and Fitness (CONTINUE)

_______ Heart Disease (CONTINUE)

_______ HIV/AIDS (CONTINUE)

_______ Lung Diseases and Disorders (CONTINUE)

_______ Mental Health (CONTINUE)

_______ Nutrition (CONTINUE)

_______ Safety and Wellness (CONTINUE)

_______ Sexually Transmitted Diseases (CONTINUE)

_______ Violence and Abuse (CONTINUE)

_______ None (CONTINUE)

5. What type of research do you conduct?² (Recruit a mix)

_______ Basic or experimental medical research

_______ Clinical studies

_______ Epidemiological studies

6. (Note sex, recruit a mix)

_______ Male

_______ Female

Appendix B: Advocacy Group Member Screener

OVERVIEW

Our desired participants are members of national health advocacy groups. Participants will represent a mix of organizations, both women-specific and non-women-specific, and include backgrounds with and without biomedical or scientific training. Participants will represent a mix of gender.

Advocacy Group Membership: Member of national health advocacy groups from a range of focus areas, e.g., Society for Women’s Health Research, American Cancer Society, American Lung Association, and American Heart Association.

Biomedical or Scientific Training: Mix of stakeholders with and without biomedical or scientific training

Gender: Mixed

The following factors will not be targeted for the recruitment, but data on these factors will be collected from participants in a questionnaire administered at the end of the focus group:

• Frequency of Internet Use

• Technology use (i.e., personal PDAs, tablets, mobile applications)

• Profession

• Professional networking sites used

We will strive to recruit a mix of demographic characteristics such as race, gender, and age as much as is feasible. Because we are only recruiting 12 participants, not every participant characteristic will be accounted for in the end sample.

SCRIPT

Hello,

This is NAME from CommunicateHealth Inc. calling on behalf of the National Institutes of Health’s National Library of Medicine. Thanks for expressing interest in participating in a research study to help the National Library of Medicine improve their Women’s Health Resources web portal. We are looking for members of health advocacy groups to provide feedback on women’s health resources they currently use, and new resources they would like to see made available.

We would like to ask you a few questions to see if you qualify for the study. If you qualify, we will see if you are available to participate in a one-hour remote study (meaning you can do the study on your home or work computer). There is a $40 incentive to participate. Does this sound like something you would be interested in?

Great. Let's find out if you qualify. I have a few simple questions to ask. This should take less than 5 minutes. At some point, I may end the questions if I discover you don't qualify. This has nothing to do with you. We simply are looking for people who meet certain criteria.

1. What health advocacy topics or issues are you most interested in? (Recruit a mix)

2. Do you have any formal biomedical or scientific training? (Recruit a mix)

_______ Yes (CONTINUE)

_______ No (CONTINUE)

3. (Note sex, recruit a mix)

_______ Male

_______ Female

Appendix C: Media Professional Screener

OVERVIEW

Our desired participants are health reporters and writers. Participants will represent a mix of media outlets including both traditional media (television, newspapers, magazines) and new media (blogs), on the national, local, and college levels. Participants will represent a mix of knowledge in the biomedical or health sciences. Participants will represent a mix of gender.

Profession: Health reporter or health writer

Media Outlets: Mix of television, newspaper, magazine, blog, or other online outlets (e.g. online magazine).

Media Outlet Reach: Mix of national, local, and college outlets

Media Outlet Target Audience: Mix of outlets that target specific racial/ethnic populations

Biomedical or Health Science Knowledge: Mix of participants with and without biomedical or health science knowledge

The following factors will not be targeted for the recruitment, but data on these factors will be collected from participants in a questionnaire administered at the end of the focus group:

• Preferred format of research resources (i.e. Web site, PDFs, printed materials, mobile applications)

• Technology use (i.e., personal PDAs, tablets, mobile applications)

• Professional networking sites used

We will strive to recruit a mix of demographic characteristics such as race, gender, and geographic location as much as is feasible. Because we are only recruiting 12 participants, not every participant characteristic will be accounted for in the end sample.

SCRIPT

Hello,

This is NAME from CommunicateHealth Inc. calling on behalf of the National Institutes of Health’s National Library of Medicine. Thanks for expressing interest in participating in a research study to help the National Library of Medicine improve their Women’s Health Resources web portal. We are looking for health reporters and writers to provide feedback on health information resources they currently use, and new resources they would like to see made available.

We would like to ask you a few questions to see if you qualify for the study. If you qualify, we will see if you are available to participate in a one-hour remote study (meaning you can do the study on your home or work computer). There is a $40 incentive to participate. Does this sound like something you would be interested in?

Great. Let's find out if you qualify. I have a few simple questions to ask. This should take less than 5 minutes. At some point, I may end the questions if I discover you don't qualify. This has nothing to do with you. We simply are looking for people who meet certain criteria.

1. What types of media outlets do you write for? Choose all that apply. (Recruit a mix)

_______ Newspaper (CONTINUE)

_______ Television (CONTINUE)

_______ Magazine (CONTINUE)

_______ Blog (CONTINUE)

_______ Online Magazine (CONTINUE)

_______ Other: _____________ (CONTINUE)

2. Does the media outlet or publication you work for reach: Choose all that apply. (Recruit a mix)

_______ A national audience (CONTINUE)

_______ Local (CONTINUE)

_______ College (CONTINUE)

3. Does the media outlet you work for specifically target any of the following audiences? Choose all that apply. (Recruit a mix)

Ethnicity:

________ Hispanic or Latino

________ Not Hispanic or Latino

Race:

________ White

________ Black or African-American

________ American Indian or Alaska Native

________ Native Hawaiian or Other Pacific Islander

________ Asian

4. Do you have a formal education in or experience with biomedical or health sciences? (Recruit a mix)

_______ Yes (CONTINUE)

_______ No (CONTINUE)

5. (Note sex, recruit a mix)

_______ Male

_______ Female

Appendix D: Professionals Focus Group Moderator’s Guide

Welcome/Opening Remarks (10 minutes)

[The moderator will greet everyone as people join the session remotely.]

Thank you for joining us today. My name is _____ I will be facilitating this discussion group on behalf of the National Institutes of Health’s (NIH) National Library of Medicine (NLM). NLM will be redesigning the Women’s Health Resources Web Portal – a website that provides information and resources related to women’s health and sex differences research. A portion of the information on the website is aimed at [indicate audience group – researchers, advocacy group members, media professionals]. We are conducting research on the information needs of the various types of users that visit the NLM Women’s Health Resources Web Portal.

The ideas and information you share with us are very important. What you tell us today will help us better understand the unique information needs of professionals in your field when it comes to women’s health research. With your help, we hope to improve the existing website and related materials so that they are helpful, clear, and engaging.

Before we begin, I’d like to make a few things very clear and explain how the discussion will work.

1. We are not trying to sell or promote any product or service to you.
   
   

2. There are no right or wrong answers—we want your opinions.
   
   

3. ____ will be taking notes and audio recording the discussion. We have to write a report on your suggestions to inform the redesign of the website. Audio recording the discussion will help us write the report. We will not use your names in the report. We will only share the report with people working on this project.

4. In order to make sure everyone’s thoughts and opinions are heard, it is important that you only speak one at a time. I may occasionally interrupt you when two or more people are talking at once in order to be sure we hear everyone’s suggestions and opinions.
   
   

5. This discussion will last approximately an 90 minutes. I want to be sure not to keep you here much longer, so I may occasionally interrupt you to keep the discussion focused.
   
   

6. Please turn off your beepers and cell phones.

[The moderator will refer to previously emailed informed consent form, saying:]

Your participation in this discussion is voluntary. You may stop the discussion at any time. Your name will NOT be used in any report.

Has everyone read, signed, and returned the consent form that was previously emailed to you? [Get a “yes” from all participants before continuing.]

[The participants will be emailed the informed consent statement form assuring them that information provided during the discussion group will be kept confidential and only be used for the redesign of the WHR website and related materials. The form will also contain language explaining that signing the form confers permission to be audio recorded.]

Introduction and Warm-up (5 minutes)

First, let’s go around and introduce ourselves. Tell us:

1. Your first name (only)

2. One thing you do in your job or advocacy work that is related to women’s health research

We’re going to start with a few warm-up questions.

• What comes to mind when you think about women’s health?

• What comes to mind when you hear the phrase ‘women’s health research’?

• Have you heard of sex differences research before? If so, what do you know about it? [If none of the participants indicate that they know what sex differences research is then the moderator will provide a few examples].

Discussion of Women’s Health Research (60 minutes)

Now we’re going to start talking about women’s health research. I’m going to ask some questions for the group to answer and discuss. Remember, there are no right or wrong answers.

[For all researchers]

• In what way is research about women’s health important to you as a researcher?

• Where do you usually go to find research information?

• What is most important about a research area or study that you need to know for your job as a researcher (such as methodology, effects of treatments, participant characteristic, etc.)? Why?

• If a publication came out detailing the findings of a new study, what would you need to know to help you apply it to your work?

[For women’s health researchers only]

• What specific women’s health topics within your research area of expertise are important to you? Why?

• Where do you usually go to get the information about women’s health research?

• What do you like about the sources you use?

• What do you dislike about the sources?

• Have you ever searched for health research using a mobile device?

• What types of materials/information do you find most useful? Why?

• What organizations, sources, or sites do you trust when looking for information?

[For non-women’s health researchers only]

• Is there a particular women’s health research area topic that you think is important to your work? Why?

• Where do you usually go to get the information about health and biomedical research?

• What do you like about the sources you use?

• What do you dislike about the sources?

• Have you ever searched for health research using a mobile device?

• What types of materials/information do you find most useful? Why?

• What organizations, sources, or sites do you trust when looking for information?

[For advocacy group, and media/communication participants]

• Is women’s health research important to you? Why or why not?

• Recount the last time you discussed or thought about women’s health research.

• Describe the conversation or thought process that lead to thinking about women’s health research.

• Do you ever search for information on women’s health research as part of your work?

[If Yes:]

• What information are you typically looking for?

• How frequently do you look for or read about this type of research?

• What type of information is most important to you about this type of research? For example, rigor of the study, funding sources, type of health condition, age or specific characteristics of the women included, etc. Why?

  • Is there a specific population of women you think is important? Why?

• Can you tell us about the last time you searched for women’s health research information?

• How did you find the information you were looking for?

• How did you use the information you found?

• Where do you usually go to get the information about women’s health research?

• What do you like about the sources you use?

• What do you dislike about the sources?

• Have you ever searched for health research using a mobile device?

• Have you ever used a social media website to search for health information?

• What types of materials/information do you find most useful? Why?

• What organizations, sources, or sites do you trust when looking for information?

[If No:]

• What type of health research do you use or read about for you work?

• Do you ever read about other types of research as part of your work? If so, what types?

• How do you think women’s health research relates to the work you do as a [insert role per target audience]?

• What about women’s health is important to your work as a [insert field for target audience]? Why?

Information Ranking Activity (10 minutes)

[For this exercise, the moderator will ask participants to choose the most and least important items from a list of WHR types of information. Results will be used in a discussion of why certain types of information are most or least useful for each individual target audience.]

Now I’m going to show you a list of some types of information you might find on a women’s health research website. On a sheet of paper, write down the 3 items that are the most important to you in your work and 3 items that are the least important to you in your work. Do this on your own. Then, we will go around and you can tell us which ones you picked and why. I’ll give you a few minutes to complete this now.

• Which type of information is most important and why?

  • Out of the 2-3 items you wrote down as important, put a star next to the item that is the most important to your work. Why did you star that item?

• Which type of information is the least important and why?

  • Out of the 2-3 items you wrote down as not important, put an X next to the item that is the least important to your work. Why did you put an X next that item?

• Are there any types of information not on the list that you think should be added or that you would like to see on a women’s health research website?

Thank You and Closing

The moderator will conclude each session by thanking participants for offering their opinions and suggestions.

Participant Questionnaire (5 minutes)

The moderator will pass out questionnaire, read instructions out loud and answer any questions.

Appendix E: Participant Questionnaire

1. What is the highest level of education you have completed?

____ Grade school

____ Less than high school graduate/some high school

____ High school graduate or completed GED

____ Some college or technical school

____ Received four-year college degree

____ Some post graduate studies

____Received advanced degree

____ Other: _____________________

3. When you look for health information, where do you go? (Check all that apply)

_____ Books

_____ Brochures, pamphlets, etc.

_____ Organization

_____ Family

_____ Friend/Co-Worker

_____ Health care provider

_____ Internet

_____ Library

_____ Magazines

_____ Telephone information number (1-800-Number)

_____ Complementary or alternative practitioner

_____ Other: _________________________

_____ Do not look or go anywhere

4. How often do you find general information via the following channels?

_____ Print: [never, sometimes, often, always]

_____ Web: [never, sometimes, often, always]

_____ Mobile sites: [never, sometimes, often, always]

_____ Mobile apps: [never, sometimes, often, always]

_____ Social media: [never, sometimes, often, always]

5. Would you use a mobile version of a website on women’s health research?
_____ Yes

_____ No

7. Which types of information have you searched for online in the past 12 months? (Check all that apply)

_____ Clinical trials

_____ Research funding

_____ Pharmaceutical information

_____ Diagnostic testing

_____ Journal/peer-reviewed article

_____ Health focused government agency website

_____ Legislation or policy related to health

_____ Complementary/alternative medical treatments

8. About how often do you use the Internet?

_____ Several times a day

_____ About once a day

_____ 3-5 days a week

_____ 1-2 days a week

_____ Every few weeks

_____ Less often

_____ Never

_____ Don’t know/refused

10. Do you have one or more of the following? (Check all that apply)

_____ A desktop computer

_____ A laptop computer

_____ A cell phone

_____ A ‘smart’ mobile phone, e.g., an iPhone, Droid, or Blackberry

_____ A tablet computer, e.g., an iPad or Kindle

_____ An iPod or other MP3 player

11. Have you ever used any professional networking sites?

_____ Yes

If so, please list the sites you have used _________________________

_____ No

12. Please tell me your race or ethnic background. Are you? (Check all that apply)

Ethnicity:

________ Hispanic or Latino

________ Not Hispanic or Latino

Race:

________ White

________ Black or African-American

________ American Indian or Alaska Native

________ Native Hawaiian or Other Pacific Islander

________ Asian

Appendix F: Consent Form

I, _________________________________________, agree to take part in this focus group study.

I understand that I do not have to be in this study. I can leave at any time without penalty. I can agree to be in the study and then change my mind later.

I allow the National Institutes of Health (NIH) to use the information from this study. I understand that the information is for research only, and that my name will not be shared with anyone else.

I agree to ask questions about the study if I don't understand something. If I have questions after the study is over, I can contact Ana Tellez at talk@communicatehealth.com or at 413-582-0425.

Audio Recording Release

I understand that I will be audio recorded during this study. I allow NIH to use the recordings of me for research purposes only. I understand that my name will not be used for any other purpose.

I give up any rights to the recording and understand the recording may be copied and used by NIH without my permission.

Summary

I have read and understood this consent form. I understand that I will get a copy of this form.

Print Name:         _________________________________________

Signature:           _________________________________________

Date:                   _________________________________________

Appendix G: Ranking Activity

Women’s Health Research Information List

• Information on funding from NIH

• Latest women’s health research news

• Health topic: Complementary/Alternative medicine

• Current women’s health clinical trials

• Clinical trial results

• Links to full peer-reviewed journal articles

• Press releases

• Health topic: Violence against women

• Impact of research on new treatments

• Impact of research on new testing methods

• Detailed information on specific health topics

• Health topic: Substance abuse

• Research information specific to different age groups

• Research information specific to difference ethnic/racial groups

• Health topic: Autoimmune diseases

• Women’s Health related events at NIH

• How to contact someone about women’s health questions at NIH

• Health topic: Heart disease

• Social media links for NIH women’s health research (Twitter, Facebook, LinkedIn)

• Info about who NIH is as an agency

• Health topic: Reproductive health

• Links to other government agencies

• Links to other women’s health research websites

• Women’s health research conference information

• Health topic: Cancer

1 Source for response choices: http://www.womenshealthresources.nlm.nih.gov/healthtopics.html

2 Source for response choices: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2689572/

2 6 Market Street, Suite 3, Northampton, MA 01060 | 413-582-0425 | www.communicatehealth.com