Download:
pdf |
pdfAPPENDIX A
MATERIALS FOR THE WEB USABILITY TESTS
Re:
National
Science
Foundation
Study
of
Nonprofits
Dear
,
The
National
Science
Foundation
(NSF)
needs
your
help
to
learn
more
about
nonprofits
in
the
United
States.
This
summer,
to
better
understand
the
important
contributions
that
nonprofit
organizations
make
to
society
and
the
economy,
NSF
is
conducting
a
pilot
survey
online
with
a
selected
number
of
nonprofit
organizations.
The
pilot
will
test
our
process
for
collecting
information
from
nonprofits
prior
to
the
launch
of
a
nationally
representative
survey
in
2017.
The
survey
will
focus
primarily
on
research
performed
or
funded
by
the
nonprofit
sector,
including
research
in
the
areas
of
biomedical
and
health
sciences,
engineering,
physical
sciences,
social
sciences,
and
humanities.
We
have
finalized
the
pilot
survey
and
are
developing
a
web-‐based
tool
that
nonprofits,
such
as
yours,
may
use
to
complete
the
survey.
We
need
your
input!
Before
we
launch
the
pilot
survey,
we
are
interviewing
representatives,
such
as
yourself
or
your
designee,
and
asking
them
to
complete
some
tasks
with
the
survey
online
and
provide
our
usability
evaluator
with
feedback
about
the
online
system.
You
will
not
be
asked
to
share
any
information
about
your
organization.
NSF
has
partnered
with
ICF
International
to
conduct
the
sessions.
To
participate,
the
only
thing
you
need
is
a
computer
and
access
to
the
Internet.
A
member
of
the
ICF
project
team
will
call
to
discuss
the
study
and
schedule
the
telephone
interview
within
the
next
couple
of
days.
The
one-‐hour
interview
will
be
conducted
at
your
convenience
between
[month,
day,
year
to
month,
day,
year].
If
you
would
prefer
to
contact
ICF
directly,
please
use
the
information
below.
Lauren
Restivo
(301)
572-‐0468
lauren.restivo@icfi.com
If
you
have
any
questions
about
this
study,
please
contact
me
at
rbritt@nsf.gov
or
at
(703)
292-‐
7765.
Sincerely,
Ronda
Britt
Project
Officer,
Nonprofit
Research
Activities
Survey
National
Center
for
Science
and
Engineering
Statistics
National
Science
Foundation
NPRA
Usability
Testing
Recruitment
Script
NPO
Name:
[insert
name]
Address:
[insert
address]
City
and
State:
[insert
city/state]
Contact
Person:
[insert
point
of
contact]
Phone
number:
[insert
phone]
E-‐mail
address:
[insert
address]
NPO
Web
site:
[insert
website]
Hello,
my
name
is
.
I
am
calling
from
ICF
International
for
the
National
Science
Foundation.
May
I
speak
to
.
If
a
receptionist/gatekeeper
asks,
“What
is
this
in
regards
to?”
say:
Within
the
last
week,
the
National
Science
Foundation
sent
an
e-‐mail
to
about
a
new
survey
it
is
conducting
on
the
research
activities
of
nonprofit
organizations.
I
am
following
up
on
this
e-‐mail.
□
Yes
[Continue]
□
No
[If
the
person
is
unavailable.
Do
not
leave
a
message.
If
someone
other
than
the
person
answers,
ask
when
can
I
call
back?
___________________
]
If
you
are
transferred
to
the
person
by
a
gatekeeper/receptionist,
say:
Hello,
my
name
is
.
I
am
calling
from
ICF
International
for
the
National
Science
Foundation.
Thank
you
for
taking
my
call.
In
the
past
week,
you
should
have
received
an
e-‐mail
from
Ronda
Britt
at
NSF
about
a
new
survey
of
Nonprofit
Research
Activities.
As
mentioned
in
the
e-‐mail,
the
National
Science
Foundation
has
partnered
with
ICF
International
to
administer
the
survey.
We
have
developed
a
web
survey
and
we
need
your
help
to
test
it.
You
will
not
need
to
share
any
information
about
your
organization.
Your
feedback
will
enable
us
to
make
improvements
to
the
website
before
we
launch
the
pilot
survey
later
this
year.
We
would
like
to
schedule
a
1-‐hour
session
with
you
or
someone
you
designate.
The
session
will
be
conducted
by
telephone
and
we
will
invite
the
participant
to
view
the
web-‐based
survey
online.
Can
we
schedule
a
session
with
you
[or
a
designee
from
your
organization]?
□
Yes
[Continue
to
the
end
to
schedule
meeting
and
verify
contact
information.]
□
No
[Say:
“Thank
you
for
your
time.
Good-‐bye.”]
Recruiter:
try
to
schedule
the
session
with
the
Executive
Director/President;
Chief
Financial
Officer
(CFO)
or
designee;
Director
of
research,
chief
science
officer,
or
someone
in
the
research
department;
head
of
human
resources
or
designee.
Great!
I
would
like
to
set
up
a
date
and
time
for
the
session.
You/your
designee
will
need
access
to
the
Internet
to
view
the
survey.
(Note:
If
the
person
asks
how
her/she
will
connect
online,
tell
them
we
will
use
join.me.
They
will
receive
information
about
this
before
the
session.)
We
will
be
conducting
sessions
the
week
of
[Month,
day-‐day,]
2016.
Rebecca
Deitch
will
facilitate
the
session.
[As
appropriate,
say:
from
the
National
Science
Foundation
and
Arlen
Rosenthal,
the
ICF
project
manager,
will
be
online
listening
to
the
session.
They
will
be
happy
to
answer
any
questions
you
have
about
the
survey.]
**********************************************************
Date:
[insert
date]
Time:
[insert
time]
Confirm
the
following
information:
Phone
number:
[insert
phone]
E-‐mail
address:
[insert
e-‐mail]
Name/title
of
interviewee:
[insert
name
and
title
of
interviewee]
I
will
send
you
a
meeting
request
so
that
the
session
is
on
your
calendar
and
Ms.
Deitch’s
(facilitator);
24-‐hours
prior
to
the
session
Rebecca
Deitch
will
e-‐mail
you
a
final
reminder
with
the
phone
number
and
login
information
you
need.
Finally,
just
a
reminder
that
this
is
research
the
National
Science
Foundation
is
conducting
to
help
refine
a
new
online
survey
of
Nonprofit
Research
Activities.
Your
participation
is
completely
voluntary.
Your
name
and
that
of
your
organization
will
remain
confidential.
If
you
have
any
questions,
please
contact
me
at:
[State
your
name
again,
phone
number,
and
e-‐mail
address.]
Again,
thank
you
very
much
for
your
time
and
for
agreeing
to
participate
in
this
research.
We
will
see
you
online
[insert
date,
time,
and
location].
Confirmation
E-‐mail
Meeting
Invitation1
To:
[Interviewee’s
e-‐mail
address]
Subject:
Usability
Testing
for
the
NSF
Nonprofit
Research
Activities
Pilot
Survey
Location:
Remote:
[URL
to
join.me]2
Start
time:
[Month,
day,
2016
Time:
XX:XX
AM/PM]
End
time:
[Month,
day,
2016
Time:
XX:XX
AM/PM]
Body
of
e-‐mail:
This
meeting
invitation
serves
as
confirmation
of
the
time
and
location
for
our
online
meeting
to
discuss
the
National
Science
Foundation’s
Nonprofit
Research
Activities
Pilot
Survey.
As
appropriate
include:
One
or
two
staff
members
from
NSF
and
the
ICF
project
manager
will
be
listening
to
the
session.
Please
be
assured
that
neither
you
nor
your
organization
will
be
identified
in
any
report
we
prepare
for
NSF.
If
you
need
to
reschedule,
please
contact
me
by
telephone
at
[301-‐407-‐
XXXX]
or
by
e-‐mail
(rebecca.deitch@icfi.com)
as
soon
as
possible.
I
will
send
you
a
reminder
the
day
before
our
session
with
more
details
about
how
to
login
to
join.me.
Thank
you
for
your
time
and
I
look
forward
to
talking
to
you
online!
Rebecca
Deitch
NSF
Nonprofit
Research
Activities
ICF
International
[Phone
number]
1
The
ICF
usability
tester
will
send
this
e-‐mail
to
each
participant
within
24
hours
of
the
recruiter
scheduling
a
session.
2
Join.me
uses
a
range
of
local
numbers
rather
than
a
toll
free
number.
We
will
include
the
join.me
information,
including
URL
and
phone
numbers
in
the
body
of
this
e-‐mail.
Reminder E-mail 24 hours prior to online meeting
To: Interviewees email address
Subject: Join.me online meeting invitation for the NSF Nonprofit Research Activities Survey
Attachments: Vignette and consent form
ICF International, on behalf of the National Science Foundation, is conducting a usability
assessment of the Nonprofit Research Activities Survey that will be available for respondents to
answer online. During this session, you will be given some scenarios and asked to complete
some tasks online. We are doing this to ensure that you are able to easily log into the survey,
and can complete the information as requested. You may be asked to describe how you think
features should work, and what labels mean to you. We also want to hear when aspects of the
system do not function the way you expected and any suggestions you have to improve the
web interface.
This session will be conducted over the phone, and will use Join.me so that you can view and
control the screen. The session is limited to 1-hour, although an additional few minutes might
be needed to configure screen sharing. Audio and video of the screen you are looking at will be
recorded during the session to facilitate note-taking.
I have attached to this email two documents you will use during the meeting. The first is an
informed consent form. This form reviews the purpose of the usability test and acknowledges
your participation. During the meeting I will briefly go over the form and ask for your verbal
agreement to participate; this form is for your information.
The second document is the vignette that will facilitate your participation in the usability test.
The purpose of this usability test will help us to understand how users interact with our online
data collection system. During the usability test, I will ask you to play the role of an executive
or financial officer within the organization presented in the vignette. Since the purpose is
monitor the interaction with and reaction to the system presentation and features, the vignette
provides specific responses to the questions asked in the survey system.
Please follow the Join the meeting link below on [Month, Day, 2016] at [XX:XX a.m./p.m.
EST/CST/PST] when your scheduled session is about to begin. The session will begin when you
and I join the meeting. If you use a headset with microphone on your computer regularly you
can use the instructions below to listen and be heard, or you can call one of the provided
telephone conference lines using the Access Code provided below.
Thank you for your participation in this effort. If you have any questions or need to reschedule
your session, please contact me at rebecca.deithch@icfi.com or by telephone at (301) 5720556.
Join the meeting: https://join.me/xxx-xxx-xxx
On a computer, use any browser.
On a phone or tablet, launch the join.me app and enter meeting code: xxx-xxx-xxx
Join the audio conference:
Dial a phone number and enter access code, or connect via Internet.
By phone:
United States - Washington, DC
Access Code xxx-xxx-xxx#
+1 (202) 602-1295
By computer via Internet:
Join the meeting, click the phone icon and select “Call via internet.” A small download might be
required.
Sincerely,
Rebecca Deitch
(301) 572-0556
NSF
Letterhead
[Month]
[day],
2016
Dear
[insert
name],
We
greatly
appreciated
your
participation
in
the
usability
testing
session
on
[Month]
[day],
2016.
Your
thoughtful
feedback
will
definitely
help
us
move
forward
as
we
finalize
the
Nonprofit
Research
Activities
Survey.
We
anticipate
launching
the
pilot
survey
in
the
summer
of
2016.
Again,
on
behalf
of
NSF,
I
thank
you
for
your
support
and
valuable
contributions.
If
you
have
any
questions
or
would
like
to
be
updated
on
the
status
of
this
project
at
any
point
in
the
future,
please
do
not
hesitate
to
contact
me.
Sincerely,
Ronda
Britt
Project
Officer
Nonprofit
Research
Activities
Survey
National
Center
for
Science
and
Engineering
Statistics
National
Science
Foundation
4201
Wilson
Blvd.
Room
965
Arlington,
VA
22230
(703)
292-‐7765
rbritt@nsf.gov
Consent
Form
The
National
Science
Foundation
designs,
conducts,
and
sponsors
surveys
on
science,
engineering,
and
technology.
In
order
to
produce
the
best
information
possible,
the
National
Science
Foundation
routinely
evaluates
its
surveys.
You
have
volunteered
to
take
part
in
a
study
to
improve
one
of
the
National
Science
Foundation’s
surveys.
In
order
to
have
a
complete
record
of
your
comments,
your
voice
and
actions
on
the
screen
will
recorded.
We
will
use
this
information
to
improve
the
web
survey.
NSF
and
ICF
staff
directly
involved
in
this
research
project
will
have
access
to
the
recording.
The
recording
may
also
be
used
for
training
others
to
conduct
this
type
of
research,
and
in
presentations
to
professional
audiences.
This
study
is
authorized
by
law
(42
U.S.C.
1862
Section
3.a.6).
The
OMB
control
number
for
this
study
is
3145-‐0174.
I
have
volunteered
to
participate
in
this
study,
and
I
give
permission
for
my
tape
to
be
used
for
the
purposes
stated
above.
National
Science
Foundation:
Nonprofit
Research
Activities
Usability
Testing
Round
1
Funder
Testing
and
Participant
Information
Moderator
Observer(s)
Date
of
Test
Time
Name/Participant
ID
M
F
Gender
NPRA
Orientation
Script
(Funder)
Log
into
Join.me
Account
[Start
video
and
audio
recording]
Hello,
my
name
is
Rebecca,
and
I’ll
be
facilitating
today’s
session.
I
work
for
ICF
International
and
we
are
conducting
this
evaluation
of
a
new
survey
about
the
research
that
nonprofit
organizations
perform
or
fund
for
the
National
Science
Foundation.
Today,
I’m
going
to
ask
you
to
perform
some
tasks
on
a
wireframe
model
of
the
survey
that
is
running
on
my
computer.
Yesterday,
I
sent
you
a
reminder
e-‐mail
that
included
two
attachments:
(1)
a
letter
about
the
survey
asking
for
your
participation
and,
(2)
fictitious
information
that
is
presented
in
a
scenario
that
you
will
use
today
to
complete
some
tasks
I’ll
give
you.
You’ll
need
this
information
for
our
session
today.
As
we
go
through
the
tasks,
it’s
important
to
remember
that
I’m
not
testing
you;
rather
I’m
testing
the
survey
and
looking
to
see
if
there
is
anything
you
and
others
suggest
that
will
improve
the
survey
and
your
online
experience.
I
would
like
you
to
tell
me
what
you
are
thinking
as
you
perform
the
tasks.
Don’t
worry
about
hurting
my
feelings
if
you
don’t
like
a
particular
section
or
a
question
frustrates
you.
I
would
really
like
to
know
when
you
feel
this
way
because
it
will
help
us
improve
the
design
of
the
survey.
If
I
have
questions
for
you
I’ll
ask
you
about
them
as
we
go
along,
and
if
you
have
questions
for
me,
feel
free
to
ask.
Before
we
begin,
I’d
like
to
review
a
couple
of
things
and
then
ask
for
your
consent
to
continue,
and
to
record
the
session.
The
recording
will
help
when
I
prepare
my
notes
and
share
the
results
of
our
session.
[If
necessary,
remind
participant
that
he/she
received
a
consent
form
with
their
reminder
e-‐mail.]
First,
during
this
usability
test
you
agree
to
participate
in
an
online
session
using
your
computer
and
telephone.
During
the
session
I
will
ask
you
to
find
information
and
complete
some
tasks.
I
will
not
be
asking
you
for
any
information
about
your
organization.
Instead,
I’ll
provide
you
with
some
fictitious
information
that
you
can
use
to
complete
the
survey.
You
understand
and
consent
to
the
use
of
the
recording
by
the
National
Science
Foundation.
The
information
and
recording
are
for
research
purposes
only
and
your
name
and
organization’s
name
will
not
be
shared
with
anyone
outside
of
this
research
team.
You
understand
that
participation
is
voluntary
and
agree
to
immediately
raise
any
concerns
you
might
have.
Do
you
agree
to
participate
in
this
session?
Do
I
have
permission
to
record
your
voice
and
your
actions
on
screen?
[If
no,
thank
participant
and
end
the
interview.]
Thank
you.
Do
you
have
any
questions
before
we
begin?
Login
Before
you
can
login,
you
need
to
create
a
password.
Using
your
6-‐digit
id
number
as
your
username,
go
ahead
and
create
a
password.
[Once
they
are
logged
in,
ask
the
following
and
observe
and
record
what
R
clicks
on.]
What
are
your
thoughts
on
the
login
process?
Do
you
have
any
feedback
on
the
links
on
the
welcome
page?
Are
they
clear
or
confusing?
Did
you
expect
to
see
something
different?
Begin
Tasks
General
Information
Using
the
information
that
you
received
about
the
R.
B.
Milford
Foundation,
please
fill
out
questions
1-‐4.
Do
you
feel
like
any
of
the
questions
were
difficult
to
fill
out?
Now
click
on
the
tab,
Research
Performed
and/or
Funded
Research
Performed
and/or
Funded
(Needs
to
be
Q5
No,
Q6
YES)
How
would
you
answer
these
questions
for
the
R.
B.
Milford
Foundation?
Did
you
read
the
information
above
the
questions?
Do
you
feel
like
these
questions
fit
with
the
guidance
above
or
not?
Did
the
guidance
above
help
you
answer
the
questions
or
not?
Now
go
ahead
and
fill
out
the
rest
of
the
questions,
and
please
remember
to
tell
me
what
you
are
thinking
as
you
fill
them
out.
[Interviewer:
observe
and
record
how
respondent
is
moving
from
question
to
question
and
from
field
to
field
within
questions.]
Use
these
probes
as
appropriate:
I.
I
noticed
that
you
changed
the
number
there.
Can
you
tell
me
what
you
were
thinking
about?
2.
I
haven’t
heard
you
speak
for
a
while.
Can
you
tell
me
what
you
are
thinking?
3.
You
clicked
on
X.
What
were
you
expecting
to
happen?
4.
If
you
were
to
click
on
X
right
now,
what
do
you
think
would
happen?
5.
[If
errors
are
triggered],
What
are
your
thoughts
on
this
message?
Is
it
clear
or
confusing?
Contact
Information
and
Survey
Time
Please
fill
out
your
contact
information
using
an
alias.
For
time
to
complete
the
survey
you
can
put
any
number.
Post-‐Test
Questions
What
are
your
thoughts
about
the
survey?
What
were
your
favorite
things
about
it?
Is
there
anything
you
would
change
about
it?
Did
you
get
frustrated
at
any
point?
If
not
mentioned
above
ask:
What
did
you
think
of
plus
sign
and
drop-‐down
information
under
each
field
in
#14?
Do
you
think
you
would
be
likely
to
print
or
save
a
copy
of
the
survey
for
your
records?
If
so,
how
would
you
do
that?
Ask
the
final
question
below
only
during
Round
2
testing.
What
are
your
thoughts
on
the
auto-‐sum
feature
for
the
question
totals?
Is
this
helpful
or
not?
What
if
you
needed
to
change
a
total
to
something
that
did
not
equal
the
auto-‐sum
(for
instance
if
you
could
not
break
out
all
of
the
sub-‐totals.)
Wrap-‐up
That
concludes
our
test
today.
Thank
you
so
much
for
your
time
today.
Instructions:
You
have
agreed
to
participate
in
a
usability
test
of
the
National
Science
Foundation’s
Nonprofit
Research
Activities
Survey.
The
test
will
explore
the
ability
of
users
to
interact
with
our
customized
data
collection
system.
In
order
to
get
an
understanding
of
how
different
users
respond
to
the
survey,
we
are
asking
that
you
role
play
the
part
of
an
executive
or
financial
officer
within
the
R.B.
Milford
Foundation,
a
nonprofit
organization.
Please
use
the
information
below
to
respond
to
the
items
on
the
survey.
If
you
have
any
questions,
please
ask
your
interviewer
before
you
get
started
with
the
usability
test.
General
Information
Organization
Name:
R.B.
Milford
Foundation
The
R.
B.
Milford
Foundation
is
a
leading
supporter
of
social
science,
engineering,
and
medical
research.
FY
2015
IRS
Form
990-‐PF
Total
Expenses
(line
26):
End
of
your
2015
Fiscal
Year
Total
number
of
Full
Time
Equivalent
Employees:
$65,984,000
December
31,
2015
42
Research
Performance
/
Funding
This
foundation
does
not
perform
any
research,
rather
they
fund
other
qualified
individuals
and
organizations
to
do
the
work
with
funds
from
the
foundation’s
endowment.
Research
Funding
Research
Award
and
Sub-‐Award
Funding
Research
grants
or
contracts
$21,500,000
TOTAL
$21,500,000
Research
Funding
by
Type
of
Organization
Kirkland
University
(University)
Combination
of
funds
provided
to
(For-‐profit
businesses)
•
•
$15,000,000
$6,000,000
Abalon
Pharmaceutical
Green
Climate
Engineering,
Inc.
West
Virginia
Office
of
Economic
Development
(State
government
agency)
TOTAL
$500,000
$21,500,000
Amount
Funded
By
Specific
Research
Field
Oil
fracking
on
coal
industry
(Natural
resources)
Combination
of
funds
provided
to:
•
•
$500,000
$6,000,000
Nanorobots
for
medicine
(Biotechnology)
Development
of
antibiotic
treatments
(Immunology)
Development
of
a
prototype
to
clean
carbon
dioxide
from
the
atmosphere
(Chemical
/
mechanical
engineering)
$5,000,000
Family
income
and
children’s
SES
attainment
(Economic)
$10,000,000
$21,500,000
TOTAL
Research
Funding
by
Source
Milford
Foundation
Endowment
(Internal
funds)
TOTAL
$21,500,000
$500,000
Contact
Information
and
Survey
Time
Use
your
own
contact
information
Please
provide
an
approximate
time
to
complete
FORM
APPROVED
OMB
NO.
XXXX-‐XXXX
EXPIRATION
DATE:
XX/XX/2016
NSF Survey of Nonprofit Research Activities
Dear Nonprofit Executive/Research Director:
Nonprofit organizations are an integral part of our economy, and the National Science Foundation (NSF) wants to
hear from you.
This survey focuses on research performed or funded by U.S. nonprofit organizations. Your answers will help
policymakers understand the prevalence and landscape of research activities within the nonprofit community.
The types of information requested in the survey are generally available for nonprofits from the following offices
within a nonprofit organization:
•
Chief Financial Officer / Finance Department
•
Human Resources
•
Director of Research / Chief Science Officer / Research Department
Your participation in this survey helps NSF with its task to compile information on the national level of research and
development activity across the government, business, academic, and nonprofit sectors. NSF is authorized to collect
this information under the National Science Foundation Act of 1950, as amended. Your organization’s response is
voluntary.
Questions?
Technical support:
ICF International
Support@NPRAsurvey.org
877-xxx-xxxx
Survey questions:
Ronda Britt, Project Officer
National Science Foundation
rbritt@nsf.gov
Please submit your response by Month dd, 2016
Thank you for contributing to this study of our Nation’s nonprofit research activities!
Under
the
Paperwork
Reduction
Act
of
1995,
individuals
do
not
have
to
respond
to
a
collection
of
information
unless
such
collection
displays
a
valid
Office
of
Management
and
Budget
(OMB)
control
number.
The
OMB
control
number
for
this
survey
is
displayed
at
the
top
right
of
this
page.
The
time
required
to
complete
these
responses
is
estimated
to
be
approximately
10
hours.
This
includes
the
initial
identification
of
the
research
activities,
contracts/grants,
and
personnel,
as
well
as
the
final
review
and
verification
of
your
organization’s
response.
If
you
wish
to
comment
on
the
time
required
to
complete
this
survey,
please
contact
Suzanne
Plimpton
of
NSF
at
703-‐292-‐7556,
or
e-‐mail
splimpto@nsf.gov
using
"Survey
of
Nonprofit
Research
Activities"
as
the
subject.
Thank
you
for
your
participation.
1
GENERAL INFORMATION
1
What
is the name of the organization for which you are reporting?
For the purposes of this survey, your organization is defined as all units included in your Internal Revenue
Service Form 990 filing.
The name of your organization listed in IRS Form 990-PF is:
| File Type | application/pdf |
| File Title | Appendix A |
| Author | Morrison, Rebecca |
| File Modified | 2016-05-03 |
| File Created | 2016-05-03 |