Attachment X - Usability Test Round 2 Report

Attachment X. Final Analysis Report and Recommendations
for Revisions to the FoodLogger from the
Second Round of Usability Testing

Report on FoodLogger usability evaluation (Round 2)
Lin Wang, Shelley Feuer, Alda Rivas, Heather Ridolfo
FoodAPS-2 project team of Census Bureau and NASS
July 29, 2021

Executive Summary
The Round-2 FoodLogger usability evaluation was conducted from June 22 to July 30, 2021. Five
primary food shoppers from five households participated in the study, with two being 60+ old and the
other three being recipients of food assistance programs. The evaluation design calls for an 8-day study
including three major parts: FoodLogger use training, 7-day food acquisition reporting, and lab-based
usability testing. FoodLogger of version 3 was the data collection instrument under evaluation. Three
use cases (a combination of Food-at-home (FAH) event and a Food-away-from-home (FAFH) event, a
FAFH event, and a school-meal event) and 18 critical tasks (a task such that failure to complete it would
result in measurement errors) were tested. Qualitative and quantitative methods were applied for data
analysis. Usability issues will be classified as high (H), medium (M), and low (L) priority. H issues are
those that prevent a task from being completed; M issues prolong task completion; L issues do not
impact effectiveness and efficiency of task completion but may affect user’s satisfaction.
Participants attained adequate skills through training, carried out 7-day food acquisition
reporting in a daily living setting (except for one participant due to technical issues), and completed use
cases in the lab-based usability testing. In general, participants reported that it was easy to report food
acquisition information using FoodLogger. Six usability issues with high priority, 3 with medium priority,
and 3 with low priority were identified. In particular, these three functions need further improvement to
reduce potential measurement errors in survey data: barcode scanning, PLU entry, and
weight/volume/piece data entry. Details were documented in the body of the report.

FoodLogger evaluation report – Round2

2

1. Evaluation Objective
The objective for the present usability evaluation of the FoodLogger native smart phone app
(FoodLogger) is to assess how potential respondents enter food acquisition data into the FoodLogger in
terms of effectiveness, efficiency, and satisfaction. The effectiveness of the FoodLogger will be
measured by the success of data entry and the accuracy of entered data; efficiency will primarily be
measured by the time taken to enter data; and satisfaction will be measured by respondent-reported
satisfaction which includes a user’s perception of difficulty, the extent to which their expectations are
met, and a user’s emotional response to data entry. It is hypothesized that effective data entry will
prevent missing or erroneous data and consequently minimize measurement errors, efficient data entry
will reflect lower respondent burden, and satisfaction with the data entry experience will help sustain
respondents’ participation in the Second National Food Acquisition and Purchase Survey (FoodAPS-2).

2. Evaluation Methods
Participating households
Seven households were recruited via advertisement and word-of-mouth. Five households
participated in the study, while the other two households dropped off the study. The characteristics of
the participating households are listed in Table 1. All participants were given a Disclaimer that states the
purpose of the study, data to be collected, rights as a participant, and the statutory authority under
which the study was conducted. Written consent to participating the study was obtained from each
participant prior to the commence of the study.

Table 1. Participants Demographic Characteristics

Household Size (person)
School Children Participation (11-15year-old)
WIC Recipient
SNAP or Other Government Food
Assistance
Residence RUCA code*
Primary Shopper Age (year)
Primary Shopper Gender
Primary Shopper Education

Race

FoodLogger evaluation report – Round2

Household
8
2
No

Household
9
4
Yes

Household
10
7
Yes

Household
12
2
Yes

Household
13
1
No

No
No

Yes
Yes

Yes
Yes

No
Yes

No
No

1
68
M
Some
college, no
degree

1
34
F
Some
college, no
degree

1
40
F
Graduate
degree

1
31
F
Some
college, no
degree

10
64
M
Some
college, no
degree

Asian

White

White

Black or
African
American

American
Indian or
Alaska
Native

3

Hispanic Ethnicity
Smartphone Use Length (year)
Smartphone Use Frequency
Using Map on Smartphone
Three most frequent uses of
smartphone

No
≥2
Everyday
Yes
Phone
calls,
google,
email

No
≥2
Everyday
Yes
text, email,
social media

Yes
≥2
Everyday
Yes
messaging,
calls,
research
online

No
≥2
Everyday
Yes
email, web
browsing,
phone calls

No
<1
Everyday
Yes
phone,
direction,
google,
facebook

* Rural-Urban Communicating Area Codes (https://www.ers.usda.gov/data-products/rural-urbancommuting-area-codes/)
Product evaluated
The product evaluated was FoodLogger version 3, developed by Westat. FoodLogger is a native
mobile application that serves as a data collection instrument for FoodAPS-2.

Evaluation design
The evaluation design calls for an 8-day study including three major parts: FoodLogger use
training, 7-day food acquisition reporting, and lab-based usability testing. Table 2 shows the timeline of
the study. Due to COVID-19 pandemic, the study was conducted virtually via MS Teams, emails, and
telephone calls. Specific methods for each of the three parts will be presented in its dedicated section
below.

Table 2. Timeline of major testing activities
Component
Being introduced to the study
Receiving a Disclaimer
Signing a Consent Form
Completing a demographic
questionnaire
Installing FoodLogger
Data entry training
Field data entry
Field data entry debriefing
Lab-based usability testing
Signing incentive voucher

FoodLogger evaluation report – Round2

Day
1
x
x
x
x
x
x
x

Day
2

Day
3

Day
4

Day
5

Day
6

Day
7

x

x

x

x

x

x

x

Day
8

x
x

4

3. FoodLogger Training
Training design
The FoodLogger training consisted of three components in sequence: Basic concepts in
FoodAPS-2 data entry, FoodLogger installation, and data entry practices. A training courseware in the
form of MS PowerPoint (Appendix A) was originally developed, and revised for Round 2 testing, by the
study team. The revision incorporated additional materials on basic concepts and screen shots provided
by Westat. All participants, including primary shoppers and participating school children (11-15 years
old), received training. The child participated in the training alongside the parent. The training started
with introducing the basic concepts. Then, under trainer’s guidance, the trainee downloaded
FoodLogger to his/her smartphone. Lastly, the trainee practiced entering specific food information (e.g.,
PLU code) into FoodLogger. One debriefing on FoodLogger installation and another on data entry
training were conducted to assess training effectiveness.

Major findings and recommendations
Debriefing was not conducted on H13 due to technical issues.
Basic concepts:
1. Participants could understand most concepts covered in training.
2. Three out of four participants understood the concept of “Food acquisition.” This is an
improvement from Round 1.
3. Two out of four participants could NOT clearly distinguish “Stop” from “Food event.”
Recommendation: Instruct respondents to understand and differentiate the two concepts, for
example, use some examples.
4. Three out of four participants could adequately distinguish “Combo meal” from “single meal
item.”
5. Average training time on the Basic Concepts part was 21.8±8.6 min. Total training time
(including FoodLogger installation) was about 60 min for participants under the age of 60. It
took 110 min for H8 (68 years old) to complete the training. Technical proficiency appears a
major obstacle for slower learners.
FoodLogger installation:
1. Participants could successfully install FoodLogger on their smartphone under trainer’s guidance.
2. Two out of four participants commented that configuration was not straightforward and
sometimes confusing. Participants sometimes got lost in configuring notification and GPS
settings because certain smartphones did not behave as predicted. We recommend that the
trainer walk respondents through configuration phase and verify configuration after
installation.
Data entry practice:

FoodLogger evaluation report – Round2

5

1. Participants could generally carry out basic data entry tasks right after learning the basic
concepts about FoodLogger. Those basic tasks include text entry, bar code scanning, PLU code
entry, taking a picture of and uploading a paper receipt.
Training School children:
1. 11-15-year-old school children attended training sessions, but were not required to install the
app nor practice data entry. Some of them volunteered for app installation and data entry
practice. They were able to learn the basics at least as good as their parents. We recommend
that all eligible household members participate in trainer-conducted training.

4. Lab-based Usability Testing
Testing design
USE CASES: Three use cases were tested in the lab-based usability testing: (1) A combination of
Food-at-home (FAH) event and a Food-away-from-home (FAFH) event (Appendix B), (2) a FAFH event
(Appendix C), and (3) a school-meal event (Appendix D). The three use cases are designed such that each
critical task (described below) will be performed at least once during testing. School-age participants
between 11 and 15 years old were tested on use case 3.
CRITICAL TASKS: A critical task refers to a task such that failure to complete it would result in
measurement errors (e.g., scanning a bar code). Eighteen critical tasks, as listed in Table 3, were
identified for successful data entry using FoodLogger, and were tested in the lab-based usability testing.

Table 3. Critical tasks for data entry using FoodLogger
Task #
1
2
3
4
5
6
7
8
9
10
11
12

Task
Start a day
Select a food stop from a list of stops identified by
FoodLogger
Add a food stop manually
Select a food event
Add a food event manually
(FAH food item:)
Enter item name
Enter weight/volume/size
Enter number of items
Enter payment
(FAFH combo food item:)
Select "combo meal" button
Enter meal name
Enter payment

FoodLogger evaluation report – Round2

Sub-task

barcode, PLU, text

pay by single item or multiple items;
payment methods

payment methods
6

13
14
15
16
17
18

Enter number of items
Enter individual meal items
(FAFH individual food item:)
Select "individual item" button
Enter meal name
Enter number of items
Enter payment

payment methods

PERFORMANCE MEASURES: The following metrics were used to assess participants’ data entry
performance.
a) Data entry accuracy – The extent to which the entered data are correct.
b) Data entry time – Duration between the start and end of a use case.
c) Navigation – The extent to which participant’s actual navigation path deviates from the
optimal path.
DATA COLLECTION: A protocol was followed to carry out the lab-based usability testing
(Appendix E). Methods for data collection include:
a) Passive observation
b) Thinking aloud
c) Retrospective Debriefing – Focused on critical design components, e.g., language
comprehension. A debriefing guide was followed to cover critical actions of interest.
DATA ANALYSES: Quantitative data were analyzed with descriptive and/or inferential statistics
accordingly. Qualitative data were summarized to identify common usability issues and their causes.
Usability issues were classified as high (H), medium (M), and low (L) priority. H issues are those that
prevent a task from being completed; M issues prolong task completion; L issues do not impact
effectiveness and efficiency of task completion but may affect user’s satisfaction (e.g., imperfect text
formatting).

Summary performance measures
All participants were able to complete assigned use cases. Table 4 shows quantitative measures
for food information entry performance. Those quantitative measures are defined as follows:
•
•
•
•

Use-Case-1 completion time (min): The time between starting a food event and before
submitting a receipt.
Use-Case-1 Average time for entering a single food item (min): (Use Case 1 completion
time)/(Number of food items in Use Case 1).
Use-Case-1 paper receipt upload time (sec): The time between selecting “Yes, I have a paper
receipt” and completing uploading the image of the paper receipt.
Number of Use-Case-1 food items not reported: The number of food items that were not
reported in Use Case 1.

FoodLogger evaluation report – Round2

7

•
•
•

•

•

•
•
•

Number of Use-Case-1 non-food items reported: The number of food items that were not
reported in Use Case 1.
Use-Case-1 deli lunch reported: A dichotomous indicator on reporting the deli purchase, with Y
indicating the purchase being reported, and N otherwise.
Optimal food name entry rate in Use Case 1: A ratio of the number of food items entered into
FoodLogger using optimal name entry method over the number of total food items entered into
FoodLogger (e.g., barcode scanning for a barcoded item is an optimal method while text entry
not)
Optimal packing selection rate in Use Case 1: A ratio of the number of food items entered into
FoodLogger with optimal packaging selection over the number of total food items entered into
FoodLogger (e.g., Carton category is optimal for milk)
Optimal weight/volume selection rate in Use Case 1: A ratio of the number of food items entered
into FoodLogger with optimal weight/volume selection over the number of total food items
entered into FoodLogger (e.g., Pound/ounce is optimal category for bulk coffee)
Use-Case-2 completion time (min): The time between starting a food event and before
submitting a receipt.
Use-Case-2 electronic receipt upload time (sec): The time between selecting an uploading
method and completing uploading the receipt.
Use-Case-3 completion time (min): The time between starting a food event and completing the
event.
Table 4. Food information entry performance summary

Use-Case-1 completion time (not
including paper receipt uploading and
nor deli lunch purchase) (min)
Average time for entering a single food
item in Use Case 1 (min)
Use-Case-1 paper receipt upload time
(sec)
Number of Use-Case-1 food items not
reported
Number of Use-Case-1 non-food items
reported
Use-Case-1 deli lunch reported
Optimal food name entry rate in Use
Case 1
Optimal packing selection rate in Use
Case 1
Optimal weight/volume selection rate in
Use Case 1

FoodLogger evaluation report – Round2

Household Household
8
9
62
33

N

Household
10
30

Household
12
20

Household
13
33

3.6

1.9

1.7

1.2

2.8

30

30

70

13

52

1

1

0

1 Timed out

1

0

0

2

0.94

Y

1.00

N

0.94

N

0

Timed out
1.00
0.50

0.88

1.00

0.85

1.00

1.00

0.71

1.00

0.81

0.94

0.91

8

Use-Case-2 completion time (not include
e-receipt uploading) (min)
Use-Case-2 electronic receipt upload
time (sec)

17
Unable to
do it

Use-Case-3 completion time (min)

NA

8

8

123 (with
TA
assistance)
5

206 (with
TA
assistance)
5

7

13

112 (with Unable to
TA do it
assistance)
3 NA

Usability issues with HIGH priority
Enter food item name – Barcode/PLU
Issues: (1) Two 60+ participants had difficulties in aiming the camera at a barcode for scanning.
(2) Not all barcodes/PLU had corresponding food item names available in the database. While the
warning message was visible, its language was not clear enough to lead participants to the next steps.
(3) The food name display was not visually salient enough to be perceived by the participant.
Recommendations: (1) Enhance training of barcode scanning and PLU entry for respondents of
60+ old. (2) Improve the warning message to include (a) where to type food name, (b) tapping the Next
button after entering the food name. (3) Consider adding a beep sound or using a pop-up window when
the warning message is displayed. (4) Enlarge and bold food name text.
Enter food item name – Text
Issues: While the type-ahead feature provided some convenience for typing, participants were
burdened with trying to find an exact match to the name of the food item to be reported. Participants
became frustrated when a food name couldn’t be matched.
Recommendations: (1) Improve the database to include generic food names only (e.g., pasta
instead of a brand name of a pasta product. (2) Instruct the respondent to enter just generic food
names.
Make packaging selection
Issues: Four out of five participants selected some categories which were not intended by the
designer, resulting in a cascade of undesirable behaviors: wrong measurement and measurement unit.
For example, selection of carton packaging for a pack of sea salt led the participant to volume measures
instead of weight measures. Its root causes appear to be (1) the categories are not exhaustive nor
mutually exclusive (for example, carton can be a carton of eggs or a carton of milk), and (2)
categorization is more or less subjective judgement.
Recommendations: Design a solution in the component of weight/volume/piece to enable the
respondent to select adequate measurement. Further discussions are warranted. Here are a few
proposed solutions: (1) adding more categories and ensuring that they’re mutually exclusive, (2) an open
text option.
Enter weight/volume/piece
Issues: (1) inadequate measurement resulted from packaging selection. (2) No measurement
could be found for FAFH items entered in FAH branch.
Recommendations: As one possible solution, in addition to default measurement for a particular
packaging type, provide an “other” option that lists all other possible measures. Another proposal is to
provide all unit options regardless of which packaging the respondent chose in case they choose the
wrong one.
FoodLogger evaluation report – Round2

9

Combo meal
Issues: One participant finished reporting a combo meal before individual food items got
entered. One possible reason could be that in the order for a regular food item data entry, food cost was
the last piece of data to be entered but for a combo meal it occurs before adding the individual items.
Recommendations: Reconcile data entry order for both individual food item and combo meal:
Ask the respondent to enter all individual food items in a combo meal, before entering the cost.
Uploading an electronic receipt
Issues: Only one out of five participants completed the task without major issues. Two
participants were unable to carry out the task. The other two participants needed assistance. One
participant complained about the multiple steps involved in the task. Problems included finding the
receipt email, downloading the receipt, find the receipt in the phone, taking a screenshot of the receipt,
uploading the receipt or receipt screenshot image, and switching back and forth between FoodLogger
and other apps (e.g., email).
Recommendations: Provide detailed training on uploading an electronic receipt if the receipt is
highly desired. The training should provide step-by-step instructions on searching for the receipt in
email, downloading receipt to the phone, taking a screenshot of the receipt, uploading the receipt image
or the receipt document.

Usability issues with MEDIUM priority
No display of weight/volume/piece information
Issues: Sometimes, FoodLogger skipped the steps of packaging and weight/volume/piece data
entry after barcode scanning. It was unclear if the information was automatically captured from
database by FoodLogger or a defect in FoodLogger.
Recommendations: If the weight/volume/piece information was indeed captured by
FoodLogger, display the information and allow the respondent to make correction is the measurement
is incorrect. If this is a programming defect, it needs to be fixed.
Combo meal – name vs. content
Issues: Participants, particularly school children, were unclear that in addition to a name of a
combo meal, they need to report all the items in a combo meal, and then got confused.
Recommendations: Since combo meal name is not a piece of essential information, suggest that,
instead of asking the participant to enter a name, FoodLogger automatically generate a combo meal
name and ask the participant to enter individual food items only.
Food type selection for FAFH
Issues: Participants sometimes could not select an appropriate food type. For example, in the
use case of school lunch, the participant could not find an appropriate category for “a slice of beef.”
Recommendations: Improve the selection list to cover all possible types of food.

Usability issues with LOW priority
Enter cost or payment
FoodLogger evaluation report – Round2

10

Issues: One participant suggested that FoodLogger automatically calculate the split of cost
among different payment modes, to reduce burden. For example, if the participant selected payments
of cash and EBT card, after entering total payment and cash amount, EBT amount is automatically
populated.
Recommendations: For sponsor’s consideration.
Uploading a long paper receipt
Issues: Few participants were cognizant of taking multiple segment images of a long paper
receipt.
Recommendations: During training, emphasize the good practice of taking multiple segment
images of a long paper receipt and provide a practice to the respondents.
Trivial stop detection
Issues: A few participants complained that too many trivial stops were detected, for example, a
stop at a traffic light.
Recommendations: Adjust the threshold for no-move time to reduce the number trivial stops.

Non-usability issues
Underreporting in mixed-type food event
Issues: Three out of five participants forgot to report the lunch meal paid separately in Use Case
2. This observation suggests that respondents may have a tendency to forget smaller food acquisitions
in a food event with more than one type of food (i.e., FAH vs FAFH). This could also indicate that
reporting multiple food acquisitions at one time can be burdensome and error-prone.
Recommendations: Emphasize such scenarios in training.

5. 7-Day Food Acquisition Reporting
Study design
This component was executed in the participant’s daily living setting on his/her own, without
TA’s observation. The participant was instructed (Appendix E) to enter information for all their acquired
foods into FoodLogger every day, and to log all the problems and difficulties encountered during data
entry. A standard log form was provided to the participant. The TA was available to provide assistance
over the phone if needed, though none of the participants contacted the TA. Upon completion of the 7Day reporting, a semi-structured debriefing session was carried out to collect participants’ experiences
in food acquisition reporting using FoodLogger in a daily living setting.

Summary of findings from debriefing
Due to technical issues, H13 did not complete this task by the time of this report writing. Data
reported below are from the other four participants. Tables 5-11 summarize responses from the
participants on debriefing questions. See Appendix F for debriefing questions.
FoodLogger evaluation report – Round2

11

Table 5. Self-reported easiness of task performance (count: person)

Confirming a Stop
Manually adding a Stop
Adding a Food Event
Scanning a barcode
Entering a PLU
Entering a food item name
Entering size/weight/volume
Reporting school meal
Entering cost for a food item
Entering cost for entire food
event
Choosing method of payment

Extremely
easy
3
3
3
1
4
1
2

Easy

Neutral

1
1
1
1

2

2
3

1
1

2

Difficult

1
1
1

Extremely
difficult

1
1

4

Table 6. Use of the "type-ahead" feature (count: person)
Yes
3

Did you use the "type-ahead" feature?

No
1

Table 7. Reaction to notifications (count: person)
Are you bothered by the notifications?

Not at all
3

Somewhat Moderately Very
1

Extremely

Table 8. Three greatest challenges
Household 8

Challenge 1
Remember doing it

Challenge 2

Scanning

Household 9

Time

Remembering to do it

Challenge 3
Being frustrated when it
doesn't work
Big grocery trips

Household 10

Entering the weight and
volume

Identifying location accurately
or populate the location

Trouble with address on
locations from other cities

Household 12

Remembering to log it
Table 9. Desired incentive amount (count: person)

How much money would you think appropriate?

FoodLogger evaluation report – Round2

Depends
1

$5
2

$7-$8
1

12

Table 10. Comfortableness in sharing GPS location (count: person)
Completely
not
How comfortable were you in sharing your GPS
location during this study?

Somewhat
not
1

Neither
yes nor no

Somewhat Completely
yes
yes
3

Table 11. Food acquisition in the past 30 days
Grocery store, in-person
Grocery store, order online
Big Box Store, in-person
Big Box Store, order online
Restaurant, eat in
Restaurant, order online
Friend or family member’s house
Food from charity
Other

Household 8
x

Household 9
x

Household 10
x

Household 12
x

x

x

x

x

x
x
x

x
x
x

x
x
x

x
x
x
x

Table 12 summarizes responses from the school-age participants (11-15 years old). In general,
responses were positive. The school-age participants could all enter data successfully.
Table 12. Responses from school-age participants
Household 9
Training attendance
Data entry completion in Use
Case 2
How do you like using
FoodLogger?
On a 5-point scale, 1 being
most difficult and 5 being
easiest, how do you rate your
experience using
FoodLogger?
Challenge in using
FoodLogger
Other comments

Household 10

Household 12

Yes
Yes

Yes
Yes

Yes
Yes

Using FoodLogger
was fine. Difficult to
remember. Took time
to enter data.
1

Very easy.

I like it. It was good.

4, because we went
to a carnival in a
parking lot, we didn’t
know which address
to choose.
Location was really
hard.
NA

4, wasn't hard.

Difficult to
remember.
GPS did not track
movement properly.

FoodLogger evaluation report – Round2

Having to switch
between accounts.
NA

13

6. Summary
The findings from the Round-2 FoodLogger Usability Testing indicate that the following four
areas need modification to improve respondents’ performance of these tasks: (1) barcode scanning, (2)
PLU entry, (3) weight/volume/piece data entry, (4) electronic receipt uploading. Compromised
performance on weight/volume/piece data entry could increase measurement errors, while
compromised performance on barcode scanning and PLU entry or electronic receipt uploading could
cause respondents’ frustration, missing data, and drop-offs. After the aforementioned four issues are
addressed, another round of lab-based usability testing focused only on these four tasks is
recommended to assess improvement in performance.

7. Limitations
Participant’s performance of using FoodLogger is associated with training they received. The
training was developed and conducted by the study team as requested and may be different from the
training that potential respondents will receive in the FoodAPS-2 field test or formal survey. Thus,
findings presented in this report may not be generalizable to a population receiving different training
and need to be interpreted with caution. Due to small sample size, it is possible that some usability
issues were not discovered in this round of testing or that some of the observed usability issues may be
associated with certain types of potential respondents.

FoodLogger evaluation report – Round2

14

Appendix A: Training Courseware

7/21/2021

FoodLogger Training
(Round 2)
FoodAPS‐2 Study Team of U.S. Census Bureau and NASS

June 25, 2021

This presentation is released to inform interested parties of research and to encourage discussion.
The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau.

1

1

Basic concepts
What is FoodLogger?

A mobile app running on a smartphone.
You will use it to report food and drink you get (for purchase or free) over 7 days.
The app uses GPS to track your smartphone’s movement, captures stops, and allows the user to
create food events and add food items for each event.
You will perform 3 main tasks with the FoodLogger: complete the Income Questionnaire, Profile
Questionnaire, and the 7‐day Food Log.

What are foods?

Anything that you can eat or drink.

What to be reported in 7 days?

Any food or drink you get (for purchase or free) ONLY during your 7‐day study week. You may or may
not eat those food and drink.
Do NOT report food in your home that you got (for purchase or free) before you began participating
in this study.

2

2

1

7/21/2021

Basic concepts (con’t)
DAY

STOP

Day 1

Stop 1

Day 3

Day 2

Stop 2

Stop 3

FOOD EVENT

Day 4

Stop 4

Food
Item 1

Day 6

Day 7

Stop 5

Food
Event 1

FOOD ITEM

Day 5

Food
Item 2

Food Event 2

Food
Item 3

Food Event 3

Food
Item 4

3

3

Basic concepts (con’t)
A food item has:

• Name (e.g., Apple)
• Weight/Volume/Size (e.g., 5 lb, 20 fl oz, 1 serving)
• Cost (free item has the cost of $0)

Multiple same food items also have:

• Count/quantity (e.g., 5 bottles of water)

A food event has:

• Payment (free event has a payment of $0)

4

4

2

7/21/2021

How do you report food/drinks you got?

5

6

6

3

7/21/2021

7

7

8

8

4

7/21/2021

9

9

1

10

5

7/21/2021

1

11

12

12

6

7/21/2021

Installing FoodLogger: iPhone Users
Go to the App Store on your phone
Search for “Westat FoodLogger”
Download the free app
Open the FoodLogger app
Enter your household’s unique PIN to login
Read through the tutorial pages carefully and follow the on‐screen
prompts. The app will guide you through the set‐up process
including allowing the app to track your location and enabling push
notifications
13

13

Download FoodLogger: iPhone
Users (con’t)

14

14

7

7/21/2021

Installing FoodLogger: Android Users
Go to the Google Play store on your phone
Search for “Westat FoodLogger”
Download the free app
Open the FoodLogger app
Enter your household’s unique PIN to login
Read through the tutorial pages carefully and follow the on‐screen
prompts. The app will guide you through the set‐up process
including allowing the app to track your location and enabling push
notifications
15

15

Download FoodLogger: Android
Users (con’t)

1

16

8

7/21/2021

Data Entry Workflow

17

17

Practice (I) – Food Items
1.
2.
3.
4.

Start a day
Add a food stop manually
Add a food event manually
Enter item name

• Barcode
• PLU
• Text

5. Enter weight/volume
6. Enter number of items
7. Enter payment
• Single item
• Multiple items
• Payment modes

18

18

9

7/21/2021

Practice (II) – Receipt uploading

19

19

Practice (III) ‐ Meals
1. Breakfast at Starbucks
• One Blueberry Scone ($2.40)
• One Banana ($0.90)
• One Caffe Latte (tall, $3.50)

2. School lunch (combo, free)
•
•
•
•

Chicken Caesar Wrap
Celery Sticks
One Fresh Orange
1% White Milk (Half Pint)

20

20

10

7/21/2021

Profile and Income Information
Complete these two
questionnaires by yourself.
Use FAKE data to complete
these to questionnaires.
Do not enter your actual
personal information!
21

21

11

Appendix B: Use Case 1 Food-at-Home Event plus Food-Away-from-Home Event
Note: Actual food items may vary slightly among participating households depending on store inventory
at the time of food purchase. However, the same critical tasks were covered across households.
Purpose: To test FAH + FAFH
Critical tasks tested:
• Start a day
• Select a food stop from a list of stops identified by the online map
• Select a food event
• Enter food item name:
o Text
o Barcode
o PLU
• Enter weight/volume/etc
• Enter number of items (quantity)
• Enter payment information
o Pay by single mode
o Pay by multiple modes
Event Set-up:
• Food came from supermarket (see delivery slip for more information)
Scenario:
Today you went to a supermarket during lunch break to buy groceries for the next few days. You
paid for your groceries with your EBT card (or food stamps) and debit card. You have the receipt for the
groceries. While you were at the supermarket, you also bought a prepared lunch from the deli that you
ate there. You paid for your lunch with cash, but you were in a hurry and forgot to take your receipt from
the deli. You remember that you paid about $6.50 for lunch. Now you have the groceries in front of you.
Please enter the information about this stop into the FoodLogger.
Groceries:
o

o

o
o

Pre-packaged food (with barcode)
 Pasta
 Crackers
 Grape tomatoes
 Farro
Produce (with PLU)
 2 bell peppers
 3 oranges
 1 potato
 1 corn (no PLU)
Food from the bakery
 1 cookie
 1 loaf of bread (made in-store, store brand)
Bulk food
 Coffee beans
 Granola

Food with store-specific barcodes (e.g., store brand pre-packaged food)
 Coconut water
 Salt
 Oats meal
 Eggs
o Multiple items packaged together
 Mineral water (4 bottles)
 Juice (4 cups)
Non-food items:
o 1 facial tissues box
o 1 soap
o

Food from the Deli (not present, just described/pictured):
o Caesar salad (small) $2.25
o Bread roll (small) $0.50
o Cup of soup (~8 oz) $2.00
o Bottle of juice (11 fl oz) $1.75

Appendix C: Use Case 2 – Food-Away-from-Home Event
Critical tasks tested:
• Start a day
• Select a food stop from a list of stops identified by the online map
• Add a food event manually
• FAFH Combo meal:
o Select "combo meal" button
o Take a picture
o Enter meal name
o Enter meal price
o Enter number of items (quantity)
o Enter individual meal items
o Enter total event cost or price
o Select payment type
o Take a picture of a receipt and upload it.
• FAFH Individual food item:
o Select "individual item" button
o Enter meal item name
o Enter number of items (quantity)
o Enter item price
o Enter event cost or price
o Select payment type
o Upload receipt
Event Set-up
• The food came from McDonald’s. See delivery receipts for more information.
Scenario:
This evening you ordered food from McDonald’s for your family’s evening meal using the
restaurant’s website. You placed the order at home, and had the food delivered to your home to eat with
your family. You paid with your credit card and have an electronic receipt in your email. Please enter the
information regarding this meal into the FoodLogger app.
•

Food from McDonald’s:
o A Big Mac
o A milk jug
o A chicken sandwich combo meal with fries and drink
 Large fries
 Medium drink
 4 Ketchup packs
o 10-piece Chicken nuggets with 3 barbeque sauce packs
o A Happy Meal (hamburger, apples, fries, milk)
o Baked Apple Pies (3)

Appendix D: Use Case 3: Not Free – Proxy Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter meal price
o Enter individual meal items
 “Other” option (those that do not conform to pre-defined list of
combo items)
o Enter payment type
Event Set-up
• Your child got this food at their school
While your child was at school today, [he/she] was served lunch. School lunches cost $3.50 and
he/she paid for the meal with her pre-loaded lunch card. [He/She] told you that [he/she] had a carton of
milk, mashed potatoes, gravy, one slice of beef the size of his/her hand, and a side of corn. Please enter
this event and food into the FoodLogger app.

Scenario 2: Free – Proxy Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter meal price
o Enter number of items (quantity)
o Enter individual meal items
 “Other” option (those that do not conform to pre-defined list of
combo items)
Event Set-up
• Your child got this food at their school
While your child was at school today, [he/she] had lunch there and didn’t pay for it. [He/She] told
you that, in the lunch box, there were a carton of milk, mashed potatoes, gravy, one slice of beef the size
of his/her hand palm, and a side of corn. Please enter this event and foods into the FoodLogger.

Scenario 3: Not Free – Self Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field OR add stop from GPS
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter meal price
o Enter number of items (quantity)
o Enter individual meal items
 “Other” option (those that do not conform to pre-defined list of
combo items)
o Enter payment type
Event Set-up
• You got this meal at your school
While you were at school today, you were served lunch. School lunches cost $3.50 and you paid
for the meal with you pre-loaded lunch card. The lunch you received had a carton of milk, mashed
potatoes, gravy, one slice of beef the size of his/her hand, and a side of corn. Please enter this event and
food into the FoodLogger app.

Scenario 4: Free – Self Report
Critical tasks tested:
• Start a day
• Add a food stop to FoodLogger in the text field OR add stop from GPS
• Add a food event manually
• Combo meal:
o Select "combo meal" button
o Enter meal name
o Enter payment type
o Enter number of items (quantity)
o Enter individual meal items
 “Other” option
Event Set-up
• You got this meal at your school
While you were at school today, you had lunch there and didn’t pay for it. The lunch box you got
had a carton of milk, mashed potatoes, gravy, one slice of beef the size of your hand palm, and a side of
corn. Please enter this event and food into the FoodLogger.

Appendix E: Instructions for Field Data Entry

In order for us to properly evaluate FoodLogger, we ask you to use FoodLogger every day for 7 days,
starting from today, to report your household’s food acquisition. It is very important that you use
FoodLogger to log all of the foods you either purchased or received for free during this 7-day period. You
should report your food by the end of each day. Don’t skip days or wait until the last day.
The foods that you should report include all the food items you will have either purchased or got for
free, regardless whether the foods are eaten or not during the 7-day period. Do not report any food that
you acquired before today. For example, if this morning you ate a bagel you bought last week, you
should NOT report that food. However, if you went to a friend’s house and had a breakfast there, you
SHOULD report that food.
If you have any problems or run into difficulties while reporting your food in FoodLogger, please log
those problems in the form we provided.

Include the date, time, and short description of the issue. This information will be very helpful for us. If
you need help with using FoodLogger, call this number, , between 7:00 am and
10:00 pm Eastern Time for assistance.

Appendix F: Debriefing Questionnaire on 7-Day Field Data Entry

Please rate your opinion on how easy/difficult it was to enter the following information into the
FoodLogger:
Stops:
Confirm a food stop
1: Extremely easy

2

3

4

5: Extremely difficult

Add a stop that was not automatically captured
1: Extremely easy

2

3

4

5: Extremely difficult

2

3

4

5: Extremely difficult

2

3

4

5: Extremely difficult

2

3

4

5: Extremely difficult

3

4

5: Extremely difficult

Events:
Add a food event
1: Extremely easy

Food items:
Scan a barcode
1: Extremely easy

Enter a PLU
1: Extremely easy

Enter a food item name
1: Extremely easy

2

When you typed text, did you notice the “type-ahead” feature? Did you use it? What is your
opinion?

Sometimes the app asks you to enter information on the size/weight/volume of your food
items. How easy or difficult was it to…

Enter the size/weight/volume for food items
1: Extremely easy

2

3

4

5: Extremely difficult

Enter information for school meals
1: Extremely easy

2

3

4

5: Extremely difficult

Payment: For each food event, the app asks you for the price of the entire purchase, and
sometimes it asks for the price of each food item. How easy or difficult was it to….

Enter the price for individual food items (item level)
1: Extremely easy

2

3

4

5: Extremely difficult

Enter the cost of the entire purchase (event level)
1: Extremely easy

2

3

4

5: Extremely difficult

Choose the method of payment
1: Extremely easy

2

3

4

5: Extremely difficult

[Whenever the user does not select “Extremely easy” ask: Can you please elaborate on why
you selected _(rate)
for _(activity) (For example, why you selected 5 for entering
information for school meals)]

Did you encounter any problems with reporting a combo meal? If so, what problems?

Up to this point, what has been your overall experience with the FoodLogger?

How do you feel about the length of time it takes you to enter food information?

How do you feel about the amount of effort that is required to report the food acquisition so
far?

[If needed] What do you believe could improve this process?

FoodLogger sends notifications to you periodically. Are you bothered by the notifications?
1.
2.
3.
4.
5.

Not at all
Somewhat
Moderately
Very
Extremely

What are the three greatest challenges you have encountered using FoodLogger in the past
seven days?

Compared to the first three days, do you feel more or less comfortable with using FoodLogger
in the last three days?

Do you have any other comments or thoughts about your experiences with the FoodLogger
over the past seven days?

I have a few more questions:
You are paid $5 a day for reporting food acquisition information, for seven days. Do you think
that’s an adequate amount for your effort? [If not: How much money would you think
appropriate?]

How comfortable were you in sharing your GPS location during this study?
1. Completely uncomfortable
2. Somewhat uncomfortable

3. Neither uncomfortable or comfortable
4. Somewhat comfortable
5. Completely comfortable
Here is a list of places people can get food, please say Yes to those places where you acquired
food for yourself or family members in the past 30 days, to the best of your memory.
1. Grocery store – in-person shopping
2. Grocery store – order online for pickup or delivery
3. Big Box Store or Warehouse Club Stores (e.g., Walmart, Target, Costco) – in-person
shopping
4. Big Box Store or Warehouse Club Stores (e.g., Walmart, Target, Costco) – order
online for pickup or delivery
5. Restaurant – eat in
6. Restaurant – order online or by phone for carry out or delivery
7. Friend or family member’s house
8. Food from a church, a food pantry, a food bank, or eat-in soup kitchen
9. Other

Now, I have a few questions for your child. Can you ask him/her come?

Hello, ! How are you doing? I have a few questions to ask you about using
FoodLogger:

1. How do you like using FoodLogger?
2. On a 5-point scale, 1 being most difficult and 5 being easiest, how do you rate your
experience using FoodLogger?
3. What is the most difficult thing you encountered when using FoodLogger?
4. Is there anything else about FoodLogger do you want to tell us?