Attachment F - Individual Diary Analysis Plan

Analysis Categories	Variables of Interest	Counteracting Influences
Matching criteria We will match research and Production the factors of: internet access Smartphone access We will analyze whether Research and Production are different on the following and match accordingly: age race sex education income We will also determine whether double-placed diaries differ from single-placed diaries with regard to expenditures, controlling for 2-week versus 1-week recall. If they do, we will match the Research sample to Production double-placed diaries.
Objective 1: Operational issues and Overall Assessment for each diary mode A simple diary should be able to be completed quickly and easily.	R’S GENERAL COMMENTS Respondents’ positive impressions (open-ended, ID_R1) How to improve the process (open-ended, ID_R3) How secure it felt (ID_R4) R’S BURDEN Respondent estimates of time needed to complete diary [paradata] Respondent assessments of ease/difficulty of logging in (ID_R2) FR ASSESSMENT OF R. FR assessment of R. cooperativeness (FR and ID_FR) FR’S GENERAL COMMENTS – Not in CAPI right now, since not at the case-level. This will asked during a debriefing session What liked about the process What disliked about the process How to improve the process How it changed workload NPC CODERS ASSESSMENT Exact questions TBD Content of Help Desk inquiries DSD to Access database contents for help desk tickets	We will only have this information for the test group, so no comparisons between research-production can be made.
Objective 2: Item level analysis
At a CU level, there should be no differences in data quality between the paper and individual diaries.	Number of entries per section1 Compare CAPI entries for cases with no expenditure entries Average number of entries per week compared with production numbers that are: CUs where no recall carried out (74% of total) – 34.7 entries (Mean) All CUs (including CUs with FR-assisted recall) – 31.6 entries (Mean) Average number of blank diary pages per week compared with production numbers that are: 2.1 blank diary pages per week (Mean)2 Expenditure amounts per section – note we only have power for the “Food” category There should be equal amount or fewer entries that are a) rounded off dollars with no cents (e.g., $4.00) – if possible to analyze b) typical estimating amounts of $1.00, $5.00, $10.00, $20.00 etc.) 7. Item missingness at category level for $ amount, etc. 8. Audit trail files (stored as txt files) that Phase 3 generates during questionable code review – if possible to analyze 9. Rate of duplicate entries for expenditures a. receipt entries by FR (into the CAPI system) that are duplicates to the CU entries b. entries by the CU (since different people can make different entries) 10. Fails phase 2 edits (case blank) 11. Screen for outliers in respondent debriefing questions Review timing data for this section NPC CODERS ASSESSMENT Exact questions TBD – such as number of overrides at NPC	If interviewers have not been adequately trained on the diary or if they do not spend adequate time explaining the sections during placement, then the beneficial aspects of the diary may be obscured.
Objective 3: Response
A mobile and web diaries should seem more convenient than a paper diary and therefore compliance should be improved.	Diary Placement Rates (overall, since this is a double-placement) 1st Diary Completion Rates – using the same completion rules as Production 2nd Diary Completion Rates – using the same completion rules as Production Type B and C Rates taking into account sceenouts Rate of Total Recalls Cross-tab how FR felt about the materials (not good) versus DQ of respondent	Interviewers will be less familiar with the new diaries and this may make them hesitant in their presentation and placement. Interviewers may also recognize that this is only a test, leading them to be less diligent in pursuing non respondents because these Type A’s will not be held against them.
Objective 4: Data Entry Patterns	Number of log-ins per case Examine relationship between log-ins per case and FR phone calls during the collection period using CHI data Start/stop time stamps by diary day (to determine multiple times per day versus only data entry at the end of the week) Analyze log-in at least once on consecutive days versus log-in sporadically Analyze log-in at least once on consecutive days versus TR Analyze data entry at least once on consecutive days versus data enter sporadically Raw paradata from ASD to be used: 1.       Browser/device information 2.       Authentication information (successful, unsuccessful, date, time) 3.       Respondent IDs 4.       Page load date/time 5.       Answer and answer change (date/time)

Debriefing Questions

ID_intro. Now I am going to ask ^NAME’s experience with the diary.

1. Enter 1 to continue

ID_R1 What did you like about this application?

ID_R2 How easy or difficult was it to log in? Would you say –

1. Very easy?

2. Easy?

3. Neither easy nor difficult?

4. Difficult?

5. Very difficult?

6. Not Present

ID_R3 How could we improve the process of recording your expenses?

ID_R4 When using the mobile optimized application/web diary to record your expenses, how secure do you feel that your data is? Would you say –

1. Completely secure?

2. Somewhat secure?

3. Neither secure nor unsecure?

4. Somewhat unsecure?

5. Completely unsecure?

6. Not Present

ID_FR Would you say the respondent was –

1. Very cooperative?

2. Somewhat cooperative?

3. Neither cooperative nor uncooperative?

4. Somewhat uncooperative?

5. Very uncooperative?

1 The Hong Kong study (Grootaert, 86) found higher reporting in individual diary cases for clothing and footwear. Pearl (1979) also expected improved individual-level reporting on the CE for ‘small clothes’, reading material, public transportation and personal services.

2 This counts a page as blank if recall was used, but does not count as a blank if page was et as a vacation day.

55645301 2 Page 2