Attachment K - Cell Phone Research

Cell Phones and Nonsampling Error in the American Time Use Survey

Brian Meekins, Stephanie Denton^1,i

¹Bureau of Labor Statistics

Abstract

Recent research on the impact of cell phones has largely focused on coverage and nonresponse error with few exceptions (Kennedy et al 2009, Brick et al 2011). In this work the authors focus on nonsampling error in the American Time Use Survey (ATUS). This nationally representative survey is conducted by the U.S. Census Bureau on behalf of the Bureau of Labor Statistics. The sample for the ATUS is derived from households that have completed Wave 8 of the Current Population Survey. Households that volunteer a phone number for that survey are then called for the ATUS using that phone number (those who do not volunteer a phone number are mailed an invitation to participate and an incentive). The vast majority of CPS respondents provide Census with a phone number. The ATUS further selects a sample member from within the household to answer relatively detailed questions including a 24 hour time use diary. In this work we examine the impact of calling cell phone numbers on nonresponse and measurement error in the ATUS. Because the sample is derived from CPS completed interviews, we are able to model nonresponse using CPS data. Almost 40% of ATUS telephone sample volunteered their cell phone number for contact in the CPS. Those who volunteer their cell phone number for survey contact in the CPS are just as likely to say that a phone interview is acceptable. Cell phone volunteers are less likely to be complete ATUS interviews due to noncontact while their refusal rate is similar to those volunteering a landline number. Differences in measurement error appear to be negligible. There are some differences in the estimates of time use, but these are largely due to demographic differences.

Key Words: cell phone, mobile phone, nonresponse, measurement error

1. Introduction

With a few recent exceptions (Lynn and Kaminska 2011, Kennedy 2010), nonsampling error as a result of incorporating cell phone numbers into a telephone survey sample has been focused primarily on nonresponse error. Recent and not so recent literature examining the differences between cell phones and landline telephone numbers in RDD surveys appears to indicate that the mechanism for nonresponse differs between these two telephone groups. Where the sampling design stipulates a portion of the sample that overlaps both the landline and cell phone frames, it has been found that typical weighting variables fail to account for differences in the respondents from the two frames (Brick et al 2007). This finding is somewhat attenuated when examining the more detailed phone categories of the overlap (dual user/cell phone mostly, dual user/landline mostly).

Cell phone numbers may result in significant nonresponse bias. In general, cell phone numbers (as dialed through Random Digit Dialing or similar methods) are more difficult to complete (Guterbock et al 2011). The primary reason for this difficulty appears to be lower levels of cooperation (Link et al 2007). In addition, households that use cell phones as their primary source for incoming calls are different from households that use landlines as their primary source for calls. They are younger, less likely to have children, disproportionately Hispanic, and unmarried (Blumberg and Luke 2010). The mode effects of interviewing by cell phone, in terms of measurement error, have not materialized, although research in that area is quite limited. The American Time Use Survey (ATUS) presents a unique opportunity to study nonsampling error, especially measurement error, due to cell phone interviewing because of its unique sampling design and wealth of sample information regarding both respondents and nonrespondents.

2. Data and Methods

The American Time Use Survey (ATUS) is a continuous federally-administered survey on time use in the United States. It is a Bureau of Labor Statistics (BLS) survey conducted by the U.S. Census Bureau. The ATUS provides nationally representative estimates of how, where, and with whom persons age 15 and over use their time. The types of estimates produced using ATUS data are the number and percent of individuals engaging in activities on an average day, the average hours spent doing activities, and the time of day when individuals do the activities. All ATUS data are collected over the telephone using computer-assisted telephone interviewing (CATI).

The ATUS sample is drawn from the Current Population Survey (CPS). The universe for the CPS is composed of the civilian, noninstitutional population residing in occupied households in the United States. Two months after households complete their eighth CPS interview, they become eligible for selection into the ATUS sample. About 2,194 households leaving the CPS sample are selected for the ATUS sample each month (approximately 26,328 households annually), with an annual response rate of 56.9 percent in 2010. Because the ATUS is drawn from the CPS, the survey contains a rich amount of demographic as well as methodological data, such as the time of day of CPS contacts and number of call attempts for each CPS interview.

During the course of the eight waves of the CPS, the respondent may provide a phone number for re-contact. If the household is selected for the ATUS, this phone number is then used to call the household to complete the ATUS (after mailing an advance letter to the household).ⁱⁱ Currently, ATUS staff does not know if a given CPS household phone number that is provided by the CPS respondent is a cell phone or landline number and, therefore, does not know the proportion of cell numbers in their sample (or if this proportion has dramatically changed).

It has been shown that the ATUS tends to complete fewer interviews than the CPS with single, young respondents who rent their dwelling (Meekins, Downey, Fricker 2010). It is likely that cell phone numbers in the sample contribute disproportionately to this nonresponse bias. Indeed, the underrepresented groups are conspicuously similar to those that rely primarily on their cell phones for incoming calls. In addition, those who do not offer a phone number in the CPS may be disproportionately cell phone only households. Research in the past has shown that households regard their cell phone number as more “private” than their landline number. It is quite possible, then, that nonresponse among households that provide a cell phone number as a contact in the CPS may contribute to the nonresponse bias that is obtained from lower response among households that do not provide any contact number in the CPS.

In addition, cell phones are typically linked to an individual, as opposed to a household. For approximately 40 percent of ATUS sample members, the target respondent is not the same as the person who completed the CPS interview. In other words, the ATUS may be calling a cell phone number to obtain information on someone else in the household other than the primary user of the cell phone. This is likely to lead in an increase in the amount of effort needed to interview the household as well as a greater amount of nonresponse in the cell phone sample.

The ATUS offers a unique opportunity to examine measurement error due to two main factors. Firstly, the sample from which the ATUS sample is drawn has proved itself to be a cooperative sample. Indeed, they have competed eight waves of the CPS. It is somewhat less likely, then, that the cell phone sample will be significantly less cooperative than the landline sample. Secondly, because the, either landline or cell phone, contact numbers were volunteered by the CPS respondent for the purposes of re-interview, it can be presumed that the respondent is comfortable being interviewed on that particular device. This again is likely to lead to an increase in cooperation across the entire sample. This increase in cooperation as a result of these two factors assists in isolating the effects due to mode. In the past it has been difficult to differentiate between nonresponse error and measurement error. As previously mentioned, controlling for demographics does not usually eliminate the effects on the estimates due to telephone type. It is generally difficult to ascertain which part of this “residual” error is associated with nonresponse and which is associated with measurement error due to mode.

ATUS sample members from October of 2009 to October of 2010 were matched to existing telephone number databases from Marketing Systems Group (MSG) in order to identify cell phone numbers within the sample of volunteered CPS telephone numbers. There were two versions of the Telcordia database that were utilized, in order to identify numbers based on 1,000 groups (January 2009 and April 2010). In addition, the Neustar database was used in order to identify landline numbers that were “ported” to cell phone numbers. The Neustar database was updated daily and contained the exact date that a number was ported. A total of 35,298 ATUS sample cases were examined. Approximately 36% of telephone sample numbers were identified using this process as cell phone numbers.

An unknown quantity of telephone numbers may be incorrectly classified as landline or cell phone due to an inexact matching of dates with both the Neustar and Telcordia databases. The Telcordia database had only a limited number of archived files available, leaving us with only two time points to match the ATUS sample. However, keep in mind that this database refers to the 1,000 groups assigned by telephone companies to dedicated cell or landline exchanges, and is unlikely to experience significant change in the months between archive files. In addition, there was some delay in the communication of the sample phone number from CPS to ATUS. This number could have been ported or discontinued in that amount of time. However, the authors are confident that the overwhelming majority of ATUS sample telephone numbers are correctly classified.

3. Results

As noted earlier the majority of research examining nonsampling error due to telephone usage type has addressed nonresponse error. Here we begin the discussion with nonresponse error and hopefully extend previous research by incorporating an analysis of measurement error associated with telephone type.

3.1 Nonresponse

The final disposition ATUS sample cases are located in Table 1. The overall completion rate (completes/total sample) was 78.5% for landline and 71.5% for cell phone. Note that there is little difference in the rate of refusal between those that volunteered a cell phone number for contact compared to those that volunteered a landline number. The cell phone cases appear to be somewhat more difficult to contact. Surprisingly, there is not much difference in effort between cases where the CPS reference person is the same as the ATUS reference person and cases where the CPS reference person is different. Table 2 shows the mean number of total attempts and mean number of noncontact attempts by cell and landline for CPS reference person and non-CPS reference person. While cell phones average about two more attempts and two more noncontact attempts, there is little difference between CPS reference and non-CPS reference person cases.

As prior noted, a number of demographic variables are collected in the CPS, including home ownership, marital status, household income, employment status, geographic region, age, and race. These variables can be compared in order to illuminate the demographic profile of those that volunteer a cell phone for re-contact and those that volunteer a landline phone. Table 3 shows this demographic profile by telephone type. As found in prior studies, those that volunteered a cell phone were significantly more likely to be younger, unmarried, renters. There was little difference in race and income between the two telephone groups.

The literature frequently postulates that there is a difference in the mechanism for nonresponse, or the manner in which sample members choose to respond, between those sample members selected from cell phone frames and those selected from landline frames. Therefore, we might expect that there is some difference between the landline and cell groups in our study in the difference in values of the demographic variables between the respondents and nonrespondents. That is, we would expect the difference in nonrespondents and respondents to be different based on whether the sample member volunteered a cell phone or a landline phone for re-contact if the nonresponse mechanism differs by telephone type. Table 4 shows these results. As is typical in surveys with significant nonresponse, nonrespondents are more likely to be younger, Black, unmarried, renters, with lower incomes. However, with exceptions of home ownership, and possibly age, the difference between nonrespondents and respondents between the two telephone groups is not very different and does not lend a great deal of support to the idea that the response mechanism is greatly different between the two telephone groups.

In order to examine this in greater detail, a proportional hazards model was used to model both the likelihood of completion and the time to completion, or in other words, the hazard of completion. In some samples, like RDD samples, the likelihood of disqualification or ineligibility is quite large, but because this sample is composed of CPS respondents, the proportion of the sample found to be ineligible is only 3.6%. In this way, the sample lends itself well to a proportional hazards model which models the survivability of sample (where “death” in this case is completion, and time is the number of attempts [up to 80]). Here we regress the hazard of completion on telephone type in a baseline model and subsequently introduce other covariates. These include demographic variables from the CPS and process variables, such as: weekend calling, time of day of attempts, reluctance shown in completing the CPS, and missing income on CPS, whether a CPS interview was done by telephone, and whether the reference person is the same as the CPS reference person. All first and second order interactions between all covariates and telephone type are also examined.

We find that there is a significant effect (p < .0001) due to being interviewed by cell phone as compared to a landline (hazard ratio = .822), where those volunteering a cell phone had lower odds of completion. This direct effect is diminished significantly (hazard ratio = .339) when introducing covariates and their interactions with each other and telephone type. We would be tempted to conclude, then that most of the difficulty in attaining completions from the cell phone sample is due to their demographic composition, but there are significant interactions and process variables to consider. Both time of day and its interaction with telephone type are fairly large effects as well as previous indications of reluctance on the part of cell phone sample members. The latter, in the opposite direction as one might expect due to the heavy influence of contact in obtaining a completion by cell phone. It may be then that the strategy of calling cell phones, at last by time of day, is not optimal in the ATUS. Further analysis on time of day by telephone type is warranted.

3.2 Measurement Error

A preliminary examination showed that those completing the ATUS interview by cell phone were no more likely to have most of the common problems associated with measurement error. For example, cell phone interviews were no more likely to have “don’t know” or “refusal” responses, they were no more likely to have activities that were not able to be coded, they were just as likely to report common activities like grooming and sleeping, and were even somewhat less likely to have their earnings allocated due to some type of nonresponse.

Cell phone interviews were, however, more likely to round their earnings, generally reported fewer activities, and the length of their interview was on average just over a minute shorter. In addition, some of the time use estimates were different: people who volunteer their cell phones for interviews spent more time working on average, and less time doing personal care and sleeping, household activities, eating and drinking, socializing, volunteering, shopping, and religious activities. Many of these differences remain even after weighting. Table 5 lists the reported activity duration by telephone type.

Of course, the reason for the differences in reported time spent doing these activities is not necessarily the result of bias being introduced by mode. In order to help examine whether the effects of mode are spurious, we used multivariate models where we regressed a number of these time estimates on telephone type, controlling for covariates. Again these covariates included both demographics and process variables.

For reasons of length we will examine only a few of these activities in detail. For example, the model where time spent conducting household activities, as reported by the respondent, was regressed on telephone type by itself yielded a large coefficient of 13.7 - which is statistically significant. Controlling for demographics and process variables, as well as interactions with each other and the telephone type we see that the coefficient for telephone type drops to a very modest 1.05 - which is not statistically significant. However a number of interactions of telephone type are statistically significant and large including the interaction of telephone type and whether the CPS reference person is the same as the ATUS reference person. In addition, the interactions of telephone type and time of day of call and telephone type and length of interview are fairly large and statistically significant. The least squared means shows the total effect of telephone type controlling for all covariates. For household activites, those volunteering a cell number report approximately 93.5 minutes while those volunteering a landline number report 102 minutes of time spent on the activity.

Table 6 shows the results for all activity categories. Few activities retain significant direct effects of telephone type after the introduction of the covariates and interactions. These include: work, consumer purchases (shopping), socializing, and unsurprisingly, telephone calls. Many of the models have significant interactions, including those with telephone type, however, and in some cases these interaction effects actually reverse the influence of the direct effect. Therefore, it is best to examine least square means for the overall influence of telephone type. Across all activity categories, we can see that there are very few meaningful differences in the amount of time reported by telephone type controlling for all demographics and process variables with the largest difference in personal care and sleeping (6.6 minutes, ~1% of total reported) and household activities (6.4 minutes, ~6% of total reported). As in the case of the nonresponse analysis, however, significant interactions of process variables with telephone type were observed in many of the activities. These process variables include: the same reference person from CPS to ATUS, the time of day, and the length of the interview. Again these are effects are worthy of further investigation.

4. Summary

ATUS sample members that volunteered a cell phone on the CPS to be re-interviewed were somewhat more difficult to contact than their landline counterparts. Surprisingly, although having a different reference person from the CPS to the ATUS did increase the number of attempts; the increase was the same for cell and landline numbers. Controlling for demographic variables and process (or survey operation) variables greatly diminished the direct effect of telephone type. However, the significance of the interaction effects, most notably that of telephone type and time of day, is cause for concern.

Those that volunteered a cell phone number did have shorter interview lengths, were more likely to round their earnings reports, and, most importantly, reported somewhat fewer activities than landline respondents. However, on a number of other data quality measures the two telephone groups showed no difference. There were a number of significant differences on the amount of time respondents from the different telephone groups reported spending on activities although most of these differences can be accounted for by demographic differences in the two samples or the interaction of the telephone group with process variables. The latter, however, are of particular concern, in that these may be true mode effects. The most important interactions are with the CPS reference person, length of interview, and time of day of call.

Although the authors were hopeful that due to the presumed high level of cooperation of ATUS sample members we would be able to differentiate nonresponse and measurement error, we find that the interaction of time of day and telephone type is a significant predictor of both, obscuring the different effects. Indeed, while cooperation may be high among ATUS sample members, in general, noncontact when calling cell phones is still an issue. In the future, it may be helpful to identify the cell phone numbers in the sample and make special efforts (as in many RDD surveys) to accommodate these sample members. One possibility may be to expend more effort in attempting contact, varying the time of day of the call. In addition, it may be useful to ascertain the social context in which the respondent is being interviewed and retry the interview at a later date if it does not appear to be conducive to a longer, more productive interview.

Table 1: Disposition	CPS ref person		Not CPS ref person
	Cell	Landline	Cell	Landline
Completion	71.5	77.8	71.4	79.1
Refusal	10.9	10.9	10.0	9.1
Noncontact	5.9	5.3	7.6	6.6
Not eligible	3.6	1.3	4.1	1.9
Unknown eligibility	8.0	4.7	7.0	3.2

Table 2. Telephone type (CPS ref person)	Mean Attempts	Mean NC Attempts
Landline	9.5	7.1
Cell	11.5	8.9
Telephone type (Not CPS ref person)	Mean Attempts	Mean NC Attempts
Landline	9.9	7.5
Cell	12.0	9.4

Table 3. Variable Name	Category	Cell	Landline
Housing Tenure	Owns	57.2	80.8
	Rents	42.8	19.2
Marital Status	Married	49.1	60.4
	Sep, Div, Wid	18.0	17.5
	Never Married	32.9	22.1
HH income	Lowest 25th	20.2	13.7
	Middle 50th	53.8	51.6
	Highest 25th	26.0	34.7
Age	Under 18	7.6	8.2
	19 to 30	28.7	11.3
	31 to 45	39.4	34.3
	46 to 65	21.2	31.5
	66 +	3.1	14.7
Race	White	77.9	80.8
	Black/AA	14.8	13.0
	Other	7.3	6.2

Table 4.		Cell		Landline
Variable	Category	Response	NR	Response	NR
Housing Tenure	Owns	62.7	42.9	83.0	72.7
Marital Status	Married	53.9	36.5	63.5	48.5
HH income	Lowest 25th	17.7	25.1	11.5	21.1
	Middle 50th	52.0	57.1	51.5	52.2
	Highest 25th	30.3	17.8	37.1	26.8
Age	Under 18	7.8	7.1	8.4	7.2
	19 to 30	26.5	34.5	10.7	13.9
	31 to 45	41.3	34.6	35.7	28.8
	46 to 65	21.5	20.4	31.5	31.5
	66 +	2.9	3.4	13.7	18.6
Race	White	80.1	72.2	82.6	74.0
	Black/AA	12.4	21.2	11.3	19.8
	Other	7.6	6.6	6.2	6.2

Table 5. Time Use Major Category	Cell	Landline
Personal care/sleep	564.9	563.2
Household activities*	99.6	113.3
Caring for other HH members*	41.6	33.1
Work*	223.1	188.0
Education*	30.1	36.0
Consumer purchases*	19.9	22.8
Household services	0.6	0.8
Eating and drinking	65.5	67.4
Socializing*	242.8	265.1
Sports and exercise	21.5	22.8
Religious	8.9	8.6
Volunteer*	7.1	10.3
Telephone calls	6.1	5.7
Traveling*	77.2	72.8
* Statistically significant at .0001

Table 6. Time Use Major Category	Coefficient (sig)	LS Means
		Cell	Landline
Personal care/sleep	4.61 (.440)	583.7	577.1
Household activities	9.17 (.108)	101.6	108.0
Caring for other HH members	4.57 (.136)	36.9	36.3
Work	-24.47 (.012)	148.8	146.2
Education	1.11 (.804)	56.4	60.0
Consumer purchases	8.87 (<.001)	15.6	19.2
Household services	.766 (.078)	0.39	0.30
Eating and drinking	4.30 (.066)	66.5	65.7
Socializing	-37.14 (<.001)	296.9	303.0
Sports and exercise	6.24 (.020)	9.9	11.1
Religious	-3.95 (.020)	7.8	5.9
Volunteer	2.70 (.208)	12.3	13.3
Telephone calls	4.74 (<.001)	6.9	6.1
Traveling	2.53 (.461)	61.5	60.5

References

AAPOR Cell Phone Task Force. 2008. “Guidelines and Considerations for Survey Researchers When Planning and Conducting RDD and Other Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers.”

Blumberg, Stephen J. and Julian Luke. 2010. “Wireless Substitution: Early Release of Estimates from the National Health Interview Survey, January - June 2010.” National Center for Health Statistics. Available from: http://www.cdc.gov/nchs/nhis.htm.

Brick J. Michael, Brick Pat D., Dipko Sarah, Presser Stanley, Tucker Clyde, Yuan Yangyan. 2007. “Cell Phone Survey Feasibility in the U.S.: Sampling and Calling Cell Numbers Versus Landline Numbers.” Public Opinion Quarterly. 71:23-39.

Guterbock, Thomas, Paul Lavrakas, Trevor Thompson and Randal ZuWallack. 2011. “Cost and Productivity Ratios in Dual-Frame RDD Telephone Surveys” Survey Practice, April, 2011.

Keeter, Scott. 2006. “The Cell Phone Challenge to Survey Research.” Pew Research Center. Washington, D.C.

Kenndey, Courtney. 2010. “Nonresponse and Measurement Error in Mobile Phone Surveys.” Dissertation. Joint Program in Survey Methodology. University of Michigan.

Link, Michael, Michael Battaglia, Martin Frankel, Larry Osborn, and Ali Mokdad. 2007. “Practicability of Including Cell Phone Numbers in Random Digit Dialed Surveys.” Public Opinion Quarterly. 71:814-839.

Meekins, Brian, Kathy Downey, and Scott Fricker. 2010. “Errors in Reporting of Activities in the American Time Use Survey.” Paper presented to the International Total Survey Error Workshop. Stowe, VT.

Piekarski, Linda. 2008. “Cell Phone Sampling.” Presentation given at the IFD & TC. New Orleans, LA.

Stroup, Craig and John Vu. 2011. “Numbering Resource Utilization in the United States.” Wireline Competition Bureau. Federal Communications Commission.

i Special thanks to Linda Piekarski, Vice President of Database and Research at Survey Sampling International (SSI), for her extremely valuable advice on this paper.

ii

If a phone number is not provided, the household is mailed an incentive along with the advance notification of the survey and requested to call into the Census Bureau to conduct an ATUS interview. For more information on ATUS advance materials and incentives, see “American Time Use Survey User’s Guide: Understanding ATUS 2003 to 2011.” http://www.bls.gov/tus/atususersguide.pdf