Don Waldman's Letter to OMB

“OMB 3060-XXXX Consumer Survey; B. Collections of Information

Employing Statistical Methods”

12/3/2010

Response to OMB review 11/30/2010

Thank you for your constructive and helpful comments. Below the contractor explains how they have addressed your comments in the revised supporting statement and draft survey.

1. Why eight experiments?

See footnote 5, para. 2, page 4 of supporting statement:

By asking respondents to complete multiple choice experiments, the contractor is able to increase parameter estimation precision and reduce sampling costs by obtaining more information on preferences for each respondent. Carson et al (1994) review a range of choice experiments and find respondents are typically asked to evaluate eight choice questions. Brazell and Louviere (1997) show equivalent survey response rates and parameter estimates when they compare respondents answering 12, 24, 48 and 96 choice questions in a particular choice task. Savage and Waldman (2008) found there is some fatigue in answering eight choice scenarios when comparing online to mail respondents. To minimize survey fatigue in this study, the contractor is considering dividing the eight choice scenarios into two sub groups of four scenarios. Here, the respondent will be given a break from the overall choice task with an open-ended payment card question between the first and second set of four scenarios.

2. Unhappy with the vagueness of the features of the media environments.

These are the features the FCC is interested in. See last two sentences of para. 3, page 6 of supporting statement:

This information is extremely useful for policy analysis. It will not only inform the FCC on the effectiveness of their competition, localism and diversity goals, but will also provide a structural basis for economy wide analysis of the welfare affects from alternative media policies.

3. Focus group protocol.

The contractor will conduct two focus groups and one pre test. Focus group I will have approximately ten subjects and will be administered by the contractor in the seminar room of the Economics building at the University of Colorado at Boulder. Focus group II will have five subjects and will be administered by RRC Associates at their premises in Boulder, Colorado. The pre test will be administered online by Knowledge Networks to approximately 75 of their online panel members.

A detailed description of the two focus groups and the pre test is provided under Point 4, page 9 – 12 of the supporting statement.

4. Objection to “negative news from African American or Hispanic neighborhoods” example for the multiculturalism feature.

The description of multiculturalism in the survey has been revised. See the first paragraph on page 10 of the survey:

Finally, let’s consider the information you get from your media environment that reflects the interests of women and minority citizens (e.g., disabled, persons of non-white ethnicity). We will call this feature multiculturalism. Examples of multiculturalism are reports on: Black History month, the Cinco de Mayo celebration, female wage inequality, or programs that help people with disabilities find a job.

5. Objection to the two questions that immediately follow Q39, Q42, Q45 and Q48, respectively.

The contractor has given this much thought and would also like to simplify this sequence of questions. The contractor is considering two options and would like feedback from the two focus groups before deciding on the most appropriate option.

Option 1 (example is DIVERISTY). Use our algorithm to assign a “Low”, “Medium” or “High” level of DIVERSITY to each respondent’s overall media environment. The algorithm will calculate a score for the overall media environment by first multiplying the level, low (1), medium (2) and high (3), for each individual media source by the share of hours devoted to that media source, and then summing across all media sources. The overall score will be compared to values in a look up table in order to assign “Low”, “Medium” or “High.” Remove the two follow up questions, here Q43 and Q44.

Option 2 (example is DIVERISTY). Reverse the order of the questions. That is, first ask respondents to indicate their assessment of the level of diversity of opinion from their overall media environment (Q44), and then ask them to indicated the level of diversity of opinion for each individual media source (Q42). Remove Q43

6. Objection to use of Low, Medium, and High, in the actual levels of the features that we will present to respondents.

The contractor wants to keep the descriptions of the levels consistent across the features and will continue to use Low, Medium, High. However, the contractor will also provide a hyperlink at the top of each choice box that permits respondents to review a more detailed summary of the levels of all features. For example, in Q52, a respondent that places their cursor over ADVERTISING would be presented with the screen:

“With low advertising, the amount of space on a newspaper or web page, or the amount of air time devoted to commercial advertising on radio or TV, is barely noticeable. With medium advertising, the space or time devoted to advertising is more noticeable. With high advertising, the space or time devoted to advertising is very noticeable, to the point of being annoying when you are viewing or listening to your media source.”

The contractor likes the advantage of an Internet survey here: only those unsure of the advertising feature will click on the hyperlink and take the time to read the enhanced description, thus reducing potential survey fatigue.

7. Stratification (small DMA’s).

See para. 2, page 2 of the supporting statement:

The sample will not be stratified. The FCC contractor is not calculating “market level” estimates of willingness to pay (WTP) for each of the 210 DMA markets. Instead, the contractor will be estimating the impact of different market structures (X_j in equation 6 on page f of this supporting statement) on an individual consumer’s WTP for the media environment (y_ij in equation 6 on page 5 of this supporting statement) they report that they have. As such, all the contractor needs is to have an adequate number of responses for different market structures. For example, given that seven to eight percent of TV households have five or fewer TV stations, the contractor expects the conservative sample size of 4000 households to contain about 280 to 320 households in “small TV markets.” This should be sufficient information to examine differences in valuations between small and “medium TV markets” or between small and “large TV markets.” However, to be sure, the contractor will have KN oversample small TV markets so they comprise ten percent of the sample, or about 400 observations.

8. Plan to address non response bias.

See page 7-8 of the supporting statement, and the “Best Practices for ICR Supporting Documentation for KnowledgePanel^® Surveys appendix.”

9. Use of response latency (the time it takes a respondent to answer a choice question) to measure bias.

This idea has been omitted from the research methodology (and supporting statement).

10. Objection to “more liberal” and “more conservative” in the diversity feature.

The description of diversity has been revised accordingly. See paragraph one and two at the top of page 8 of the survey:

Now consider the diversity of opinion in your media environment.

42. A low diversity media environment provides information on news and current affairs from only one viewpoint. For example, if you have a (print or online) subscription only to the Wall Street Journal, the level of diversity from newspapers would be low. In a medium diversity environment the information would come from a few different viewpoints. For example, you could watch CNN or Fox news for different opinions about a national issue. If so, the level of diversity from TV would be medium. Alternatively, you could listen to a few radio talk shows with different opinions about an issue in your community. If so, the level of diversity from Radio would also be medium. A high diversity environment provides information from many different viewpoints.

11. Description of random utility model.

An intuitive description of the econometric approach is provided on page 2 -6 of the supporting statement. A detailed description of how the random utility model (in step 1) is estimated is provided in the “model and estimation appendix.”

References

J. Brazell, & J. Louviere, Respondent's Help, Learning and Fatigue, presented at the 1997 INFORMS Marketing Science Conference, University of California, Berkeley.

R. Carson, R. Mitchell, W. Hanemann, R. Kopp, S. Presser & P. Ruud, Contingent Valuation and Lost Passive Use: Damages from the Exxon Valdez, Resources for the future discussion paper, Washington, D.C (1994).

S. Savage & D. Waldman, Learning and Fatigue During Choice Experiments: A Comparison of Online and Mail Survey Modes, 23 J. of Applied Econometrics 351-371 (2008), available at http://spot.colorado.edu/~waldman/index_files/Learning%20and%20fatigue.pdf.

5