Fuel Economy Advertising Study SS fin

Federal Trade Commission
Supporting Statement for a Paperwork Reduction Act
Submission to OMB
Study of Consumer Perception Concerning Fuel Economy Claims
OMB Control No. 3084-NEW
A. JUSTIFICATION
(1)

Necessity for Collecting the Information

The study will provide Federal Trade Commission (FTC) staff with information about
how study participants interpret fuel economy advertising claims. This information, along with
information received from public comments, will help the FTC in considering changes to the
Guide Concerning Fuel Economy Advertising for New Automobiles (“Fuel Economy Guide” or
“Guide”) (16 C.F.R. Part 259). The Commission first issued the Guide in 1975 to prevent
deceptive fuel economy advertising and to facilitate the use of fuel economy information in
advertising. The Guide helps advertisers avoid unfair or deceptive claims under Section 5 of the
FTC Act. 1 To accomplish this goal, the Guide advises marketers to disclose established
Environmental Protection Agency (EPA) fuel economy estimates (e.g., miles per gallon or
“mpg”) whenever they make fuel economy claims based on those estimates. In addition, if
advertisers make fuel economy claims based on non-EPA tests, the Guide directs them to
disclose also EPA-derived fuel economy estimates and provide details about the non-EPA tests
such as the source of the test, driving conditions, and vehicle configurations. On May 15, 2014
(79 Fed. Reg. 27,820), the Commission published a Notice resuming its regulatory review of the
Guide, which had been previously postponed 2 pending new fuel economy labeling requirements
from the EPA and completion of the FTC’s review of its Alternative Fuel Rule (16 C.F.R. Part
309).

1

15 U.S.C. 45(a). The Commission’s industry guides, such as the Fuel Economy Guide, are
administrative interpretations of the application of Section 5 of the FTC Act, 15 U.S.C. 45(a), to
advertising claims. The Commission issues industry guides to provide guidance for the public to
conform with legal requirements. These guides provide the basis for voluntary abandonment of
unlawful practices by industry members. 16 C.F.R. Part 17. The guides do not have the force
and effect of law and are not independently enforceable. However, failure to follow industry
guides may result in corrective action under Section 5 of the FTC Act. The Commission,
therefore, can take action under the FTC Act if a business makes fuel economy marketing claims
inconsistent with the Guide. In any such enforcement action, the Commission must prove that
the act or practice at issue is unfair or deceptive.
2
76 Fed. Reg. 31,467 (June 1, 2011).

(2)

Use of the Information

The FTC plans to conduct Internet-based research to explore consumer perceptions of
certain fuel economy marketing claims to help the Commission better advise marketers on how
to comply with the law. The proposed study will compare participant responses regarding the
meaning of such claims. Specifically, using a treatment-control comparison methodology, the
study will compare participant responses regarding their understanding of a variety of claim
types, such as general fuel economy claims (e.g., “this car gets great gas mileage”), specific
MPG claims (e.g., “39 mpg”), and driving range claims. To aid in developing possible changes
to the Fuel Economy Guide, FTC staff will consider the consumer research results in conjunction
with the broad range of issues raised by commenters during the Guide review.
Having considered the costs and benefits of various data collection methods, the FTC has
concluded that an Internet panel with nationwide coverage will provide the most efficient way to
meet the research objectives. The study will draw participants from an Internet panel maintained
by a commercial firm. The FTC staff plans to collect responses from U.S. automobile consumers
representing a broad spectrum of the U.S. adult population. All participation will be voluntary.
While the results will not be generalizable to the U.S. population, they will provide useful
insights into consumer understanding of the claims being considered, particularly when
comparing the responses to various scenarios to determine how consumers may view those
scenarios differently. The FTC has contracted with Great Lakes Marketing Research, a
consumer research firm with substantial experience assessing consumer communications via the
Internet and alternative protocols, to administer the Internet study.
(3)

Consideration of the Use of Information Technology to Reduce Burden

Consistent with the aims of the Government Paperwork Elimination Act, 44 U.S.C.
§ 3504 note, the proposed study will use the Internet for data collection. This should minimize
burden on respondents while facilitating data collection in a cost-efficient manner. For example,
people who choose to participate in the study will be able to view the questionnaire, as well as
submit their responses via computer, at a time and location of their choosing.
(4)

Effort to Identify Duplication

The FTC has issued several requests for comment on the above-noted issues in the past,
including a request for relevant study findings, published in the Federal Register (as described in
Part A.8(a) below). FTC staff has reviewed the relevant academic literature as well as industry
sources for studies of consumer perception of the claims at issue. We know of no other studies
that provide the specific information being sought here.

2

(5)

Efforts to Minimize Burden on Small Organizations
Not applicable. Only individual consumers are being surveyed.

(6)

Consequences of Not Conducting the Collection of Information

If this information is not collected, the Commission may lack sufficient information to
address important issues and to target more effectively future law enforcement and consumer
education actions. The survey scope and burden has been reduced as much as possible short of
sacrificing the value of the information to be collected.
(7)

Circumstances Requiring Collection Inconsistent With Guidelines

The collection of information in the proposed survey is consistent with all applicable
guidelines contained in 5 C.F.R. § 1320.5(d)(2). The contractor will ensure a stratified sample of
the population across consumer groups broadly representing the characteristics of U.S. noninstitutionalized respondents 18 years and older, relative to the most recent Census Bureau
Current Population Survey. Part B of this Supporting Statement has further detail about survey
methodology.
(8)

Consultation with Outside the Agency
a.

Public Comments

As required by section 3506(c)(2) of the Paperwork Reduction Act (“PRA”), 44 U.S.C.
§§ 3501-3521, the FTC published a notice seeking public comment on the proposed collections
of information. See 79 Fed. Reg. 26,428 (May 8, 2014). In response to the PRA notice, the
Commission did not receive any comments. However, in response to its May 15, 2014 Notice
(79 Fed. Reg. 27,820) announcing resumption of the regulatory review of the Guide, two
commenters discussed the proposed research. Pursuant to the OMB regulations, 5 C.F.R. Part
1320, that implement the PRA, the Commission is providing this second opportunity for public
comment.
As noted above, in response to a separate Notice, the Commission received two
comments related the proposed study.
Consumer Groups: A group of consumer, energy efficiency, and environmental
organizations 3 urged the FTC to test various format and font requirements to determine the most
effective way to communicate fuel economy information in radio, print, television, and the web.
Though such information may be helpful for advertisers, the FTC staff does not plan to devote
3

Comment #573-#00006. See http://www.ftc.gov/policy/public-comments/initiative-573.

3

the limited resources of this study to explore font and format issues. Instead, the FTC staff plans
to focus on fundamental issues related to respondents’ understanding of various claim types. In
addition, the FTC has already provided substantial guidance on conspicuous disclosures in other
contexts. 4 Finally, the study’s online format may not provide the best means to draw
conclusions about format and font requirements given different screen sizes and font settings that
respondents may use.
National Automobile Dealers Association (NADA): The NADA 5 suggested that the FTC
consider existing research, including that conducted by the EPA, National Highway Traffic
Safety Administration (NHTSA), and the U.S. Department of Energy (DOE), before conducting
its own study. Staff has consulted with these agencies and searched for relevant studies and has
not identified any recent studies that address the types of fuel economy advertising issues under
consideration in the present project.
b.

Consultation Outside the Agency

In addition to soliciting comments widely on the Fuel Economy Guide review, FTC staff
has actively consulted with staff at other federal agencies that have expertise in fuel economy
issues. Specifically, FTC staff has discussed the study with the staff at the EPA, which
administers the fuel economy labeling program upon which the Guide is largely based. The staff
has also communicated with staff at DOE and NHTSA. FTC staff has also consulted with the
FTC contractor for this project, Great Lakes Marketing Research, and the FTC’s survey
consultant, Manoj Hastak, Ph.D., Associate Professor of Marketing at American University’s
Kogod College of Business Administration in Washington, DC.
(9)

Payments and Gifts to Respondents

Great Lakes Marketing Research (or its contractor) will provide incentives to members of
the Internet panel who will participate in the online study. Specifically, these incentives are
similar to frequent flyer miles that can be redeemed for rewards. The total dollar value of the
points that will be awarded to the combined 8,000 respondents to the pretest and Internet
questionnaire will be approximately $15,900 (or an average of $2 per person).

4

See, e.g., “.com Disclosures: How to Make Effective Disclosures in Digital Advertising,” FTC,
March 2013, http://www.ftc.gov/sites/default/files/attachments/press-releases/ftc-staff-revisesonline-advertising-disclosure-guidelines/130312dotcomdisclosures.pdf.
5
Comment #573-#00008. See http://www.ftc.gov/policy/public-comments/initiative-573.

4

(10) & (11) Assurances of Confidentiality/Matters of a Sensitive Nature
The information collected will be kept anonymous. Information of a sensitive nature will
not be collected. Responses to the study questionnaire will not include any identifying
information and the FTC will not receive any information about the identity of individual
respondents. Indeed, no identifying information, including email addresses, is collected from
consumers or other individuals for the purpose of this study. The email addresses for consumers
will be generated from the panel database. When a respondent completes a study questionnaire,
the respondent’s email address is stored in the contractor’s own Internet panel database, and a
unique ID number is then assigned to that email address. Respondents’ responses will be stored
in a separate database that includes the unique ID number, but not the underlying email address.
Thus, the study database with participants’ questionnaire responses will be anonymous. The
contractor will not under any circumstances use any email address to retrieve response records.
The contractor also will be required to have sufficient procedures in place to prevent
unauthorized access to respondent information, such as: storing personally identifying
information on separate servers from questionnaire response data; using firewalls to secure those
servers; and maintaining audit records of log-ins, file accesses, and other security incidents.
Moreover, the proposed data collection will not include sensitive questions.
(12)

Estimated Annual Hours Burden

The staff is revising certain prior assumptions in its previously published burden
analysis, 6 based on a more precise target population for completing the questionnaire and further
consultation with its contractor regarding the anticipated response rate. Great Lakes Marketing
Research anticipates that approximately 50% of those invited to participate in the survey study
will complete the pretest and questionnaire. Accordingly, Great Lakes Marketing Research
might contact as many as 8,000 persons to achieve the contracted aim of fully surveying 3,600
respondents and pretesting an additional 100 respondents beforehand. Staff projects that those
who will prematurely end the process will do so in under one minute. Thus, this activity will
total 72 hours (4,300 respondents x 1 minute).
As before, staff estimates that respondents to the Internet questionnaire will require, on
average, approximately 20 minutes to complete it. Staff will pretest the questionnaire with
approximately 100 respondents to ensure that all questions are easily understood. Allowing for
an extra three minutes for questions unique to the pretest, the pretest will total approximately 38
hours cumulatively (100 respondents x 23 minutes each). Once the pretest is completed, the
FTC plans to seek information from up to 3,600 respondents for approximately 20 minutes each
for a total of 1,200 hours. Thus, cumulatively, for all respondents, responding to the FTC’s
6

See 79 Fed. Reg. at 26,429.

5

pretest and questionnaire will consume approximately 1,310 hours. The cost per respondent
should be negligible. Participation will not require start up, capital, or labor expenditures by
respondents.
(13)

Estimated Annual Cost Burden

The cost per respondent should be negligible. Participation is voluntary and will not
require any labor expenditures by respondents. There are no capital, start-up, operation,
maintenance, or other similar costs to the respondents.
(14)

Estimated Cost to the Federal Government

The total cost to the Federal government for the information collection will be
approximately $122,139. Staff projects it will cost approximately $51,000 to pay Great Lakes
Marketing Research to review the study questionnaire, program it into an online format, execute
the pretest and online study, and analyze and report the data. The estimated cost of FTC staff
time is approximately $71,139. 7 This is necessarily an estimate because several factors in this
calculation may vary, including the amount of staff involved and the actual time required.
(15)

Program Changes or Adjustments
Not applicable. This is a proposed new study.

(16)

Plans for Tabulation and Publication

The results of the study will be used to inform the FTC about the perceptions of
consumers who encounter fuel economy advertising claims in commerce. As part of the
regulatory review, the FTC staff plans to release a summary of the findings to the public to allow
interested parties to digest and comment, as appropriate, upon the Commission’s use of the data.
The collection of information will begin after the completion of the OMB review process,
subject to OMB approval. The projected duration of the information collection is approximately
eight weeks.

7

This estimate is based on 900 hours of staff time (Attorneys: 10 hours per week x 21 weeks x 3
attorneys = 630 hrs. x $ 90.3 per hr. = $56,889; Economists: 10 hours per week x 15 weeks x 1
economist = 150 hrs. x $95 per hr. = $14,250).

6

(17)

Display of Expiration Date for OMB Approval
Not applicable.

(18)

Exceptions to Certification
Not applicable.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
(1)

Description of Study Design, Respondent Universe, and Sampling Methodology

Because fuel economy claims are so pervasive, a wide segment of the U.S. population
likely is exposed to these ads. Therefore, the research targets a broad sample of consumers. The
study will use a nationwide Internet panel to identify potential respondents and the questionnaire
will be administered online. The FTC will work with Great Lakes Marketing, which has
substantial experience assessing consumer communications using Internet protocols and
alternative protocols. This expertise should be helpful in designing the research and
administering the study. By using random assignment across treatment groups, the research will
provide useful information about whether consumer perception of the fuel economy claims
studied differs across the various disclosure conditions the study will examine.
a.

Study Design

The primary focus of the research is to gather information about respondent perceptions
of fuel economy claims and how additional disclosures affect those perceptions. The study has
six sections. The first two sections focus on how respondents interpret the strength of fuel
economy claims and potential implicit comparisons generated by those claims. The third section
focuses on how respondents interpret fuel economy claims involving the term “up to.” The
fourth section focuses on how respondents interpret certain claims about electric vehicles. The
fifth and sixth sections focus on how respondents interpret miles-per-gallon claims (e.g., whether
the claim specifies that the miles per gallon is for highway driving, combined city and highway,
or does not specify). The sections form three parts, consisting of the first two sections, the
middle two sections, and the last two sections. Within each part, the order of the two sections
will be randomized for each respondent, so that half of the respondents (1,800) will view one of
the sections first and the other half will view the other section first.
For each section, all participants will answer one or more open-ended questions. In
addition, in each section, the study will randomly assign each respondent to one of several closeended questions corresponding to different fuel economy claims and disclosures, either treatment
or control, independently across sections for each respondent. Each respondent will view only
one close-ended question for each section. For any given closed-ended question, 600
7

respondents will view that closed-ended question before viewing the other section in that part,
while another 600 respondents will view that closed-ended question after viewing a different
closed-ended question within that part but from the other section. Staff will compare responses
across groups that viewed different closed-ended questions to determine how differences in
disclosures included in fuel economy claims affect respondents’ interpretation of the claims.
This structure will mitigate any potential effects from ordering in each part.
To explore different types of fuel economy claims, each section of the study will present
respondents with a particular type of fuel economy claim. The study will then ask respondents to
interpret the claim using one or two open-ended questions. After the respondent has answered
the open-ended question(s) for the section, the study will present the respondent with one of
three closed-ended questions, focusing on either a fuel economy claim without additional
disclosure – the control statement – or the same fuel economy claim with an additional
disclosure that qualifies the fuel economy claim. The closed-ended questions will ask the
respondent to select an answer that best describes their interpretation of the fuel economy claim,
or to indicate that they do not understand the claim. All the closed-ended questions in the same
section will have the same possible answers. The control question will present a fuel economy
statement that may typically appear in an ad for an automobile. Responses to this question will
establish a baseline for comparison with responses to the treatment questions, which include the
same fuel economy claim with the addition of a more specific disclosure related to the fuel
economy claim. The comparisons will reveal differences, if any, between how respondents
perceive the original claim and the claim with additional disclosure.
b.

Sampling Frame

After considering the costs and benefits of various data collection methods, the FTC staff
has concluded that the most efficient way to collect the data needed to meet the research
objectives within a feasible budget is to employ an Internet panel with nationwide coverage.
Therefore, the FTC has contracted with Great Lakes Marketing to design an Internet study that,
while not technically representative of the nation as a whole, nonetheless reflects the views of a
broad population.
The sampling frame for this study encompasses members of the contractor’s Internet
panel. Great Lakes Marketing is contracting with a provider of an Internet panel consisting of a
million individuals drawn from throughout the country, derived from a series of convenience
sampling procedures, rather than true probability sampling. As also discussed in Part A of this
Supporting Statement, FTC staff, in consultation with its contractor, has determined that a
screening sample of up to 8,000 respondents will be needed to ensure 3,700 completed responses
– 100 for the pretest and 3,600 for the main study.
The FTC staff understands that an Internet panel is not a nationally representative
probability sample, and therefore recognizes that the study results will not be projectable to the
general population. Accordingly, the study will focus on comparing respondent interpretations
8

across various fuel economy claims and will not seek to project the percentage of the population
at large that holds a particular view. The FTC staff will consider this point further in the ensuing
analysis of the data and note this limitation in any discussion of the findings. Nevertheless,
given that the study is part of a broader analysis, which will include this information as well as
other information from relevant sources such as public comments, the Internet panel should be
suitable for the study’s purpose.
c.

Sampling Methodology

The FTC has determined that a sample size of 3,600 consumers is appropriate for the
study questionnaire based on several considerations, including the funds available for the study,
the cost of different sample sizes, the number of claims, product and scenario cells into which the
sample will be divided, and a power analysis.
The contractor will randomly allocate each of the 3,600 respondents to one of 18 cells in
each part – comprised of two sections – of the study. Thus, each cell in the each section will
contain approximately 200 respondents. However, for each closed-ended question, three cells of
participants (600) will view the closed-ended question first. Primarily, the study will examine
whether the distribution of responses in interpreting each type of claim differs significantly
between control and treatment versions of the claim.
For all six sections, the closed-ended question will have a sample size of approximately
600 containing only those respondents who see the closed-ended question first. Each closeended question will have either four, five, or six possible answers. Given the structure of
treatment questions, one focal answer will likely be chosen by a significant plurality of
respondents. If the true control distribution is roughly uniform (e.g., 20% for each of five answer
options) and the treatment leads to the response frequency of one answer doubling while the
other answers remain uniform (e.g., 40% for one answer and 15% for the other four), then with a
sample size of 600 per cell there is better than a 99% probability of finding a statistically
significant difference using a chi-squared test with a 5% significance level. If the control
distribution is not uniform, that will tend to increase the likelihood of detecting a statistically
significant difference unless the control distribution is skewed towards the same answer as the
treatment question. However, because there are two treatment questions for each section, with
different focal answers, it is unlikely that responses to the control question will be skewed in a
way that makes detection of a difference in both control questions more unlikely.

9

(2)

Description of the Information Collection Procedures

The FTC has selected Great Lakes Marketing Research to recruit 8,000 individuals 18
years of age or older for the study. The FTC’s questionnaire will include questions designed to
enhance the FTC’s understanding of how consumers interpret certain claims. The questionnaire
will use both open-ended and closed-ended questions to accomplish this goal.
The procedure for administering the questionnaire will be determined in consultation
with the contractor. Staff expects that the contractor will develop an online program that will
allow respondents to view appropriate questions and materials based on their answers to prior
questions. In this way, Internet questionnaires allow more control over the sequence in which
respondents view questions and thus can improve upon mail questionnaires where respondents
might preview later questions and materials intended to be viewed only after earlier questions.
The contractor will provide the FTC with raw data, as well as tabulated data, simple
graphical representations of the data, and simple significance test results. For example, the
contractor will provide a graphical depiction of the percentage of respondents interpreting a
given claim in a certain way within various scenarios and present results of appropriate tests to
determine if the differences are statistically significant.
(3)

Methods to Maximize Response Rates/Reliability of Sample Data

The existing Internet panel used for sampling consists of people who have expressed
interest in sharing their opinions via the Internet and who do so regularly. To help maximize the
participation rate, FTC staff and/or the FTC’s contractor will:
·

Design an experimental protocol that minimizes burden (short in length and clearly
written);

·

Test the draft protocol with a pretest of 100 respondents to ensure that the protocol
minimizes burden, and then refine the protocol as appropriate;

·

Administer the experiment to individuals who have expressed interest in participating in
Internet studies; email reminders to respondents who do not complete the protocol soon
after they receive the original invitation to participate; provide participation incentives
similar to frequent flyer miles that can be redeemed for rewards; and

·

Provide contact information where respondents may obtain help in the event of technical
difficulties when attempting to complete the questionnaire.

The FTC staff expects that potential non-response issues for this particular study can be
addressed. For example, it may be possible to compare how responders compare to nonresponders in a cost-efficient manner.
10

(4)

Testing of Procedures or Methods Undertaken

Again, staff will pretest the questionnaire in the online format using 100 respondents to
ensure that all questions are easily understood and that the online procedure is sufficient to
generate reliable data. This pretest is also discussed in Part A and is part of the collection of
information for which staff seeks OMB approval.
(5)

Individuals Consulted on Statistical Aspect of the Study

The study design has been reviewed internally by Douglas Smith, an economist in the
Bureau of Economics; and Manoj Hastak, Ph.D., a faculty member in the Kogod School of
Business at American University and a consumer research consultant with the Bureau of
Consumer Protection’s Division of Advertising Practices. Dr. Hastak has served as a consultant
on experimental and survey studies for the FTC on numerous occasions. The contractor, Great
Lakes Marketing Research, is experienced in conducting statistically rigorous Internet-based
studies. The contractor has conducted numerous Internet-based consumer studies in the last five
years.

11