Findings and Recommendations from Cognitive Testing for the Quarterly Summary of State and Local Government Tax Revenue (QTAX)(Excerpt)
Prepared for:
Tyler Kerr, Economy-wide Statistics Division (EWD)
Kristina Frates, EWD
Prepared by:
Kristin Stettler, Data Collection Methodology and Research Branch (DCMRB)
Krysten Mesner, DCMRB
Economic Directorate
U.S. Census Bureau
September 30, 2020
Item Specific Findings and Recommendations
Finding #1: Items 4h, i, and j are clear
Finding #2: Creating codes and separate accounts for marijuana sales would require effort
Finding #1: Item 5h was clear to participants
Finding #2: Many states do not have a separate code for Marijuana licenses
Appendix B: New Tax Code Descriptions for F-72 Classification Manual
Appendix C: Interviewer Protocol
The Quarterly Summary of State and Local Government Tax Revenue (QTAX, F-72) provides quarterly estimates of state and local government tax revenue at a national level, as well as detailed tax revenue data for individual states. The redesign of the QTAX proposes to add several new data items to include new or different forms of sales and license taxes. The State Finance and Tax Statistics Branch proposed these new items due to increasing requests for their publication from data users. The proposed new data items have been adopted across many states, and continue to increase as a percentage of state total tax collections. The U.S. Congress, federal agencies, state and local governments, education and research organizations, investment and banking institutions, and the general public employ these results. Some major uses include the following:
Development of government component of the gross domestic product estimates
Development of the national income accounts
Tax policy research
The draft questions were discussed with current QTAX respondents using cognitive interviewing. The objectives of these interviews were to learn about the following:
Do respondents understand what information they are being asked to provide with these new questions?
Do respondents understand the terminology used in the questions?
Are respondents able to answer the questions that are being asked?
Is the requested information available in respondent records?
Are the instructions helpful to respondents?
The questionnaire draft can be found in Appendix A. The new tax code description for F-72 classification manual can be found in Appendix B. The interviewer protocol can be found in Appendix C.
From January through May of 2020, staff from the U.S. Census Bureau’s Data Collection and Methodology Research Branch (DCRMB) conducted 14 cognitive interviews over one round of testing, regarding proposed new questions. Participants in the interviews were recruited from a list of recent respondents to the Quarterly Summary of State and Local Government Tax Revenue. Generally, those recent respondents were the primary participants in the interviews, but on occasion these respondents brought one or more colleagues to the interviews, because they were integral to the typical survey data collection process. On occasion, these people who accompanied the original participant turned into the primary participant in these interviews. Interviews were conducted over the phone.
Cognitive interviews were the testing methodology used for this project. Cognitive interviews are used to, ‘(a) understand the thought processes used to answer survey items, and (b) to use this knowledge to find better ways of constructing, formulating, and asking survey questions’ (Forsyth and Lessler, 1991).1 Cognitive interviews traditionally focus on the four steps of Tourangeau’s (1984) cognitive response model: comprehension, retrieval, judgment, and communication/reporting.2 Comprehension refers to the respondent’s interpretation and understanding of the question’s language, structure, and grammar. In order to answer the question, a respondent must understand what information is being requested on the survey. Retrieval is the step where relevant information is obtained, either from records or from memory. The next step, judgment, describes the respondent’s evaluation of the completeness or relevance of the data obtained. It is here that estimates are made based on partial or incomplete data. The last step, communication or reporting, deals with mapping the response to the answer space provided and possibly altering the answer.
While Tourangeau’s model is suitable for household and social surveys, the establishment survey setting presents additional factors that must be considered. First, instead of or in addition to a reliance on memory, establishment surveys rely heavily on records and the information contained within them. Second, organizations tend to have distributed knowledge. Some people are experts in one type of information, while others keep information about something else. Third, competing priorities, both for the organization and the individual(s) completing the questionnaire, mean that the survey sometimes does not receive the amount of attention that researchers and data collectors would like. Finally, organizations regularly authorize only a few individuals to release data. If the data provider is not authorized to release the data, an additional step must be added to the response process. Tourangeau’s model was expanded by Sudman et al (2000) to account for these factors. 3
Participants were current QTAX respondents. Interviews were conducted with a variety of sizes and types of state governments.
Table 1. State governments participating in this study
State governments |
Total: 14 |
The job titles of participants varied, including Financial Reporting Supervisor, Accounting Officer Senior, Accountant, Chief Economist, Assistant Director of General Accounting, State Financial Consultant, Management Auditor, Director of the Bureau of Research, Manager, Federal Reporting Team Lead, Supervisor of Financial Reporting and Management, Financial Reporting Manager, Accountant, Economist, Administrator, Financial Reporting Supervisor, and Finance Administrator.
Items 4h, i, and j, Medical Marijuana sales, Recreational marijuana sales, and Total marijuana sales (h plus j)
The concept of 4h, i, and j are clear to participants. The definition of these items matched the participants’ personal definitions of these items.
For the states that have legalized recreational and/or medical marijuana, many are not sure where this data would come from or if marijuana is differentiated by recreational or medical. It would be a process for many states to create separate accounts and parse out marijuana by type of use.
A couple participants noted that they had some confusion about the distinction between general sales tax and a specific gross receipts tax on marijuana. They had to work through whether general sales taxes on marijuana should be reported here, even though it was already reported in Item 2a. This seemed like a more general issue as well.
Most participants understood the concept of Marijuana licenses, though there was some confusion re: whether permits or fees should be included.
For states where marijuana use is legalized, many states reported that they do not have a separate code for marijuana licenses. Several participants noted they do have access and could easily break out. Several others said they would need to reach out to others to see if this is obtainable.
F-72 draft (8-14-19) |
|
|
|
|
OMB No.: 0607-0112 Approval Expires: 08/31/2019 |
||||
ID |
|
State name |
|
|
|
Period covered by this |
Calendar Qtr. |
1st-2019 |
|
|
|
|
|
|
|
report--------------------------> |
Fiscal Qtr. |
|
|
|
TAX COLLECTIONS |
|
|
|
|
|
Omit cents |
|
|
|
|
|
|
|
|
|
|
|
|
(Net of refunds, except when otherwise indicated) |
|
|
|
|
|
|
|||
Please report either in terms of 3-month totals or |
|
|
|
|
|
|
|||
in terms of individual month amounts, whichever is |
|
|
|
|
|
|
|||
more convenient. |
|
|
|
|
Total |
January |
February |
March |
|
1. Property taxes |
|
|
|
|
|
|
|
|
|
|
a. Directly State-collected |
|
|
T01 |
|
|
|
|
|
|
b. Received from local government collection |
|
|
|
|
|
|||
|
agencies |
|
|
|
T02 |
|
|
|
|
2. General sales and gross receipts taxes |
|
|
|
|
|
|
|||
(and related use taxes) |
|
|
|
|
|
|
|
||
|
a. State imposed taxes (plus any retained |
|
|
|
|
|
|||
|
portion of local supplements) |
|
T09 |
|
|
|
|
||
|
b. Collections of locally imposed supplements |
|
|
|
|
|
|||
|
to State general sales taxes (net of |
|
|
|
|
|
|
||
|
State-retained amount) |
|
|
|
|
|
|
|
|
|
c. Online sales and gross receipts taxes |
|
|
|
|
|
|||
|
Portion of 2a collected from internet-based |
|
|
|
|
|
|||
|
retailers and "remote sellers" |
|
|
|
|
|
|
||
3. Motor fuel sales taxes |
|
|
|
|
|
|
|
|
|
|
a. Gross collections |
|
|
|
|
|
|
|
|
|
b. Refunds during period |
|
|
|
|
|
|
|
|
|
c. Net collections (a minus b) |
|
T13 |
|
|
|
|
||
4. Selective sales and gross receipts taxes, |
|
|
|
|
|
|
|||
other than motor fuel |
|
|
|
|
|
|
|
|
|
|
a. Alcoholic beverages |
|
|
T10 |
|
|
|
|
|
|
b. Public utilities |
|
|
T15 |
|
|
|
|
|
|
c. Insurance |
|
|
|
T12 |
|
|
|
|
|
d. Tobacco products |
|
|
T16 |
|
|
|
|
|
|
e. Pari-mutuels |
|
|
T14 |
|
|
|
|
|
|
f. Amusements |
|
|
|
T11 |
|
|
|
|
|
g. Sports betting |
|
|
|
T18 |
|
|
|
|
|
h. Medical marijuana sales |
|
|
|
|
|
|
|
|
|
i. Recreational marijuana sales |
|
|
|
|
|
|
||
|
j. Total marijuana sales (h plus i) |
|
T17 |
|
|
|
|
||
|
k. Other selective sales and gross receipts |
|
|
|
T19 |
|
|
|
|
5. License taxes |
|
|
|
|
|
|
|
|
|
|
a. Alcoholic beverages |
|
|
T20 |
|
|
|
|
|
|
b. Public utilities |
|
|
T27 |
|
|
|
|
|
|
c. Motor vehicle |
|
|
T24 |
|
|
|
|
|
|
d. Motor vehicle operator |
|
|
T25 |
|
|
|
|
|
|
e. Corporations in general |
|
|
T22 |
|
|
|
|
|
|
f. Hunting and fishing licenses |
|
T23 |
|
|
|
|
||
|
g. Amusements |
|
|
T21 |
|
|
|
|
|
|
h. Marijuana licenses |
|
|
T26 |
|
|
|
|
|
|
i. Occupation and business licenses not |
|
|
|
|
|
|
||
|
elsewhere classified |
|
|
T28 |
|
|
|
|
|
|
j. Other license taxes |
|
|
T29 |
|
|
|
|
|
6. Individual income taxes |
|
|
|
|
|
|
|
||
|
a. Gross collections |
|
|
|
|
|
|
|
|
|
b. Refunds during period |
|
|
|
|
|
|
|
|
|
c. Net collections (a minus b) |
|
T40 |
|
|
|
|
||
7. Corporation net income taxes |
|
|
|
|
|
|
|
||
|
|
|
|
|
T41 |
|
|
|
|
8. Death and gift taxes |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
T50 |
|
|
|
|
9. Severance taxes |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
T53 |
|
|
|
|
10. Documentary and stock transfer taxes |
|
|
|
|
|
|
|||
|
|
|
|
|
T51 |
|
|
|
|
11. Other miscellaneous taxes |
|
|
|
|
|
|
|
||
|
|
|
|
|
T99 |
|
|
|
|
CENSUS USE ONLY |
|
|
|
|
13. Remarks |
|
|
||
|
|
|
|
|
T77 |
|
Local |
|
|
12 |
Name |
|
|
|
|
|
Total |
|
|
DATA |
Title |
|
|
|
|
|
|
|
|
SUPPLIED |
|
|
|
|
|
|
|
|
|
BY |
|
|
|
|
|
|
|
|
|
|
Telephone |
Area Code |
Number |
|
Extension |
|
|
|
|
New Tax Code Descriptions for
F-72 Classification Manual
Code T09 General Sales and Gross Receipts Taxes
Definition: Taxes applicable with only specified exceptions (e.g., food and prescribed
medicines) to sales of all types of goods and services or to all gross receipts, whether at a single rate or at classified rates; and sales use taxes.
Includes: This includes sales or gross receipts taxes on the purchase or lease of motor vehicles, if there is no specific and separate tax law covering this activity.
Excludes: Taxes imposed distinctively on sales of or gross receipts from selected
commodities, services, or businesses (report at appropriate Selective Sales and Gross Receipts Taxes, codes T10 - T19). If a sales tax on vehicles is authorized by a law distinctly separate from a general sales tax law, use Other Selective Sales and Gross Receipts, code T19. Also excludes sales tax from internet-retailers (online) and remote sellers (use Online Sales and Gross Receipts Tax, Code T06).
Code T17 Marijuana Sales Tax
Definition: Taxes on marijuana and marijuana products, including both medical and recreational purposes. Medical marijuana is defined as taxes on marijuana and marijuana products sold for use as a medicinal tool as prescribed by a doctor/medical professional. Recreational marijuana is defined as taxes on marijuana and marijuana products sold for use other than medical purposes.
Excludes: Criminal or civil fines for possession of marijuana in excess of allowable amount (use Fines and Forfeits, code U30). For states that do not have a specific marijuana and marijuana products tax rate, and charge at the general sales tax rate (use General Sales and Gross Receipts, code T09).
Code T26 Marijuana License Tax
Definition: Licenses for growing, importing, wholesaling, manufacturing, retailing, and use of marijuana products.
Excludes: Taxes based on volume or value of transactions (report at Marijuana Sales Taxes, code T17) or on assessed value of property (use Property Taxes, code T01).
Draft Cognitive Interview Protocol, 8/14/19
(Kristin Stettler and Krysten Mesner)
Research Questions:
Do respondents understand what information they are being asked to provide with these new questions?
Do respondents understand the terminology used in the questions?
Are respondents able to answer the questions that are being asked?
Is the requested information available in respondent’s records?
Are the instructions helpful to respondents?
Expected Length of Interview: 30-60 minutes
Materials Needed:
Draft questions
New definitions from Classification Manual
Consent forms
Digital recorder
General probes that may be used throughout the interview:
Reflect back on respondent’s answer: “you said…”
In your own words, what is this question asking?
How did you arrive at this number/answer this question?
What records (if any) did you look at? What line or lines were of interest?
Specifically, what did you include in this number? What did you exclude?
Would you consult other people to obtain this answer?
INTRODUCTION
Explain purpose of meeting: get feedback on potential redesigned questions for the Quarterly Summary of State and Local Government Tax Revenue (F-72).
There may be times throughout this interview that this seems like a test of you, I assure you that it is not. The purpose of these interviews is to ensure that we are communicating the data requests clearly with you, so in that way, these interviews are actually a test of us. On that note, we want your honest feedback throughout, and if it’s not clear to you how to answer, please do your best. There will be ample opportunities for you to let us know how to make improvements so that you could do a better job answering.
ABOUT THE RESPONDENT(S)
First, I’d like to learn a bit about each of you, and your roles at your organization.
What is your job title? What is your background, in terms of education and job experience?
What is your role in completing government surveys?
How much experience do you have completing the QTAX survey?
Do you work with anyone else in your organization to get the data for the QTAX survey?
How are those other people involved?
How many people are involved?
OVERALL QUESTIONS
Before we get started with our draft questions, I want to ask you to about your financial records.
What sorts of financial records are available?
What type of financial records systems do you have?
[If in person] Are you able to show us your system(s)?
COGNITIVE INTERVIEW QUESTIONS
Thank you for that background information! Now I’d like to have you review the current version of the QTAX survey, just to refresh your memory.
What was your experience completing this survey?
How long would you estimate it takes to complete, each quarter, including gathering data, etc.? What takes the longest?
Have you looked through the Classification Manual? Just once, or do you refer to it regularly?
Do you have any comments or suggestions for us regarding the current form?
Next, I am going to show you the revised version that we have drafted, along with some new definitions for the Classification Manual. Glancing through the form, what is your first impression?
Next let’s look at Items 4h, i, and j, Medical marijuana sales, Recreational marijuana sales and Total marijuana sales (h plus i).
In your own words, what are we asking for here?
How would you answer these questions?
Where would the data come from in your records?
Do you have this information in your records?
What difficulties might you have retrieving this data?
What types of things would you be including here?
What does “medical marijuana” and “recreational marijuana” mean to you?
Do you have any other terms for this?
Is there any additional information you would like to see provided here?
Would you be likely to look at the definition in the Classification Manual?
Looking at the definition in the Classification Manual, what is your reaction?
Would you like any additional information or clarifications?
Could anything be deleted?
If you were not able to provide medical and recreational marijuana separately, would you be able to provide the total, in j?
Where was this being recorded in previous forms?
Do you have any other thoughts or suggestions on this item?
OK, let’s look at Item 5h, Marijuana licenses.
In your own words, what are we asking for here?
How would you answer this question?
Where would the data come from in your records?
Do you have this information in your records?
What difficulties might you have retrieving this data?
What does “Marijuana licenses” mean to you?
Do you have any other terms for this?
What types of things would you be including here?
Are individual licenses included here, or somewhere else?
Is there any additional information you would like to see provided here?
Would you be likely to look at the definition in the Classification Manual?
Looking at the definition in the Classification Manual, what is your reaction?
Would you like any additional information or clarifications?
Could anything be deleted?
Where was this being recorded in previous forms?
Do you have any other thoughts or suggestions on this item?
Classification Manual
Now I would like to review the current Classification Manual.
In general, what do you think of the definitions?
Are there additional definitions/instructions you think would be helpful?
Are there any definitions/instructions you think are NOT necessary?
Do you have any suggestions for improvement?
WRAP-UP
Wrap-up
We are almost done, and I just have a few wrap-up questions.
Overall, what did you think about the new questions we reviewed?
Do you have any suggestions for improvement?
Do you think these questions would be easy to answer, difficult to answer, or somewhere in between? Why?
Could you provide an estimate of how long you think it would take you to answer just the questions we reviewed today? What is driving that estimate?
Compared to the current version of the QTAX form, what do you think of this revised version? Would this version be easier or harder for you to complete?
Is there anything else we haven’t touched on today, that you think is relevant?
Do you have any questions for us?
Thank you very much for your time today!
1 Forsyth, B.H. and Lessler, J.T. (1991). ‘Cognitive Laboratory Methods: A Taxonomy.’ In Measurement Errors in Surveys, P.P. Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowitz, S. Sudman (eds). New York: John Wiley & Sons, Inc.
2 Tourangeau, R. (1984). ‘Cognitive Sciences and Survey Methods.’ In Cognitive Aspects of Survey Methodology, T.B. Jabine, M.L. Straf, J.M. Tanur, and R. Tourangeau (eds). Washington, DC: National Academy Press.
3 Sudman, S., Willimack, D.K., Nichols, E., and Mesenbourg, T.L. (2000). ‘Exploratory Research at the U.S. Census Bureau on the Survey Response Process in Large Companies.’ Paper prepared for presentation at the Second International Conference on Establishment Surveys, Buffalo, NY.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kristina Marie Pasquino Frates (CENSUS/EWD FED) |
| File Modified | 0000-00-00 |
| File Created | 2021-03-24 |