DARPA BOLT Phase 3 Speech-to-Speech Machine Translation - Bilingual Assessor Questionnaire

Bilingual Expert Questionnaire

This statement will be printed on the top of each Bilingual Expert questionnaire.

This collection of information contains Paperwork Reduction Act (PRA) requirements approved by the Office of Management and Budget (OMB). Notwithstanding any other provisions of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to comply with, a collection of information subject to the requirements of the PRA unless that collection of information displays a currently valid OMB control number. Public reporting burden for this collection is estimated to be 13 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed and completing and reviewing the collection of information. Send comments regarding this burden estimate or any aspect of this collection of information, including suggestions for reducing this burden, to the National Institute of Standards and Technology, Attn: Kay Peterson, MS 8940, 100 Bureau Dr., Gaithersburg, MD 20899.

OMB Control No. 0693-0043

Expiration Date: 10-31-2016

Bilingual experts will review several dialogs, both with and without the system solicited clarification option, during the

post-evaluation assessment period. After reviewing each dialog with the system solicited clarification option turned ON,

the bilingual experts will answer the following three questions.

Instructions: Please rate the following statements.

Dialog ID	Question	1 Strongly Disagree	2 Disagree	3 Slightly Disagree	4 Neutral	5 Slightly Agree	6 Agree	7 Strongly Agree	N/A
(example) I_S01F01_CLA_T0001_UT000	The initiator of the dialog achieved his/her goal.
	The clarification(s) helped during the dialog.
	The overall quality of the translations was adequate.

After reviewing each dialog with the system solicited clarification option turned OFF, the bilingual experts will answer the following two questions.

Instructions: Please rate the following statements.

Dialog ID	Question	1 Strongly Disagree	2 Disagree	3 Slightly Disagree	4 Neutral	5 Slightly Agree	6 Agree	7 Strongly Agree	N/A
(example) I_S01F01_NOC_ T0001_XY012	The initiator of the dialog achieved his/her goal
	The overall quality of the translations was adequate.