Spoken Language Communication & Translation Use (TRANSTAC)

FLE Post-Scenario Survey

TRANSTAC Evaluation

Foreign Language Expert Post-Scenario Survey
Please supply responses that best match your experience for the scenario you just completed.
1. The system instructions provided a clear understanding of how to use the system.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

2. I found the system easy to understand in this interaction.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

3
Neutral

4
Agree

5
Strongly Agree

4
Agree

5
Strongly Agree

3. What the system said made sense to me.
0
N/A

1
Strongly Disagree

2
Disagree

4. The system made it easy to have this interaction.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

5. In this interaction, it was easy to get the information I needed.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

6. I knew what I could say or do at each point of this interaction.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

7. The system worked the way I expected it to in this interaction.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

8. The pronunciation of the was clear and understandable.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

9. The words were put together in a way that was coherent and
comprehensible.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

Thank you for your participation.

5
Strongly Agree

FLE Post-Scenario Survey

TRANSTAC Evaluation

10. When the system didn't understand me, it was easy to correct.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

11. Please include, if necessary, additional comments regarding the system's use of :
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
12. How widely will the spoken by the system be understood by other speakers?
0
N/A

1
Almost None

2
Few

3
Many

4
Most

5
Nearly All

13. How easily will other speakers be able to speak in a way that is
understood by the system?
0
N/A

1
Nearly
Impossible

2
With
Difficulty

3
Some
Effort

4
With Little
Effort

5
Effortlessly

14. How appropriate is the spoken by the system to the situation(s) simulated
in this session?
0
N/A

1
2
3
4
5
Completely
Somewhat Minimally
Rather
Completely
Inappropriate Inappropriate Acceptable Appropriate Appropriate

15. How much of the English could you hear during the dialog?
0
N/A

1
None

2
A Little

3
Some

4
A Lot

5
All

16. If you heard any of the English, what influence did it have on you?
0
N/A

1
None

2
Not Much

3
A Little

4
Some

5
A Lot

17. Please choose one of the following options to describe your experience in the interaction you
just completed.
0
N/A

1
Many
Problems

2
Some
Problems

3
A Few
Problems

4
Very Few
Problems

Thank you for your participation.

5
No
Problems

FLE Post-Scenario Survey

TRANSTAC Evaluation

18. If you experienced problems in the last dialog-interaction as reported above, please indicate
the cause(s) of these problems (check all that apply).
O Scenario content
O Scenario flow
O Translation system problems
O Human partner problem(s)
O Other (please describe): _________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________

19. Towards the end of the conversation, I was interacting with the system in a different manner
than at the beginning.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

20. If you changed the way you interacted with the system, it involved the following, please
check all that apply.
O Speaking more slowly
O Speaking more quickly
O Speaking more clearly
O Speaking more loudly
O Speaking more softly
O Waiting more time to speak once my “speech button” is pressed
O Waiting less time to speak once my “speech button” is pressed
O Using simpler and/or shorter sentences/phrases
O Using longer and/or more complex sentences/phrases
O Speaking more “naturally” for me
O Other (please describe): _________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________

21. If the translation system seemed to have problems with certain words or phrases, please list
any you can recall.
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
22. I would use this system in the field in its current state of functionality.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

23. Based on my experience in this interaction, I would recommend this system for future similar
interactions.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

Thank you for your participation.

5
Strongly Agree

FLE Post-Scenario Survey

TRANSTAC Evaluation

24. If the system had worked as intended in this interaction, I would recommend this system for
future similar interactions.
0
N/A

1
Strongly Disagree

2
Disagree

3
Neutral

4
Agree

5
Strongly Agree

25. What situations can you imagine that might best match this system's capabilities?
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
26. What did you like most about this system, or what was most helpful and useful?
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
27. What did you like least about this system?
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
28. What would you change about this system?
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
29. Please provide any additional comments here.
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

NOTE: This survey contains collection of information requirements subject to the Paperwork Reduction Act. Notwithstanding
any other provision of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to
comply with, a collection of information subject to the requirements of the Paperwork Reduction Act, unless that collection of
information displays a currently valid OMB control number. The estimated response time for this survey is 6.1 minutes. The
response time includes the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information." OMB Number: Expiration: .

Thank you for your participation.