Objective evaluation of second language learner’s translation proficiency using statistical translation measures
DOI:
https://doi.org/10.36505/ExLing-2008/02/0055/000114Keywords:
objective proficiency evaluation, word n-gram, translation probabilityAbstract
For objective evaluation of a second-language learner’s translation proficiency, we tested two measures commonly used in statistical machine translation. One is the word $n$-gram probability of a target language to measure the likelihood of translated sentences in that language. The other is the translation probability from source-language sentences to target-language sentences to measure translation accuracy between the sentence pairs. The subjective proficiency scores of Japanese learners were compared with these objective measures extracted from the English sentences they translated. Statistical analysis showed no correlation for the word $n$-gram probabilities, but a high correlation for the translation probabilities, which suggests their usefulness for the objective evaluation of learners' translation proficiency.
References
Papineni, K., Roukos, S., Ward, T. and Zhu, W. J. 2002. BLEU: a method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002), pp. 311-318. Philadelphia, Pennsylvania.
Yasuda, K., Sumita, E., Yamamoto, S., Yanagida, M., Maekawa, K. and Sugaya, F. 2003. A proposal for automatically gauging of English language proficiency. IPSJ SIG Technical Report, 2003-NL-155, 65-70.
Brown, P. F., Della Pietra, V. J., Della Pietra, S. A. and Mercer, R. L. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2), 263-311.
Takezawa, T., Sumita, E., Sugaya, F., Yamamoto, H. and Yamamoto, S. 2002. Toward a broad-coverage bilingual corpus for speech translation of travel conversations in the real world. Proceedings of the 3rd International Conference on Language Resources and Evaluation (LREC 2002), pp. 147-152. Las Palmas, Canary Islands, Spain.
Downloads
Published
Issue
Section
License
Copyright (c) 2008 Hajime Tsubaki (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles are published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited.
