Using corpora for post-editing neural MT in highly specialised domains: the case of complex noun phrases.” - Université Paris Cité Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Using corpora for post-editing neural MT in highly specialised domains: the case of complex noun phrases.”

Résumé

The present study focuses on the use of specialised comparable corpora for post-editing machine translation in LSPs, a recent development brought to the methodological framework which was introduced in our translation syllabus 20 years ago and which has been greatly improved over the years (Kübler 2003, Kübler 2011, Kübler et al. 2016, Kübler et al. 2018). This framework for teaching specialised translation to master’s students involves a wide range of competences. Students have classes in corpus linguistics, corpus query tools, terminology and specialised translation. An experimental protocol with the Earth and Planetary Sciences (EPS) department was created to put the framework into practice. Our translation trainees first collaborate with EPS students on the terminology of highly specialised English-language articles and then translate excerpts from the articles into French. Since 2013, our team has been assessing the impact of corpus use on students’ translations through the study of learner corpora. Every year, we assemble two translator learner corpora containing translations produced 1) without using corpora, and 2) using all available resources, and specifically the specialised comparable corpora the students compile during terminology analysis. Both student translations sub-corpora are annotated with the MeLLANGE error typology (Castagnoli et al. 2011) on a Brat server. A previous comparison of the two shows a positive impact of corpus use on some error categories, but limited impact on other errors categories, among which Distortion, which consists in an erroneous or an inadequate transfer of meaning from source language to target language mainly affecting complex noun phrases (Kübler et al. 2016, 2018). Our presentation aims to contribute to research in corpus-based translation studies (E.G. Zanettin 1998, Aston 1999, Maia 2003, ) and more particularly to research targeting the evaluation of the usefulness of corpora for translation teaching (E.G. Beeby et al. 2009, Castagnoli et al. 2011, Frankenberg-Garcia 2015). It also offers insights into research and practices based on translation learner corpora (E.G. Granger & Lefer 2018, Bowker & Bennison 2003, Loock et al. 2014) and, most significantly, on the problems of machine translation and post-editing processes (CF. O'Brien et al. 2014, Toral 2019, Toral et Sanchez-Cartagena 2017, Scansani et al. 2019). For the past two years, a further component has been introduced in our framework, i.e. post-editing MT output using corpora. As stated in the European Master’s in Translation (EMT) Competence Framework (2017), “the ability to interact with machine translation in the translation process is now an integral part of professional translation competence” - hence the need to train translation students for this task. The development of neural MT, which is state-of-the art in machine translation today, clearly increases quality compared with the previous generation of statistical engines (Toral & Sanchez-Cartagena 2017), but shows conflicting results when evaluated, that is post-edited, by humans (Castilho et al. 2017, Toral & Sanchez-Cartagena 2017). Even though professional translators still seem to mistrust neural MT, Scansani et al. (2019) show that students in translation show no difference in trust when revising human translation or post-editing MT. Most of our trainee translators are familiar with MT, and have a rather positive image of the technology (Kübler et al. to appear). Our previous results of comparative analysis of human-translated vs post-edited texts in highly specialised domains shows that the intervention of neural MT leads to specific errors, however it globally results in improvements in both productivity and quality (Martikainen & Mestivier 2019, Kübler et al. in press). Within this framework, we have previously studied the use of corpora in the translation of complex noun phrases (CNP), which are characteristic of the highly specialised texts our students work on. Our previous results show that CNPs are a frequent source of distortion in translation. In the present research, we focus on the impact of neural MT on the translation of CNPs in highly specialised domains and on the use of corpora in the post-editing process of this type of construction in particular. During the 2018-2020 academic years, we have revised our procedure for evaluating the impact of corpus use on translation: error-annotating student translations now includes annotating MT output and post-edited texts. Students thus submit several texts: a) the translation of a 250 word fragment for which they may only use available term-bases and Internet searches b) a second version of this translation improved by using corpora, c) the translation of a 250 word fragment achieved by post-editing MT output (aligned with the MT output) and d) and improved version achieved by using corpora. The comparison of the four text-types and students’ comments on the translation process allow us to formulate tentative answers for several research questions: How well do MT systems perform on the translation of CNPs? Are translation error typologies adequate for annotating MT output and post-edited texts? What are the differences between MT and human translation where CNPs are concerned and how does the former influence the latter when post-editing? Is corpus use efficient in post-editing MT when translating CNPs (studied mainly through student comments)? Can we devise better adapted teaching material to draw student’s awareness to the CNP translation difficulties? Regarding neural MT performance on complex NPs, preliminary results suggest that the calque solutions produced by the generic MT engine, although adapted in some contexts, frequently require some amount of post-editing in this highly specialised domain. In some cases, minimal post-editing effort is required, as for the segment “an exhumed serpentinite-sediment contact”, for which MT output is “un contact serpentine-sédiment exhumé”. A single edit operation would be necessary here, to replace the erroneous term “serpentine” by the correct one, “serpentinite”. Trainee translators, however, sometimes over-edit such calque translations of CNPs, specifically when they contain specialised terms derived from general vocabulary or from other specialised domains, such as “contact” and “exhumé” in this example, both of which can be observed in the specialised comparable corpora. In other instances, MT calque solutions for complex NPs require extensive post-editing. This is the case for the segment “hydrothermally altered rocks”, for which the translation produced by the generic MT engine (“roches altérées *hydrothermiquement”) contains a non-existent word, or a ‘hallucination’ (Moorkens 2020), namely “hydrothermiquement”. When the complex NP is composed of more familiar elements, as is the case here, students tend to under-edit MT output. In this second example, replacing the erroneous term by the correct one (“hydrothermalement”) is not sufficient for producing an acceptable translation. Research in the specialised comparable corpora shows that, in this instance, it is necessary to unpack the compact construction by adding either a verbal construction (e.g. “roches ayant subi une altération hydrothermale”) or a prepositional phrase (e.g. “roches altérées par hydrothermalisme”). These preliminary results thus consolidate our previous results on the usefulness of comparable corpora for LSP translation, and confirm the need to train translation students on using corpora for post-editing as well.
Fichier non déposé

Dates et versions

hal-04063844 , version 1 (10-04-2023)

Identifiants

  • HAL Id : hal-04063844 , version 1

Citer

Natalie Kübler, Hanna Martikainen, Alexandra Mestivier, Mojca Pecman. Using corpora for post-editing neural MT in highly specialised domains: the case of complex noun phrases.”. Using Corpora for Contrastive and Translation Studies UCCTS, Université de Bologne, Sep 2021, Bertinoro, Forli, Italy. ⟨hal-04063844⟩
35 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More