Taghian, M. (2024). Assessing the Accuracy Criteria of AI Tools-aided Translation: A Case Study of Two-word Prophetic Hadiths. CDELT Occasional Papers in the Development of English Education, 87(1), 217-262. doi: 10.21608/opde.2024.384369
Muhammad A. A. , Taghian. "Assessing the Accuracy Criteria of AI Tools-aided Translation: A Case Study of Two-word Prophetic Hadiths". CDELT Occasional Papers in the Development of English Education, 87, 1, 2024, 217-262. doi: 10.21608/opde.2024.384369
Taghian, M. (2024). 'Assessing the Accuracy Criteria of AI Tools-aided Translation: A Case Study of Two-word Prophetic Hadiths', CDELT Occasional Papers in the Development of English Education, 87(1), pp. 217-262. doi: 10.21608/opde.2024.384369
Taghian, M. Assessing the Accuracy Criteria of AI Tools-aided Translation: A Case Study of Two-word Prophetic Hadiths. CDELT Occasional Papers in the Development of English Education, 2024; 87(1): 217-262. doi: 10.21608/opde.2024.384369
Assessing the Accuracy Criteria of AI Tools-aided Translation: A Case Study of Two-word Prophetic Hadiths
Lecturer of Applied Linguistics/Translation, Faculty of Arts, Helwan University, Egypt.
Abstract
The surge of advanced neural machine translation (NMT) tools like Google Translate and large language models (LLM) like ChatGPT and Gemini has fueled their application across diverse religious texts, like prophetic hadiths. However, the effectiveness of these tools, particularly LLMs, in the domain of religious translation and their understanding of classical Arabic (CA) remains largely unexplored. This study bridges this gap by comprehensively assessing the machine translation (MT) accuracy of these three systems in translating 25 succinct, two-word prophetic hadiths from Arabic into English. These two-word hadiths, categorized as Jawāmiʿ al-kalim (comprehensive words), encapsulate profound meanings despite their brevity. This study delves deeper by evaluating the ability of Google Translate, ChatGPT (GPT-3.5), and Gemini to capture the essence of these two-word hadiths, considering their unique linguistic features and cultural context. Through meticulous evaluation by Subject Matter Experts (SMEs)/translators, the study reveals recurring challenges in MT, including word sense disambiguation, handling out-of-vocabulary terms, part-of-speech tagging, morphological segmentation, and comprehending the contextual, cultural, lexico-semantic, and pragmatic dimensions. These limitations were particularly evident with GT, whose reliance on literal translation often resulted in inaccurate renderings in the target language. Despite these challenges, the potential of LLMs for religious translation remains promising, especially with AI tools like Gemini showing promise. Their ability to learn and adapt finds that with further development, including domain-specific fine-tuning, incorporation of diacritics and contextual information via accurate prompts, development of religious text-specific evaluation metrics, and human post editing, MT systems can significantly be improved. It also underscores the vital role of human expertise while paving the way for future research to develop more accurate and culturally sensitive MT systems for religious communication across languages and faiths. The study concludes that LLMs displayed promise in capturing the essence of certain hadiths.
View on SCiNiTO
Top