Managing AI reliability in L2 writing: Designing a systematic framework

Abstract

Despite the widespread institutionalized application of AI tools as learning aids, the onus to manage and check the reliability of AI output, especially in writing, lies with the learners. This study conceptualizes reliability management in AI-assisted L2 writing as epistemic trust calibration. Based on this conceptualization, L2 writers need to continually evaluate the reliability of AI output for a given purpose, helping them decide whether to accept, revise, verify or reject such output. Thus, this paper proposes an epistemic trust calibration framework which integrates five theoretical constructs through a trust checking loop linking AI suggestions to writer judgment and outcomes. The framework also provides three operational tools: a risk taxonomy that identifies lower and high-risk zones, a five-step verification routine for high-risk AI output, and a reliability management rubric that operationalizes calibration as a developmental competence. This framework can be utilized to show how AI can boost learning when writers calibrate trust and verify high risk AI output. The study concludes with pertinent recommendations for future research directions.