Why evaluating students’ use of AI writing tools is important: Actionable strategies for institutions

What does your approach to AI writing look like as we enter the 2024 academic year? Chances are it’s far less frantic than the previous year. And one year on from ChatGPT, you’ll ideally have a more consolidated approach to students’ use of AI writing tools that is, to some degree, reflected in your institution’s strategic vision and codified in academic policy.

Although the initial novelty of AI writing tools has worn off as the education sector becomes more familiar with their use – and in some cases, folds them into daily practice – the ongoing impact of generative AI on learning and teaching is still unfolding. While academics are answering the call to study the effects of AI-assisted writing on student learning practices in an emerging body of research, administrators and front-line educators face a similar endeavor.

Observing and understanding how students are engaging with AI writing tools beyond a surface level is a priority for institutions, and it’s a topic we explore in our new guide, ‘Why evaluating students’ use of AI writing tools is important: Actionable strategies for institutions’. Comprehensive in scope, it spans four key pillars:

1. safeguarding academic and integrity standards
2. cultivating students’ original thinking
3. empowering educators to drive change
4. adapting teaching and assessment for authentic learning

Safeguarding academic and integrity standards

Revising academic and integrity policies to capture the new reality of generative AI in education is crucial to maintaining a quality education offering and safeguarding an institution’s reputation into the future. Students’ access to AI writing tools is challenging tightly-held expectations around authorship, assessment methods, and learning artifacts; determining how and to what extent generative AI tools are permitted within the academic environment is the priority.

To this effect, our guide explores three key imperatives:

1. Adoption of AI writing tools is increasing, requiring a framework to govern their use.
2. Identifying AI-generated text serves to uphold institutional guidelines around fairness and accuracy in learning.
3. Turnitin’s AI writing detection supports institutions to maintain academic standards and integrity in the age of AI.

Indicative of AI’s momentum, Tyton Partners’ ‘Time For Class’ 2023 survey* and report on the North American demographic reveals that more than 50% of educators strongly believe GenAI tools will be needed for students’ future professional success. There is clearly a role for education in building students’ AI writing preparedness, and the task becomes reconciling mastery of these tools within the boundaries of the learning process.

At Turnitin, we recognize that for educators, there is a pressing and immediate need to know when and where AI writing tools have been used by students. Turnitin’s AI writing detection capabilities overcome human limitations in distinguishing what is human vs machine generated, to uphold the benchmarks institutions put in place to regulate AI writing tools and foster a culture of responsible use. Rather than viewing the results of our AI writing detection as a punitive measure, the overarching purpose of the tool is to facilitate student conversations and interventions on the use of AI writing tools that may not otherwise occur, and use it as one strategy in your toolkit.

*Turnitin was a partner in providing compensation to conduct this study.

Cultivating students’ original thinking

The advent of AI writing tools puts further pressure on institutions to strengthen the student experience and students’ workforce readiness. The drive to nurture student creativity and originality will arguably be even more acute as AI is embedded into society, but it does mean reshaping our expectations around authorship and what constitutes original student work.

To this effect, our guide explores three key imperatives:

1. AI writing is a skill students need to develop in order to thrive in the future of education and work.
2. Students are seeking guidance on AI writing within their courses and how they can upskill.
3. Turnitin’s solutions nurture students’ originality and critical thinking via formative feedback.

In an illuminating Hong Kong study on student perceptions of the use of GenAI in higher education, Chan and Hu (2023) discovered “a complex and nuanced picture of both enthusiasm and concerns”. Amongst the 399 surveyed students across 10 different faculties, students viewed the integration of AI as an essential part of university curricula and expressed the desire to be better equipped to compete and thrive in an AI-powered world.

Refining assessment methodology to draw out proof-of-learning that only a human can provide is one necessary step in empowering students, but the approach should also be balanced and encourage collaboration rather than deterring the use of generative AI tools outright. Making AI writing work for learning requires delineation as to where student and machine contributions begin and end, and top of mind is acting in the best interests of students when AI writing detection is applied.

Turnitin strives to maximize the effectiveness of our AI writing detection while keeping our false positive rate - incorrectly identifying fully human-written text as AI-generated - under 1% for documents with over 20% AI writing. It’s crucial to note that Turnitin does not make a determination of misconduct in our similarity checking or AI writing detection technology. Rather, we provide data for educators to make an informed decision based on their academic and institutional policies.

Empowering educators to drive change

Are you confident that your faculty are AI ready? Are the criteria for AI writing readiness understood and actively practiced? In these early stages of generative AI adoption there is much for institutions to learn and test, and it begins with an openness to make AI work for education, supporting educators with appropriate scaffolding to tackle challenges while empowering them to be agents of change.

To this effect, our guide explores three key imperatives:

1. Educator understanding and engagement with AI writing tools will help set the tone for their institution’s success.
2. Training teaching faculty to deliver AI-enabled pedagogy is an essential investment.
3. Turnitin’s AI writing tool and resources equip educators to tackle AI writing confidently.

According to Tyton Partners’ Time For Class 2023 report, faculty’s use of generative AI tools has jumped from 9% in March 2023 to 22% in September 2023. Although an improvement, the figure still lags behind students’ use, and inconsistencies in faculty’s use of generative AI tools has emerged. Therein lies a huge opportunity to invest in educators’ professional development towards fostering institutional innovation, and to mitigate inequalities and reputational harm that are likely to arise from differentiated access and division in the approach to generative AI.

At an instructional level, AI writing detection facilitates student-teacher exchanges by identifying any signatures of AI-generated text in student submissions to inform educators’ decision-making process. In the context where AI writing tools are not permitted in an assignment, Turnitin’s detection model can alert educators to possible non-compliance from students and trigger intervention efforts. In the scenario in which students are encouraged to harness AI writing, our AI detection provides insight into how students are engaging with the tools so that educators can better evaluate their skill development.

Dr. Leslie Layne, Associate Professor of English and Linguistics Coordinator of College Writing at University of Lynchburg uses Turnitin’s AI writing detection tool to facilitate authentic learning: "I teach with ChatGPT and get the students to use it to assist with structuring their writing and brainstorming. I don't use it [Turnitin] as ‘Gotcha!’ detection. It’s a heads up for me to follow up with my students and ask them to use more of their own voice.”

Adapting teaching and assessment for authentic learning

The assessment reform facing higher education has been accelerated by the advent of generative AI, but it’s also a response to broader realities in the digital age for which traditional assessment practices are no longer keeping pace. Given generative AI’s direct bearing on workforce demand and skill recalibration, authentic learning and assessment is poised to evaluate student achievement in a way that prepares them for professional, AI-enabled practice.

To this effect, our guide explores three key imperatives:

1. Assessment reform is a priority across the sector to dispense with methods that are no longer fit for purpose.
2. The shift to assessing the process rather than the product of learning, to ensure AI and industry readiness.
3. Turnitin’s integrity and assessment solutions facilitate authentic learning and AI-enabled assessment.

Unpacking generative AI’s implications for assessment design, Jason M. Lodge argues for higher education to move away from measurement of outputs which are now machine replicable, to evaluation of the learning process behind student output: “The ability to trace this journey, through the assessment of learning processes, ensures the ongoing relevance and integrity of assessment in a way that a focus on outputs cannot” (2023). In doing so, he also raises the importance of edtech tools to accomplish this with precision and at scale.

The role of AI writing detection is not one of substitution for assessment reform, but rather, a tool to understand how AI-generated text is presenting and indeed shaping students’ written work. The resulting data insights can paint a clearer picture of how their institution is being affected to inform decision-making amongst administrators, and in turn, educator-level interventions. An ability to reliably parse or analyze student versus machine contributions is also critical for high-stakes or accreditation-based assessment where an institution’s due diligence and reputation cannot be compromised.

In addition to our AI writing technology, Turnitin’s portfolio of integrity and assessment solutions addresses the time and resource-intensive elements of authentic, formative learning and assessment. It supports next steps for learning, and crucially, makes the exercise scalable. For instance, use of Turnitin Feedback Studio in tandem with educator-administered “Where to next?” feedback has been shown to scaffold students’ learning, leading to enhanced learner performance.

We’ve only scratched the surface of our full-length guide, which provides a holistic view of what department heads and education leaders ought to consider as they chart a course through the age of AI writing. To unpack further research data, insights and advice, download ‘Why evaluating students’ use of AI writing tools is important: Actionable strategies for institutions.’