For Educators: Handling AI-Detection Flags Fairly
WriteProof is built for writers whose work has been questioned by an AI detector. This page is for instructors, teaching assistants, and academic-integrity staff. It explains what WriteProof’s records can and cannot show, and suggests a fair way to handle a flag.
Why a detector score should not decide a case
- Detector makers say so themselves. Turnitin’s guidance says its AI writing detection model “may not always be accurate” and “should not be used as the sole basis for adverse actions against a student.”
- Detectors make mistakes in both directions. OpenAI reported that its own AI text classifier correctly identified 26% of AI-written text while labeling human-written text as AI-written 9% of the time. It withdrew the classifier in July 2023 “due to its low rate of accuracy.”
- Some writers are flagged more often. In a 2023 study, seven widely used detectors labeled, on average, 61% of TOEFL essays written by non-native English speakers as AI-generated (Liang et al.).
- Small error rates add up. When Vanderbilt University disabled Turnitin’s AI detector in 2023, it noted that a 1% false positive rate, applied to the roughly 75,000 papers it submitted in 2022, could have meant around 750 papers incorrectly labeled.
- Our own checker is no exception. In our test it never placed a human-written passage in its top band, but it missed most AI samples, and it has not been tested on student writing. See our test results and method.
A fair process
- Treat a score as a reason to look closer, not as a finding. Read the passages the tool points to and ask whether they are ordinary features of the assignment, such as the fixed structure of a lab report or a five-paragraph essay.
- Talk with the student before deciding anything. Ask open questions about how they researched, drafted, and revised the work, and why they made particular choices.
- Ask for process evidence, and accept it in any form. Drafts, notes, outlines, sources, and version history from Google Docs or Microsoft Word can all help. Some students write offline, on shared or school computers, or in apps without version history, so missing history does not by itself show misconduct.
- Check what was actually allowed. Grammar checkers, translation tools, and assistive technology may be permitted, and AI policies differ from course to course.
- Follow your institution’s procedure. Give the student a chance to respond in writing, and record what evidence you relied on and why.
What WriteProof records can and cannot show
| Item | What it shows | What it does not show |
|---|---|---|
| AI-pattern score | How strongly a text shows three measurable patterns, with the sentences that influenced the score | Who wrote the text, or a probability that AI was used |
| Writing Timeline | Versions of a draft saved in the writer’s own browser | Anything verified: the writer can edit or delete these versions |
| Writing Session Record | A server-signed summary of activity reported by the writer’s browser (elapsed time, number of editing sessions, the share of characters typed versus pasted, and final word count) plus a SHA-256 fingerprint of the final text | Who was typing. The activity was reported by the browser and is not independently verified. |
| Defense Kit report | The writer’s own account of their process, together with any results, summaries, and links they chose to include | Anything checked by WriteProof. It is a document the writer prepared. |
| Draft comparison, Word metadata, and reference check summaries | What changed between two versions; the dates and editing time stored in a .docx file; whether references match records in Crossref | Authorship. File properties can be changed, and a reference missing from Crossref is not necessarily fake. |
How to check a Writing Session Record
Each record has its own link on this site. Opening it re-checks the record’s signature against WriteProof’s stored values, which confirms that the fingerprint and statistics have not changed since the record was created. For current records, the page also has a box where you can paste the submitted text. Your browser computes its fingerprint locally, without sending the text anywhere, and tells you whether it is identical to the text the record was created from. A match shows that the text is the same. It does not show who wrote it.
Ways to make the writing process visible
- Ask for staged submissions: a proposal, an outline, a draft, and a final version.
- Include a short in-class writing task or a brief conversation about major assignments.
- Ask for a short reflection on how the work was produced, including any tools used.
- State clearly in the syllabus which tools are allowed, for which tasks, and how students should disclose them.
Sources
- Turnitin, “Using the AI Writing Report,” Turnitin Guides.
- OpenAI, “New AI classifier for indicating AI-written text” (updated July 20, 2023).
- Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., and Zou, J. (2023). “GPT detectors are biased against non-native English writers.” Patterns.
- Vanderbilt University, “Guidance on AI Detection and Why We’re Disabling Turnitin’s AI Detector” (August 16, 2023).
Questions or corrections about this page: contact@evalaitools.com.