Skip to content

How Accurate Is WriteProof? Test Results and Method

Short answer: WriteProof’s AI-pattern check is a rough, explainable signal that is tuned to avoid false accusations. In our test it never placed a human-written passage in its top band (“many AI-associated patterns”), but it also missed most of the AI-generated samples. No detector, including ours, can prove who wrote a text.

Test run: November 2, 2026. Page updated: November 3, 2026. These results describe the local pattern check used on the site in November 2026. The optional second estimate from Google’s Gemini model was not part of this test.

What the checker measures

The score combines three named signals: how evenly sentence lengths vary, how often common AI-associated phrases appear, and how often vocabulary repeats. They are combined into a 0–100 pattern score, which is shown in three bands:

  • Few AI-associated patterns: 0–24
  • Some AI-associated patterns: 25–49
  • Many AI-associated patterns: 50–100

The score is a pattern score, not a probability that a text was written by AI.

What we tested

We ran the exact scoring code the site uses on 72 English passages:

  • 48 human-written passages (113–245 words each), six from each of eight Wikipedia articles as they stood in December 2019, almost three years before ChatGPT’s public release in November 2022: Behavioral economics, Compiler, French Revolution, Impressionism, Marie Curie, Photosynthesis, Pride and Prejudice, and Sleep. Wikipedia text is available under the CC BY-SA license.
  • 24 AI-generated passages (123–145 words each), written by one AI model (Claude, from Anthropic) in six styles: academic (6), student-style essay (4), creative (4), technical (4), blog (3), and persuasive (3).

Results by band

BandHuman-written passages (48)AI-generated passages (24)
Few patterns (0–24)34 (71%)7 (29%)
Some patterns (25–49)14 (29%)13 (54%)
Many patterns (50–100)0 (0%)4 (17%)

Human-written passages scored 16.8 on average (median 15.5, highest 44). AI-generated passages scored 33.4 on average (median 27, lowest 19, highest 68).

If a score were used as a cutoff

Some people treat any score above a cutoff as a “flag.” This table shows what different cutoffs would have meant on our test set. We show it to explain the trade-off. We do not recommend using any cutoff to make decisions about a person.

CutoffAI samples flaggedHuman passages wrongly flagged
30 or more11 of 24 (46%)8 of 48 (17%)
40 or more6 of 24 (25%)2 of 48 (4%)
50 or more4 of 24 (17%)0 of 48 (0%)
60 or more1 of 24 (4%)0 of 48 (0%)

Overall ranking quality (AUC) was 0.82. In plain terms: if you pick one AI-generated passage and one human-written passage at random, the AI passage gets the higher score about 82% of the time (ties count as half).

Results by writing style (AI samples)

StyleSamplesAverage scoreIn the “many” band
Student-style essay4442
Academic6381
Persuasive3361
Technical4290
Blog3270
Creative4220

Creative writing was the hardest to spot: none of the four creative samples reached the “many” band, and three of them scored in the “few” band. With only three to six samples per style, these per-style numbers are especially rough.

What this means if you got a result

  • “Few patterns” does not show that a person wrote the text. 7 of the 24 AI samples (29%) scored there.
  • “Some patterns” is common for human writing. 14 of the 48 human passages (29%) scored there.
  • “Many patterns” is the strongest signal the checker gives, but it is still not proof. No human passage in this test reached it, but we did not test student essays, writing by non-native English speakers, or very formulaic writing, any of which may score higher.
  • The checker is cautious on purpose. A false accusation can hurt a real person, so we accept missing some AI text in exchange for fewer false flags.

A change we tested and did not ship

The current score weights the three signals 40% phrasing, 30% sentence-length evenness, and 30% vocabulary repetition. We also tested 45% phrasing, 45% sentence-length evenness, and 10% vocabulary repetition. That version ranked passages a little better overall (AUC 0.86) and put more AI samples in the “many” band (9 of 24 instead of 4). But it also put 3 of the 48 human passages (6%) in that band, where the current version put none. We kept the current version because we would rather miss AI text than wrongly flag a person.

Limitations

  • The test set is small: 72 passages.
  • All human passages are encyclopedia prose from one source. We did not test student essays, informal or creative human writing, or writing by non-native English speakers. Researchers have found that several popular AI detectors flag non-native English writing far more often than native writing (Liang et al., 2023), so do not assume our results carry over to that kind of writing.
  • All AI passages came from one AI model and were not edited or paraphrased by a person. Text from other models, or AI text that a person has edited, may score differently.
  • The passages were short (113–245 words). Longer texts may behave differently.
  • We built the checker and ran this test ourselves, so this is not an independent evaluation.
  • The optional Gemini estimate shown with some results was not tested.

If we change how the score is calculated, we plan to rerun this test and update this page. Until then, these results apply only to the version described above.

Help us test better

Our biggest gap is real student writing, especially by multilingual writers. If you have writing you produced yourself and might be willing to share for a future test, email contact@evalaitools.com with a short description of what you have. Please do not attach any writing to your first message.

Try the checker · About WriteProof · For educators