The AI Compilatio content detector allows to make the difference between human texts and texts generated by Artificial Intelligences, thus verifying the authenticity of written documents.
1. How do you measure the reliability of the Compilatio system for detecting texts written by AI?
The effectiveness of AI content detection relies on both:
Detection engine capacity to label "AI" or "human" each of the extracts from consistent text presented to it (texte homogeneous = text entirely generated by AI or text entirely written by human).
Compilatio analysis capability to identify in a heterogeneous document passages attributable to a human or AI author (heterogeneous text = text containing a mixture of text written by a human and text generated by AI).
Note : Performance measurements in this article are valid for the Compilatio AI text detection system version 4.5.3, in use since September 4, 2025.
👉 Discover now Compilatio AI Checker.
2. The reliability of the AI text detection engine
The role of the detection engine: labelling short text extracts as "AI" or "human
The Compilatio detection engine uses a language model (an artificial intelligence specialized in language processing) specifically trained to determine whether a text is similar to an AI or human production.
This "detection engine" receives texts from unknown sources and determines, according to the writing style which are similar to texts written by humans and those written by artificial intelligence.
Detection engine reliability measures
To get a comprehensive look at reliability, we need to measure several indicators: precision, recall and accuracy. For a better understanding of how these indicators are calculated, consult the following articles: "Precision and recall" and "Accuracy and precision".
The Compilatio detection engine achieves a reliability rate of 94 to 99% on academic content, depending on document length, language, and the nature of the analyzed content.
The Compilatio detection engine displays a false positive rate below 1%.
This means that out of 100 passages identified as written by a human, fewer than 1 are incorrectly identified as generated by AI.
These measurements were conducted on approximately 7,400 texts in 24 languages. The sample consisted of 3,700 texts written by humans and 3,700 texts written by Artificial Intelligence.
The questions posed to the AI to generate the texts were simple questions without specific instructions on writing style.
3. The reliability of Compilatio analysis
Compilatio's task when analyzing a document is not simply to judge whether texts are 100% AI or 100% human (as the AI detection engine does).
The role of analysis is to identify and quantify passages likely to be written by an AI or a human in a text containing a mixture from both sources (heterogeneous document).
|
Compilatio's final performance in detecting texts written by AIs is 9/10. |
This means that in a document containing 10 passages to be labeled (human or AI), 9 are correctly labeled.
The illustration below is representative of the level of efficiency measured, for the system currently offered by Compilatio:
4. Precautions to be taken regarding efficiency measures
The statistics provided describe the overall performance service on a large number of documents representing student work.
In practice, the sources (AI or human) of some passages/documents may be perfectly identified, and others less so. Remember that AI detection relies on the recognition of stylistic characteristics typical of texts written by an AI; it may happen that a human has a style similar to that of an artificial intelligence.
No AI detector can be 100% reliable.
The reliability rates communicated by AI detectors are not comparable! It is difficult to objectively compare the reliability of AI detection tools if test environments differ.
Measurements can change depending on a number of factors: corpus source and language, number of documents tested, generative AI model used, origin of human documents, company commitment not to manipulate results, etc.
It is important to remember that Compilatio tools provide indications on suspicious passages. It is always up to the examiner to interpret this information to validate or impute potential fraud. If in doubt, carry out a closer examination of the student's knowledge of the suspect passages.
To find out whether our solutions are adapted to your needs, please contact our advisors.
5. Is the Compilatio AI Detector multilingual?
Yes, the Compilatio AI Detector is multilingual. It can identify AI content in several languages: French, English, Spanish, Italian, Portuguese, Russian, Arabic, Hindi and other languages from all over the world!
Reliability measurements have been carried out on texts in 24 languages: Arabic, Croatian, Czech, Danish, Dutch, English, Finnish, French, German, Modern Greek, Hindi, Hungarian, Italian, Norwegian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swedish and Ukrainian.
Will the Compilatio AI detector keep pace with ongoing AI advances?
Read the answer here: https://support.compilatio.net/hc/en-us/articles/17435773405329
|
* Reminder: The Compilatio AI detector is available with the Compilatio Magister+ subscription. To find out more, visit our website: https://www.compilatio.net/en/magister-plus |
📌 Questions about Magister, Magister+, plagiarism or AI?
Get answers live during our "Q&A Webinar".
👉 Register for the next session
This article has been automatically translated. If you notice a translation error, please contact us.