When AI Becomes Judge (Part 2): The Double Standard: When AI Judges Students

This will likely be the last thing I write as a university student—at least for a while. Over the past two years, alongside my work in Generative AI, I have been researching the rise of what I call the "Digital Leviathan": the creeping normalization of algorithmic authority over human judgment and the harmful societal effects that result from it.

To some extent, this is a logical continuation of my BA thesis in law, which concerned the mandatory data retention of telecommunications companies. In one week, I defend my Master's thesis, Beyond Fragmentation: A Life-Value Alternative for AI Governance. I will share more of that soon, but first, let me finish what I started.

The lesson from Bifröst is that we cannot trust algorithms to be judges. The lesson from Turnitin is that we cannot trust institutions to protect us from these algorithms.

In Part 1, I discussed the incident at Bifröst University where administrators, according to news reports, used AI to judge the professional career of their staff. We saw the outrage. We saw the "2/10" rating. We saw the system reject the idea that a "black box" should determine professional worth.

But while we were busy being outraged on behalf of the professors, we forgot about the students.

The concern also reaches student assessment when institutions rely on automated reports as judgments. Turnitin uses different tools from Claude, and their use and settings must be checked institution by institution.

We call it Turnitin.

The Double Standard

If it is administrative malpractice to use Claude to judge a professor's contribution, why does a different rule apply to students? Both systems are:

• Opaque: No one knows exactly how the algorithm processes the data or what weight different factors have.
• Extractive: They rely on data harvested without the meaningful consent of authors.
• Hallucinations and Statistical Guesses: Unlike traditional plagiarism protection, which is based on facts, the AI scanner relies on probability assessments produced by the system, which administrators often treat as infallible truth.

One truth, one guess

It is important to distinguish between two different "features" within Turnitin.

On one hand, there is the Similarity Report, which compares text to a database and finds direct matches—that is a text-match report, not a finding of plagiarism.

On the other hand, there is AI Writing Detection, the new tool meant to catch AI. That tool does not compare text; it looks for statistical patterns in sentence structure. This is not proof, but a prediction. Research and testing repeatedly show that these scanners produce "false positives," where honest students are given a failing grade by an oracle that cannot explain its conclusion. When universities use this probability assessment as a basis for disciplinary sanctions, the legal security of students is sacrificed for the convenience of technology.

The difference? The professors at Bifröst had a union and an ethics committee. The students check a box that says: "I Agree."

Is there not a certain irony in Icelandic students granting a licence to use their work by agreeing to a contract in English?

The Rent Model of Education

Turnitin was acquired for $1.75 billion. The acquisition announcement describes both its software and its database of submissions; it does not quantify how much of the price each contributed.

When you submit an essay, you are not just getting a grade. You are performing unpaid data labor. Your submission may contribute to the comparison repository, depending on the assignment settings. This is the money-sequence in action: your intellectual property converted into their capital. Your tuition pays for the privilege of feeding the machine.

The "Trap"

Most students don't read the End-User License Agreement (EULA). I did.

"...You hereby grant to Turnitin... a non-exclusive, royalty-free, perpetual, worldwide, irrevocable license to use such papers..."

Translation: You pay tuition to create the work. You retain ownership; the agreement grants a continuing, unpaid licence for specified service purposes, with a route to request deletion.

The Solution: "Do not store"

There is a way out. Turnitin has a setting called "Do not store the submitted papers." If a professor enables this option, your paper is scanned but not saved in the database. It is checked for plagiarism but never added to the permanent repository.

Why isn't this the default setting?

Because of what I call Responsibility Fog. The collection setting is the default. Few likely question it. No one "owns" the decision. And by the time you realize what happened, your work may already be in the comparison repository.

How to Opt Out

Students can request deletion through their institution under the Turnitin EULA. GDPR Article 17 also provides a right to erasure of personal data when its conditions are met; completion of assessment alone does not establish that right, and exceptions apply.

Do not accept the Benevolent Cage. Use these templates or create your own; the institution must assess the request against its legal basis for retention and applicable exceptions.

For Submission: The Statement

Attach this or a similar text to your assignment:

I am submitting this assignment to Turnitin to comply with this school's requirements. However, I do not consent to the permanent storage of my intellectual property in a commercial database. I will be filing a formal request for deletion immediately following assessment.

For Deletion: The Demand

Send this to your instructor after grading:

Subject: Formal Request for Deletion from Turnitin Repository
Please forward this to the School's Turnitin Administrator:
I submitted the assignment [Title] (Paper ID: [ID]) for assessment. Now that grading is complete, I request deletion under the Turnitin EULA ("License by You") and, where its conditions apply, GDPR Article 17. If retention is still required, please explain its purpose, legal basis and duration.
Please confirm by email when this action is complete.

Conclusion

The only lesson left is this: we must protect ourselves.

Further Reading:

1. Turnitin User Agreement
2. Insight Partners Acquisition
3. Resisting EdTech (Hybrid Pedagogy)
4. Turnitin Acquired by Advance Publications
5. Turnitin "Do Not Store" Feature Guide