The AI Detection Business Is Selling Certainty It Can’t Deliver

There is a growing market for software that claims to tell us whether something was written by artificial intelligence. Paste in a piece of writing, wait a few seconds, and receive a score. Maybe it’s 87% AI-generated. Maybe it’s 12%. The number looks precise, the interface looks authoritative, and the implication is that we’ve just established something meaningful about who wrote the work. But what, exactly, have we established?

AI detection has become an attractive business because it promises to address a problem people are increasingly uncomfortable with. Schools want to know whether students are doing their own work. Publishers want to know whether submitted material was generated by a machine. Businesses want to know whether content produced by contractors is original. Readers want to know whether the words in front of them came from a person or a language model. Those are understandable concerns. The trouble begins when a statistical assessment is interpreted as something much more conclusive than it actually is. Not every detection company claims to provide definitive proof, but the presentation of a precise percentage can easily create that impression. We’re developing an appetite for certainty in an area where certainty is particularly difficult to establish. And an industry has emerged to meet that demand.

A Percentage Is Not Proof

AI detectors examine characteristics of a text and use them to estimate whether the writing resembles material produced by generative AI. Depending on the system, that may involve patterns in word choice, sentence structure, predictability, and other statistical features. The result is an assessment based on the finished text. It isn’t a record of the writing process. The software wasn’t present when the words were written. It doesn’t know the author’s intentions, have access to the original drafts, or understand the sequence of decisions that produced the finished document. It sees the text and evaluates patterns.

That distinction matters because the patterns detectors look for aren’t exclusive to machines. A human writer can produce predictable, formulaic prose. An AI-generated draft can be extensively rewritten by a person. A document can be developed collaboratively, edited by multiple people, translated, reorganized, or refined using AI tools without any simple way to assign a single authorship label to the finished result. The score doesn’t explain any of that. And a displayed percentage doesn’t necessarily mean what people assume it means. Depending on the product, it may represent an estimate of the proportion of text classified as AI-generated, a confidence measure, or another proprietary scoring method. Those aren’t automatically the same thing as the probability that a particular person used AI.

A score of 95% sounds like a verdict. A score of 4% sounds like an acquittal. Neither interpretation is necessarily justified. A measurement can look precise without being conclusive. The fact that software produces a number with a decimal point doesn’t mean the underlying question has been settled.

The Research Is More Complicated Than the Marketing

To be fair, AI detection isn’t entirely without merit. Some systems can distinguish between human-written and AI-generated material under particular testing conditions. Research has demonstrated that detectors can identify patterns associated with AI use, sometimes with useful levels of accuracy. But that’s not the same as reliably determining the authorship of any piece of writing encountered in the real world. Performance varies depending on the text, the language, the subject, the length, the AI model involved, and how much human editing has taken place. Hybrid writing creates a particularly difficult problem because the finished material may reflect several different stages of human and machine contribution.

A 2026 study published in the proceedings of the Conference on Empirical Methods in Natural Language Processing examined 135,389 document pairs from a professional English editing service. The researchers compared manuscripts written by non-native English speakers with professionally edited versions of those same manuscripts. Because the original authorship and underlying content remained the same, the researchers could examine how changes in writing style affected 13 AI detectors. Across those tested systems, false-positive rates on the human-written material ranged from 0% to 100%. That is a range across the detectors in this particular study, not a claim that every detector has a false-positive rate somewhere in that range in ordinary use.

The more revealing finding was that the same editing changes affected different detectors in opposite ways. Some became more likely to flag the writing as AI-generated, while others became less likely to do so. In other words, professional editing could change a detector’s assessment even though the underlying authorship and content had not changed. The researchers identified professional editing style as a significant confounding factor in detector outputs. That’s especially relevant because editing is precisely what good writing often requires. A person who improves clarity, tightens sentences, corrects grammar, and makes prose more polished should not automatically make their work look more machine-generated. Yet the study demonstrates that stylistic changes can influence detector results in ways that aren’t consistent across systems. The study is titled “Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing.”

A separate 2025 study published in Advances in Simulation compared three detection tools across five levels of AI involvement, ranging from entirely human-written material to fully AI-generated text. The researchers used introductions from 30 previously published healthcare journal articles and tested different levels of AI assistance under controlled conditions. The tools were able to distinguish between the test conditions, but their scores varied considerably, with intraclass correlation coefficients ranging from 0.57 to 0.95. That study shouldn’t be treated as a universal evaluation of every commercial detector. It does, however, illustrate an important distinction: a tool may be able to recognize broad differences between test categories without producing a consistent or definitive assessment of an individual document. The researchers concluded that detection tools may assist in identifying potentially inappropriate AI use, but should not be relied upon alone. That’s a much more reasonable conclusion than treating a detector score as proof. The study is titled “Ability of AI Detection Tools and Humans to Accurately Identify Different Forms of AI-Generated Written Content.”

What Happens When the Detector Gets It Wrong?

Imagine a professional writer who has spent years developing a particular style. Their writing is clear, consistent, grammatically polished, and organized. They avoid unnecessary words, use familiar sentence structures, and explain complicated subjects in straightforward language. Now imagine that a detector flags the writing as likely AI-generated. What has happened? The writer hasn’t suddenly become less human. Their experience hasn’t disappeared. Their work hasn’t become less original. The software has identified patterns that it associates with machine-generated text, but those patterns are not exclusive to machines.

The consequences can be considerably more serious than an inconvenient score. A student can be accused of academic misconduct. A freelancer can lose a client. A publisher can reject a submission. A professional’s credibility can be questioned. And the person being accused may have very little ability to prove a negative. This is particularly troubling when the detector’s result is treated as the starting point for a presumption of guilt rather than a reason to investigate further. A detection tool can be useful as one signal among several. But when the output becomes the evidence, the process has been turned upside down. The software’s conclusion is being used to justify the investigation that should have been necessary to evaluate that conclusion in the first place. The burden of proof should not quietly shift to the person whose writing has been flagged.

The Hybrid Problem Nobody Can Easily Solve

One of the biggest weaknesses in the way we talk about AI detection is the assumption that writing belongs in one of two categories: human or AI. That distinction is becoming increasingly difficult to apply. A person might develop an argument independently, use AI to organize research, write the first draft, rewrite half of it, use another tool to improve clarity, and then edit the final version manually. Another person might write every sentence themselves but use AI to identify repetitive phrasing or correct grammar. Are those AI-written documents? Human-written documents? Something in between?

The answer depends on what we’re trying to measure and why. If the concern is whether a student demonstrated independent understanding, the relevant question isn’t simply whether AI touched the document. It’s whether the student did the intellectual work required by the assignment. If the concern is whether a publisher is receiving original work, the relevant questions involve originality, attribution, accuracy, and the publisher’s own standards. If a business hires a writer to produce a useful article, the important questions may be whether the work is accurate, original, relevant, and professionally executed. These are different problems. A single percentage cannot answer all of them.

There are legitimate reasons to establish rules about AI assistance, disclosure, and authorship. But those rules need to define what matters in the particular context. Simply labeling a finished document as AI-generated or human-generated can obscure the more important questions. A policy about acceptable AI assistance is one thing. A software score presented as proof that someone violated that policy is quite another.

The Business of Making People Feel Certain

This is where the commercial side of AI detection deserves scrutiny. There is a market for uncertainty, and there is a market for selling people relief from uncertainty. The more anxious organizations become about AI-generated material, the more attractive a tool becomes if it promises to identify it. The product is easy to demonstrate: upload a document, receive a score, and present the result as useful intelligence. It’s a compelling sales proposition because it turns an ambiguous problem into something that looks measurable. But the appearance of measurement can create its own problem.

A customer may not understand the difference between a detector’s confidence score, its measured accuracy on a particular test set, its false-positive rate, and the probability that a specific document was written by AI. Those are not interchangeable measurements. Even a tool that performs well in a controlled evaluation does not automatically provide a reliable verdict in every situation. A vendor can publish impressive accuracy figures without those figures answering the customer’s actual question. What happens when the writing has been edited? What happens when the document is short? What happens when the author has a highly structured writing style? What happens when AI was used for only a small part of the work? What happens when several people contributed? Those are the questions that matter when a result could affect someone’s reputation, employment, education, or income.

The responsible vendors should be clear about those limitations. The responsible customers should demand that clarity. And anyone using these tools should understand what the software can establish and what it cannot. There is nothing wrong with selling a tool that provides a useful signal. The problem is allowing the customer to mistake that signal for a verdict.

We’re Confusing Detection With Evaluation

There is a larger issue here that extends beyond the software itself. We’ve become increasingly interested in identifying how something was produced and less interested in evaluating what was actually produced. That was the argument behind our recent article, AI Slop Is the New “Do You Do UI/UX?”. The question of whether AI was involved has become an easy substitute for examining the quality of the work. AI detection takes that same shortcut and attempts to automate it.

Instead of reading an article and evaluating its accuracy, originality, reasoning, and usefulness, we scan it and look for a number. Instead of examining a student’s understanding, we look for a classification. Instead of asking whether a writer has produced something valuable, we ask whether a machine might have helped produce it. Those questions aren’t entirely irrelevant. But they aren’t substitutes for judgment. A poorly researched article remains poorly researched whether it was written by a person or generated by AI. A false statement doesn’t become true because a human typed it. An original argument doesn’t become worthless because AI helped refine the wording.

If the objective is quality, we should evaluate quality. If the objective is academic integrity, we should establish what constitutes acceptable work and examine the evidence. If the objective is authentic authorship, we need a process that considers more than the statistical characteristics of a finished document. The tool should support the evaluation, not replace it.

What Should We Be Asking Instead?

There are legitimate reasons to care about whether AI was involved in creating something. But the question needs to be more specific than a percentage on a screen. What was the person expected to produce? What role did AI play? Was the use of AI permitted? Was the work reviewed and verified? Can the author explain the argument, demonstrate the research, provide drafts, or substantiate the claims? Those questions lead somewhere useful.

A student who cannot explain the work submitted under their name may have a problem regardless of what a detector reports. A writer who can demonstrate their research, explain their reasoning, and stand behind the finished work has something more meaningful to offer than a software score. For publishers and businesses, the focus should be on standards that can actually be evaluated: originality, factual accuracy, expertise, transparency where required, and accountability for what gets published. None of that requires pretending AI detection has no value. It simply means recognizing that detection and proof are different things.

The technology may continue to improve. Better models, better datasets, and more sophisticated detection methods may produce more useful results. That’s worth pursuing. But improvements in detection do not eliminate the need to understand what the results mean or how much weight they deserve. The more consequential the decision, the less acceptable it should be to rely on a single automated assessment.

Certainty Is a Product. Judgment Is Still the Responsibility.

The appeal of AI detection is obvious. It promises to make a difficult question easier. It gives organizations a number they can put in a report, a result they can point to, and a process that appears objective. But an objective-looking result isn’t automatically an objective conclusion. The danger isn’t simply that a detector might get something wrong. Every measurement system has limitations. The danger is that people will stop questioning the result because it came from software.

We should be especially careful when a tool is being used to make decisions about people. A false accusation isn’t corrected by explaining that the detector was only providing an estimate. A lost opportunity isn’t restored because a percentage turned out to be wrong. AI detection can provide information. It cannot assume responsibility for the decisions people make with that information.

And perhaps that’s the larger lesson here. We’ve spent years developing tools that make it easier to produce, classify, measure, and evaluate information. But none of those tools can eliminate the need for someone to exercise judgment. If a piece of writing is inaccurate, show what’s inaccurate. If it lacks originality, explain what’s missing. If someone has failed to demonstrate their understanding, establish that through evidence. Don’t turn a probability into a verdict simply because the verdict is easier to sell.

The software can give you a score. Deciding what that score actually proves is still your job.