Key Takeaways

• AI text detectors are binary classifiers built on a handful of statistical features (perplexity, burstiness, token-level likelihood), and like any classifier they live and die by where you set the decision threshold.
• The hard limit on their accuracy is not engineering. It is the overlap between the distribution of human writing and the distribution of model output, and that overlap grows every model generation.
• A 2026 paper reframes the problem as a composite hypothesis test and shows a false-positive floor that no amount of better modeling can push below.
• Distribution shift, not adversarial cleverness, is why detectors degrade: humanizers and newer base models both move text toward the region where the two classes are indistinguishable.
• For builders, the practical takeaway is to treat detection scores as calibrated probabilities with real error bars, not labels, and to design pipelines around that uncertainty.

If you have ever trained a binary classifier on features that partially separate two classes, you already understand AI text detection better than most of the people writing policy about it. Strip away the branding and the “99% accurate” marketing, and an AI detector is a logistic-regression-shaped object sitting on top of a few linguistic features, drawing a line through a feature space where human text and machine text overlap. Everything interesting about these tools, and everything broken about them, follows from that one fact.

This piece is for people who build with models rather than react to headlines about them. We will walk through what detectors actually compute, why the interesting failure mode is a threshold-and-overlap problem rather than a bug, what a recent mathematical result says about the hard floor on false positives, and what all of this means if you are shipping a product that generates or ingests text.

What a detector actually computes

Start with the generative process, because the detector is just its mirror image. An autoregressive language model factorizes the probability of a sequence into a product of next-token conditionals. At each step it produces a distribution over the vocabulary and samples (or greedily selects) from it. The text it emits is, almost by construction, text that the model itself assigns high probability to. That is the whole tell.

The first-order feature every detector leans on is perplexity, the exponential of the average negative log-likelihood the model assigns to a passage. Low perplexity means the passage sits in a high-probability region of the model’s own distribution: unsurprising continuations, safe word choices, the statistically obvious path. Human writing tends to score higher because people make locally improbable choices constantly. We reach for the odd word, the unexpected clause, the domain jargon a model would have smoothed away.

The second feature is burstiness, which captures the variance of that surprise across a document rather than its mean. Human text is spiky. A long serpentine sentence lands on a two-word fragment. A dense paragraph is followed by a throwaway aside. Decoding tends to regress toward a stable rhythm, so machine text is flatter, more metronomic, lower-variance in sentence length and per-token likelihood. Burstiness is essentially a second moment on top of perplexity’s first moment.

Modern detectors do not stop at two scalars. They stack these with vocabulary diversity, token-rank distributions, punctuation and syntactic features, sometimes the log-probability curvature that methods in the DetectGPT family exploit, and feed the lot into a trained classifier: a neural net or a gradient-boosted model that outputs a probability. But the epistemics are the same as a two-feature logistic regression. You have features that partially separate two populations, you fit a boundary, and you convert distance from that boundary into a score. The sophistication is in the feature engineering, not in escaping the geometry.

The whole game is the decision threshold

Here is the part that anyone who has shipped a classifier feels in their bones and that most institutional users never think about. A detector does not output “AI” or “human.” It outputs a real number, and a threshold turns that number into a verdict. Slide the threshold and you trade the two error types against each other along an ROC curve. Push it to catch more machine text and you sweep up more genuine human writing as collateral. Pull it back to stop falsely accusing people and you wave through more actual model output.

There is no threshold that makes both errors small when the underlying class distributions overlap. That is not a tuning failure. It is the definition of overlap. If the human-perplexity distribution and the AI-perplexity distribution share meaningful mass over the same range, then any cut through that range pays for its true positives with false positives, and the exchange rate is set by how much the two distributions intersect.

This is why the “accuracy” numbers vendors quote are close to meaningless without the operating point and the evaluation set attached. A tool reporting 99% accuracy has almost always been graded on a friendly split, with untouched model output in one pile and polished human prose in the other, two clusters a ruler could separate. That is the easy regime. The text that actually shows up in production, lightly edited model drafts, human writing about saturated topics, prose from non-native speakers, hybrid human-and-model documents, lands in the contested middle where the classes blur. The reported accuracy was measured exactly where the problem is easiest, then advertised as if it held everywhere.

Base rates make it worse in a way builders will recognize from any rare-event detection system. Even a genuinely low per-document false-positive rate becomes a firehose of false accusations once you multiply it across a large population where most submissions are legitimately human. This is standard base-rate reasoning, the same math that makes a 1% false-positive medical screen dangerous when the condition is rare. A threshold that looks conservative on a balanced benchmark can generate thousands of wrong flags at deployment scale. Some builders in this space now ship their own checking layer for exactly this reason: tools like UndetectedGPT’s AI detector exist partly to let a team see the probability a downstream detector would assign before anything ships, treating the reading as a calibrated estimate rather than a verdict.

The overlap is not a bug you can patch

The tempting response, if you build models, is to assume this is solvable with a better detector. Richer features, a bigger classifier, more training data, adversarial fine-tuning against known evasion. And for narrow, in-distribution cases, better engineering genuinely helps.

But there is a structural result here that is worth internalizing, because it changes what “better” can even mean. A 2026 paper, “AI Detectors Fail Diverse Student Populations: A Mathematical Framing of Structural Detection Limits” (arXiv 2603.20254), reframes the whole task in a way that maps cleanly onto how ML people already think about detection. The standard theoretical framing treats it as a simple hypothesis test between two known distributions: human text versus AI text. The paper points out that this framing quietly assumes the assessor knows the individual writer’s distribution, which in the real deployment they emphatically do not. You are not testing against one known human distribution. You are testing against a diverse population of human writers, each with their own style, which makes the null hypothesis composite rather than simple.

Feed that composite null through the variational characterization of total variation distance and a bound falls out: any text-only, one-shot detector with non-trivial power must produce false accusations at a rate governed by the distributional overlap between human writing and model output. The paper states the result qualitatively rather than pinning it to a single headline percentage, and the honest read is that the floor is conceptual, not a magic number, but the direction is unambiguous. That floor is a property of the two distributions, not of your architecture. It is, in the paper’s own framing, logically independent of AI model quality and not something better detector engineering can remove. You cannot train your way under a bound that is set by the data-generating processes themselves.

For an ML audience this should feel familiar and a little uncomfortable. It is the Bayes error of the problem. When two classes overlap in feature space, some irreducible error is baked in, and no classifier, however expressive, beats it. The paper’s contribution is showing that AI text detection has a large and structurally guaranteed Bayes error precisely because human writing is heterogeneous, and that the error concentrates on whichever subgroups sit closest to the model’s output distribution. A subgroup mixture bound in the paper ties the abstract limit to the observable pattern everyone has already noticed empirically: the writers who get flagged most are the ones whose natural style, simpler vocabulary, more regular structure, already looks like low-perplexity text. The bias is not a separate defect layered on top of the tool. It is the same overlap, viewed one subpopulation at a time.

It is worth being precise about what this does and does not say. It does not say detection is worthless or that scores carry no information. A detector can be meaningfully better than chance and still be unfit as sole evidence, because “better than chance” and “reliable enough to accuse an individual” are different bars. The score is a genuine signal. It is just a probabilistic one with error bars that the deployment context routinely strips off.

Distribution shift is the real adversary

The second structural force is one every ML practitioner respects: distribution shift. A detector is trained on a snapshot of human text and a snapshot of model output. Both distributions move, and they move toward each other.

On the model side, every capability jump pushes generated text into a higher-perplexity, higher-burstiness regime. This is not the labs optimizing for evasion. It is a side effect of models simply getting better at writing. More varied phrasing, more natural rhythm, more contextually surprising choices are exactly the qualities that make output feel human, and exactly the qualities that erode the statistical gap a detector depends on. Better generation is, almost definitionally, harder-to-detect generation. The class you are trying to flag drifts, generation by generation, into the region occupied by the class you must protect.

On the transformation side, you have paraphrasing and humanization, which act as an explicit push toward the overlap zone. A rewriting system that raises perplexity variance and breaks up uniform sentence rhythm is, in feature-space terms, transporting a point from the low-perplexity machine cluster into the contested middle where human writing also lives. It is not hiding a watermark or injecting invisible characters. It is moving the sample across the decision boundary by changing the very features the boundary is drawn on. Once a document sits in the overlap region, no threshold can recover it without also sweeping in the genuine human text sharing that space, which is the whole point of the structural bound restated operationally.

The consequence for anyone maintaining a detector is a permanent retraining treadmill. You collect fresh model output, relabel, refit, and recover some in-distribution power, right up until the next base model ships or the next rewriting technique lands and the boundary you fitted is stale again. It is the classic non-stationary classification problem, with the added cruelty that both the positive class and the transformations acting on it are improving faster than your labeling pipeline. You are not converging on a solution. You are chasing a distribution that has a head start.

What this means if you are building

None of this argues that you should ignore detection. It argues you should model it correctly, the way you would model any noisy classifier feeding a decision.

First, treat detector output as a calibrated probability, not a label, and demand the operating point. A raw score with no threshold and no evaluation distribution is uninterpretable. If a vendor cannot tell you the false-positive rate at the operating point on text that resembles yours, you do not have a measurement, you have a slogan. When you evaluate, build your own held-out set from your actual domain rather than trusting the friendly benchmark the number was quoted on.

Second, respect the base rate in your own pipeline. If you are gating a large volume of mostly-legitimate content, even a small per-item false-positive rate produces an error count that dominates your review workload. Size that before you automate any consequence off a score. The place a detector earns its keep is as a soft signal in a larger system, one feature among many, never the sole trigger for an irreversible action.

Third, if your product generates or transforms text and has to clear a downstream detector, understand that you are working the same feature space from the other side. This is where a programmable rewriting layer belongs in a builder’s stack. A humanizer API lets you fold the transformation into a generation pipeline the way you would any other post-processing step, so that output aimed at a detector-gated channel is shaped for that channel before it ships, rather than getting bounced and reworked by hand after the fact. The engineering question is not “how do I cheat a detector,” it is “how do I place my output in the region of feature space my channel actually accepts,” which is a routing and post-processing problem like any other.

Because make no mistake: detection is a gate that matters, even though it is imperfect. Publishers, platforms, marketplaces, and institutions run these classifiers and act on them, and being on the wrong side of a threshold has real consequences whether or not the threshold is philosophically sound. The right posture is not “detectors are junk so ignore them.” It is “detectors are noisy classifiers with a structural error floor, deployed as if they were oracles, so I will engineer around both the noise and the deployment reality.” Passing the gate reliably is an operational requirement. Believing the gate is infallible is the mistake.

The uncomfortable symmetry

Here is the clean way to hold all of it in your head. The features that let a detector guess “machine” are the same features that let a rewriting system move text out of the machine cluster, because they are literally the same coordinates in the same space. Perplexity, burstiness, the shape of the token-likelihood curve: measuring them and manipulating them are two views of one geometry. That symmetry is why the arms race never ends and why the structural bound has teeth. Both sides are operating on the overlap between two distributions, and that overlap widens with every model release.

For builders the lesson is not cynicism about detection or triumphalism about evasion. It is precision. Know what the classifier computes, know that its accuracy is a threshold choice over overlapping distributions, know that the overlap sets a floor no engineering removes, and know that the floor is drifting in the direction of “indistinguishable” a little more each generation. Design your systems for a world where text authorship is a probabilistic estimate with real error bars, because mathematically, that is the only world on offer.