Pick any creative tool's homepage and you'll find a version of the same sentence: know which ads will win before you spend. Sometimes it's a number out of 100. Sometimes it's a traffic light. Sometimes it's "AI-predicted performance", which is the same thing wearing a coat.
The claim is testable. Almost nobody tests it, including the people buying it.
This is an opinion piece, so here's the opinion up front: predicted-performance scores are mostly base rates in a trench coat, and the public record is full of evidence against them. Not because the vendors are lying — most believe it — but because the thing they're predicting has far more variance than the thing they're measuring.
Below is the evidence, the test you can run on any vendor in one meeting, and the part of this category that genuinely works.
What a "score" is actually made of
Three kinds of tool get sold under one word, and they fail differently.
| Type | How it works | Where it breaks |
|---|---|---|
| Attribute scorers | Tags your ad (hook, format, offer, emotion) and scores against what those tags historically did | Two ads with identical tags routinely produce opposite results |
| Embedding models | Converts the creative to a vector, scores against outcome-labelled training data | Learns brand size, objective and category; the creative signal is thin once those are controlled |
| Benchmark comparators | Compares your ad to a category average | Tells you about the category, not the ad |
All three share a structural problem. They're trained on ads that ran, which means they're trained on ads that someone decided to run — already filtered by human judgment, already skewed toward brands with budget. The model inherits that filter and calls it insight.
Exhibit A: the same video file, 52 days and 196 days
This is the piece of evidence I'd put in front of any vendor.
That is one video. Not two similar videos — one file, running as two ads in the same account, with the same body copy word for word. The only difference between them is the opening line. One begins "If you run ads on Facebook, you have to see this." The other begins "Supplement stores. If you run ads on Facebook, you have to see this."
Two words of audience naming. The version without them ran 52 days. The version with them ran 196 — close to four times as long.
Now ask what a scoring tool sees. Hook type: Curiosity, both. Ad type: screen recording and educational, both. Offer type, emotion, social proof, authenticity, intended audience: identical, both. An attribute scorer gives these two ads the same number. An embedding model gives them near-identical vectors, because it is scoring the same frames.
The difference that mattered — naming the reader in the first two words — is real, it is in the creative, and it is invisible to every dimension a score can read. That is the problem in miniature. The score isn't blind because creative doesn't matter. It's blind because the part of the creative that decides the outcome is usually too small and too specific to be a feature in anyone's model.
Exhibit B: same brand, same format, 220 days against 6
A pilot, talking about how magnesium changed his sleep and his mornings. Personal-experience hook, interview format, customer testimonial as the proof. Still running — live 220 days.
The same brand, the same account, the same format class: real person, talking to camera, testimonial. This one is founder-led and answers a very specific objection — fishy burps from fish oil, a genuine purchase blocker people type into search engines. It ran 6 days.
Here's the uncomfortable part. If you handed both to a creative strategist cold, a good number would pick the objection-handler. It does more jobs. It names a real hesitation. It has the founder's face on it, which every piece of received wisdom says builds trust faster. Every heuristic a scoring model encodes says it should be the stronger ad.
It wasn't. The pilot was, by a factor of thirty-six.
Exhibit C: the better-written hook lost
Three Qurist ads, one sleep product, one account.
Low sleep score? Fix it tonight. Still running — live 101 days. The hook works for a mundane reason: the viewer's watch told them that number this morning, so the ad is finishing a sentence their phone already started.
Thousands already sleep better with these. A volume claim rather than a moment. Still running — live 42 days. Solid, unremarkable, doing its job.
Credit score? Not our thing. Sleep score? Now that's our specialty.
Read that against the first one again, because this is the sharpest comparison in the post. Both ads are built on exactly the same insight — that your watch hands you a sleep score every morning and the brand should own that number. The first states it plainly. This one does it with a joke, a parallel structure and a genuinely good turn of phrase, and by any copywriting standard it is the better-written ad. It's the one I'd have bet on, and I suspect most strategists reading this would too.
It ran 4 days, against 101 for the plain version. A sibling with the same line ran 3.
That's not a lesson about which hook is better. It's a lesson about how little a pre-launch opinion is worth against a live auction — the same idea, dressed more cleverly, lasted a twenty-fifth as long — and the opinion doesn't get better for being generated by a model rather than a person.
Exhibit D: 236 days and 13, same brand, same product line
Plix Kids' bundle builder: any three products for ₹999. Ran 236 days — the longest run in this post.
The same brand's single-product launch for a children's body wash, three weeks earlier in the same account, pitched at the same mothers. Ran 13 days.
This one has a readable explanation — a bundle at a price point is an offer, a product introduction is an announcement — and that's exactly the point. The explanation is available afterwards. Before launch it is one of six plausible stories, and the model has no way to pick between them either. Hindsight is not prediction, however confidently it is formatted.
The variance isn't where the score is looking
Put the four exhibits together:
| Exhibit | What was held constant | Shortest run | Longest run | Spread |
|---|---|---|---|---|
| Foreplay | Same video file, same body copy | 52 days | 196 days | 3.8× |
| Ace Blend | Same brand, same testimonial format | 6 days | 220 days | 36× |
| Qurist | Same product line, same sleep-score insight | 4 days | 101 days | 25× |
| Plix Kids | Same brand, same product line, same audience | 13 days | 236 days | 18× |
Each row is one brand, one product line, one account. The spread inside a single account is larger than the spread a scoring model is trying to detect between accounts.
A model has to beat that. It has to separate ads whose measurable attributes are identical — Exhibit A is literally the same file — in accounts where same-creative outcomes already differ by an order of magnitude. That is a hard problem, and "82/100" is not a solution to it. It's a confident restatement of the category average.
How to audit a scoring vendor in one meeting
You don't need a data team. Five questions, asked in the demo:
- "What was the model trained on, and does it include losers?" If the training set is only ads that ran long enough to be noticed, the model learnt survivorship, not performance.
- "Show me the holdout." Ask for performance on ads the model never saw, in my category. Not a case study — a confusion matrix, or at least a hit rate against a coin flip.
- "Does the score change if I swap the brand logo?" If a known brand's logo moves the number, you're being sold a brand-equity lookup.
- "What's the score's accuracy on ads from one account?" Cross-account accuracy is mostly base rates. Within-account is the only number that matters, and it's the one nobody reports.
- "What would make this score wrong?" A vendor who can't answer this hasn't tested it. A score that can't be wrong isn't a measurement.
Ask these politely. The answers are usually honest and usually revealing — most product teams know exactly where the model is weak, and almost nobody asks.
What this category is actually good at
Rejecting prediction isn't rejecting the tools. The useful half is real, and it's the half that doesn't require a forecast:
- Duplication detection. Decode five variants and see that four share a hook type and an emotion. That's a measurable fact, available before launch, and it's the most common way a test budget gets wasted.
- Account memory. "This hook ran here in March and lasted nine days." Not a prediction — a lookup. Teams forget; accounts don't.
- Category saturation. Counting how many competitors are already running your argument is arithmetic, not AI, and it kills more bad ads than any score.
- Fatigue detection. A decaying creative has a measurable shape. CTR, CPM, ROAS and frequency, weighted, moving over time. This is the one place where modelling genuinely earns its keep — because you're describing a curve that already exists rather than one that doesn't yet.
- Coverage gaps. What the account has never tested is a fact about history, not a forecast.
Note what every item has in common: it's checkable. Someone can come back in six weeks and tell you it was wrong. A predicted score mostly can't be falsified, which is why it survives.
And one practical implication of Exhibit A, since it's the cheapest test in this post: before you brief a new concept, try running your existing winner again with the audience named in the first two words. It costs nothing, it changes no attribute a model can see, and in that account it was worth 144 extra days.
Our position, including the inconvenient part
Marxx tested predictive scoring. Creative embeddings against outcomes, the standard approach. Once brand and objective base rates were controlled for, the signal did not hold. We didn't ship it.
That cost us a demo slide. It's the right call, and the exhibits above are why: in a world where the same video file runs 52 days and 196 days in the same account, a number predicting which creative wins is theatre.
What Marxx does instead is report what is actually knowable — what ran, what is decaying, what competitors are doing, and what the account has never tested — decoding creative across hook type, offer type, ad type, emotion, social proof type, authenticity type and intended audience, over 180 days of the connected account's history rather than the last thirty.
Book a demo and we'll decode your last quarter live — including the duplicates you didn't know you'd shipped. The creative intelligence page explains what gets decoded and how fatigue is scored.
FAQs
Do AI ad prediction tools actually work?
Not for ranking creative by future performance. The public record is full of near-identical ads with wildly different outcomes inside the same account — one exhibit above is the same video file running 52 days and 196 days — which is the condition any predictive score would have to overcome, and generally doesn't.
Why do scoring tools seem accurate in demos?
Because demos use ads that already ran. Scored retrospectively, a model that encodes brand size and category will look sharp. Ask for a holdout on unseen ads in your own category and the picture changes.
Is there any creative signal at all?
Yes — but it sits at a resolution no model reads. Two words of audience naming were worth nearly four times the run length in Exhibit A, while every taggable attribute stayed identical. The signal is real and too specific to be a feature.
What should I use instead of a predicted score?
Duplication checks, account history lookups, category saturation counts and fatigue detection. All of them are facts rather than forecasts, and all of them can be shown to be wrong.
Is longevity a good proxy for performance?
It's the best public one, with limits. An ad that has run 200 days is almost certainly working. A 4-day ad might have been killed for reasons unrelated to the creative. Treat long runs as strong evidence and short runs as weak evidence.
Should I stop testing creative then?
The opposite. If you can't know in advance, the return comes from making tests cheap, fast and genuinely different from each other — and from writing the kill rule down before you launch.
What's the single cheapest creative test from this post?
Re-run a current winner with your audience named in the opening two words. Exhibit A is two ads off one video file, and that is the only difference between 52 days and 196.