What we measured
Most products tell you what they built. This page tells you what we tested, what the numbers said, and which of our own features lost badly enough that we switched them off. Every figure below comes from a comparison where the judge did not know which version was which.
A note on how to read this. Every test works the same way: two versions of the same request go out, the answers come back, and a judge who cannot tell them apart picks the better one. Before any test runs, we write down in advance what result would count as a win. Then we honour it, including when it goes against us. That last part is the reason this page has anything interesting on it.
We spent months building a library of experts, then switched it off
The original idea behind Intent Amplifier was simple and, we thought, obviously right. The best people in the world have already published how they think. Read them, organise what they said, and quietly slip the relevant piece into your prompt. You would get the benefit of an expert's instincts without ever knowing which expert or which idea was involved.
We built it. 59 experts, 1,224 individual pieces of their thinking, sorted and scored so the right one could be found in a fraction of a second. Then, before relying on it, we measured whether it actually made anyone's answer better.
Then we ran the test that explained it. Instead of an expert's notes, we pasted in a complete stranger's personal background and asked the same question. It scored 26.2%: almost exactly what the expert notes scored.
That is the finding that changed the product. What was damaging the answer was not that the material was low quality. It was that the material was not true of the person asking. An expert's method describes how someone else works. Paste it into your prompt and the AI quietly writes as though it describes how you work, and the answer drifts away from your actual situation.
So the expert library is off. The part that reads your request and works out which specialism it belongs to still runs, because that half was never the problem, but nothing from the library is pasted into your prompt any more. The switch still exists. If the evidence ever turns, we will turn it back on and say so here.
One good prompt beat a five step workflow, sixteen times out of seventeen
There is a whole category of tools that break your job into a chain: step one researches, step two drafts, step three refines, and so on. It sounds better than one prompt. It looks more thorough. We wanted it too, so we built it, twice, and put it up against a single amplified prompt with a blind judge in between.
The reason is worth understanding, because it is the reason we design the way we do.
Every real job has gaps in it. You ask for a proposal but never said the price. You ask for homepage copy but never named the company. A chain has to hand something finished to the next step, so it fills the gap by inventing something. Step two then builds on the invention, step three builds on that, and by step five the fiction is holding up the whole document.
These came out of the losing chain. Each one was checked line by line against the original request, and none of them appeared anywhere in what the person actually asked for:
| What the chain invented | Where it put it |
|---|---|
| A company called Harborline, with a street address and a phone number | Homepage copy marked ready to publish |
| A client called Northbeam Supply, with revenue and conversion figures | Inside a proposal meant to be signed |
| "SOC 2 Type II certified" | A pricing page for a three person startup with no revenue |
| Quotes from two named executives | An article formatted ready to publish |
| A customer testimonial, complete with a hospitalised relative and exact figures | A draft ready to be sent |
The single prompt did the opposite thing, and that turned out to be exactly why it won.
A square bracket is not a failure. It is the only honest move.
When an answer reaches something it genuinely cannot know, it has two options. Leave a visible gap, like [your monthly price]. Or fill it with something plausible and say nothing. We ended up measuring both, because the chained version above had been told never to leave a bracket.
This is the shape that wins, and it is an uncomfortable one to defend, because a document with brackets in it looks less finished. It isn't. It is a finished document that is honest about its own edges, and you can fill five brackets in two minutes. You cannot fix an invented client name that you never noticed.
There is now a standing rule in our code because of this. We will never add a setting that tells the AI to fill in every gap. We measured what that produces, and it produces a document you cannot trust.
This is a comparison against multi step workflow tools, not against writing your own prompt. Adjusted for length, a request you type yourself is actually slightly more bracket dense than an amplified one, because amplified answers run around six times longer. And our own output is not immune: on one test case it asserted a compliance certification it should have bracketed. A strong tendency, not a guarantee. We would rather say that here than have you find it on your own.
On about half of real requests, you did not need us
For a long time every number we had came from test requests we had written ourselves. That was a mistake. We found it by going and getting real first messages that real people had typed into AI chats and running the product on those instead, word for word.
The score fell from about 80% on our own test set to under 50% on theirs. Not because the product got worse, but because the requests we had been writing for ourselves were softer than the ones people actually send.
That split is not random, and it is the most useful thing we know about our own product. Sort those 24 requests by how good they already were before we touched them:
The mechanism is clean once you see it. Amplifying turns an open question into a detailed specification. If your question was vague, that is precisely the help you needed. If your question was already precise, the specification is narrower than what you asked, and the AI answers the narrower thing.
One example. Someone asked what they should do about a technical problem. Amplified, that became a menu of three named approaches, and the answer stayed inside the menu, so a fourth option nobody had listed never came up. The original open question would have found it.
On 12 of those 24 real requests, the message the person typed already scored full marks on its own. There was nothing for us to add.
So it tells you when you do not need it
The honest response to the section above is not to hide it. It is to say so at the moment it matters. Before amplifying, the product quietly asks itself whether your request already looks complete. If it does, you get this instead:
It advises. It never blocks. That is a deliberate call, based on a number: the check is right about 69% of the time, which is useful but nowhere near good enough to take a decision away from you. If it is wrong, it costs you one extra click. If it blocked you and was wrong, it would cost you the feature.
We have now tried five different ways to predict in advance which requests we will help and which we will hurt, including one built from what real users actually did. All five failed. We are telling you that too, because the notice is a genuinely useful hint and a genuinely unreliable one, and you deserve to know which.
What it actually looks like
Everything below is a screenshot of the extension, filled with what our live backend returned for one request. No mockups, no retouching, no invented copy. The request was this, typed the way people actually type:
"i need to write to my landlord about the damp in the bedroom wall, its been 3 months and 2 emails already and nothing. i dont want to sound aggressive but i want it fixed"
We deleted the cleverest part of the thing you see
Before it writes anything, the Amplifier can show you its read of what you are actually after, so that you can correct it first. That read used to do two things: name your goal, and then offer the lever, an insight about what really decides your outcome. The lever was the part that felt smart.
Five separate measurements pointed at the same seam, and every one of them put the problem in the second half. So we deleted the lever and kept the goal.
Here is the specific failure it was making. An industrial chemist moving into design ended his request with the words as I am from a non technical background. The read told him he already had user research skills, an understanding of user needs, and the ability to communicate complex ideas simply. He had claimed none of those. That read then goes into the prompt as if it were his own words, so the AI answers a far more confident person than the one who asked.
That is the same finding as the expert library, arriving from a different direction. What damages an answer is material that is not true of the person asking, however flattering, insightful or well sourced it is.
We had never looked at this until recently. Of 23 real reads shown to people who had agreed to share anonymously, 20 contained a lever. Every single rejection, five corrections and four abandonments, came from those 20. All three of the plain reads were accepted and used. Every correction someone typed was shorter than the read it replaced, and every one was a refusal of something the read had asserted about them.
Three out of three is not proof on its own and we will not present it as such. It is real people spending real attention, and it points the same way the measurements did.
The empty slot was then filled with something that cannot be wrong about you: the read now tells you what it is about to ask for on your behalf. You were approving a short read, and the product was then sending a request around five times longer that you never saw. Now the read names the shape of that request and which parts it is deliberately keeping wide open, and if you correct it, the correction binds.
Good ideas that measured as nothing
These were all built properly and tested fairly against a standard agreed in advance. Four measured as an active harm and the rest as nothing at all. None of them shipped. They are on this page because a product that only publishes its wins is telling you half a story.
| What we tried | What happened | |
|---|---|---|
| Tell the AI not to invent things One clear sentence forbidding made up specifics. |
Barely moved invention at all, and cost 10 points of "could you use this answer straight away". Answers got 15% shorter and no more truthful. | Not shipped |
| Point out what your request is missing Add an observation about the gap in your own question. |
It worked at the halfway point, coverage went from 19% to 56%, then changed the final answers by nothing at all. Naming an ambiguity makes the AI discuss it rather than resolve it. | No effect |
| Carry your own questions through Repeat the questions you asked, word for word, at the end. |
Nothing. Every measurement sat inside the noise. Adding to a prompt that already points the right way does not help. | No effect |
| Make the read longer and richer Four to six sentences instead of two to four. |
The read itself measurably improved on the exact metric we cared about, and the answers got slightly worse. A longer read commits to more, and a confident wrong commitment costs more than a right one earns. | Not shipped |
| Delete the irrelevant parts Have a judge cut anything not serving your goal. |
Moved exactly as much as deleting nothing at all. Twice, in two separate experiments. | No effect |
| Break the job into steps The chained workflow, built twice. |
Lost 11 to 7. We rebuilt it from scratch and it lost 16 to 1. Four rounds and roughly 500 test runs. | Not shipped |
| Put the expert library in the read instead If it cannot go in the prompt, put it where you can correct it. |
Tripled the claims your message did not support, and doubled how much of the library's own vocabulary appeared in a read about you. One person asking how databases work was told their real problem was database locks. They had not reported a problem. | Not shipped |
| Ask you better questions Improve what we ask before amplifying. |
Roughly half the questions land on nothing: either you cannot answer them, or you already did. Of the things that genuinely mattered and went unsaid, only about one in six could be asked at all. | Closed |
And one that shipped even though it failed its own test, because we decided the trade was worth it. If you tell the product words you never want to see, it obeys, on every single answer we measured. It also costs seven to nine points of completeness, because any restriction on word choice makes an answer shorter. That is a judgement call rather than a passed test, and it is the one place in the product deliberately paying a measured price. There is a switch to turn it off.
Why we do not trust a result until we test nothing against nothing
Here is the trap that ruins most before and after comparisons, including several of our own early ones.
AI answers are not identical twice. Ask exactly the same question twice and you get two different answers, and one of them will be better. So if you compare version A against version B and B wins, you have learned nothing yet, because A would also have beaten A.
Every claim on this page has been run against that floor. Where a result sits inside it, we have said so rather than quoting the number and moving on.
What these numbers do not say
Five things, so that you can discount this page accurately rather than generously.
If you want this without the charts, the docs explain how the product works, and the changelog records what changed and when.