Every vendor deck has the number: "X% accuracy on retrosynthesis." It is almost always the wrong number, measured the wrong way, and it describes the half of the problem that was never the hard part. To see why — and to know what an AI route is actually worth — you have to take the system apart layer by layer and interrogate each metric until it confesses what it does and doesn't measure.
Two layers, and conflating them is the first mistake
Computer-aided synthesis planning is two models with very different jobs. The single-step model answers a local question: given this molecule, what reactants could have produced it in one step? The multi-step search answers a global one: chain those single-step guesses into a tree that bottoms out in purchasable building blocks. They fail differently, they're evaluated differently, and — as we'll see — a model that wins at one can lose at the other. Treating "retrosynthesis accuracy" as one number erases that distinction, which is exactly where the marketing hides.
Inside the single-step model
Three architectural families compete, and the differences are not cosmetic.
Template-based models encode a reaction as a SMARTS transformation rule extracted from data — typically with RDChiral, Coley's stereochemistry-aware extraction wrapper that turns an atom-mapped reaction into a reusable retrosynthetic template. [1] The model's job becomes classification: which of tens of thousands of templates applies to this product? The strongest of these, LocalRetro, restricts templates to local atom and bond edits and adds a global-attention correction for the occasional non-local reaction, which is enough to top the template-based leaderboard. [2] The fatal structural limit is coverage: a template-based model cannot emit a disconnection whose template is not in its library. Reaction-template frequency is brutally long-tailed — a large fraction of templates appear once in the entire training set — so the model is, by construction, blind to the novel and the rare, which is precisely the chemistry discovery cares about.
Template-free models drop the rule library and treat retrosynthesis as translation: a sequence model rewrites the product SMILES string into reactant SMILES, character by character. The headline result here is instructive because it isn't about the neural architecture at all. Root-aligned SMILES (R-SMILES) — which forces a near one-to-one atom alignment between the product and reactant strings so the model has to edit rather than regenerate — reaches roughly 56% top-1 on USPTO-50k, the best template-free result of its time, mostly by changing the string representation and the augmentation, not the transformer. [3] The lesson the field keeps relearning: in molecular ML, representation and data augmentation move the needle more than model size.
Semi-template methods split the difference: first predict the reaction center, cut the product into synthons, then complete the synthons into reactants. They buy back some interpretability without the hard template-coverage ceiling. The point of cataloguing all three is that "accuracy" means something different in each, and the gap between them on a clean benchmark is smaller than the gap created by how the benchmark is built — which is the next problem.
The benchmark leaked the answer — for years
USPTO-50k is the field's MNIST: 50,016 atom-mapped reactions in ten reaction classes, curated from the patent literature. It also contained a subtle, now-notorious information leak. The dataset's product molecules retained their atom-map numbers, and RDKit's canonical SMILES ordering is influenced by whether atom maps are present. The upshot: the canonical token order of the product quietly encoded where the reaction center was, and models could learn to read it off the input rather than reason about chemistry. A meaningful slice of reported top-1 gains evaporated once the leak was closed by stripping atom maps and re-canonicalizing the product before the model ever saw it. [4]
This is not a footnote. It means a chunk of the published "progress" on single-step retrosynthesis was the field getting better at exploiting a preprocessing artifact. If you are evaluating a tool today, the first question is not "what's your top-1?" but "did you strip atom maps and re-canonicalize, and what was the number before and after?"
Single-step accuracy does not predict multi-step usefulness
Here is the finding that should reframe the whole conversation. When Microsoft Research's Syntheseus group re-ran the major algorithms under a single, carefully controlled evaluation, the ranking of state-of-the-art models changed — and, critically, single-step top-k accuracy turned out to be a poor predictor of how well a model actually plans multi-step routes. [4] The model that tops the single-step leaderboard is not necessarily the one that finds you a synthesis.
The reason is structural. Multi-step search (Retro*'s neural-guided A* over an AND-OR tree, or AiZynthFinder's Monte Carlo tree search) doesn't need the single right disconnection ranked first; it needs a diverse, recoverable set of plausible disconnections so that when one branch dead-ends against the building-block stock, another is reachable. [5] [6] A model with sharp top-1 but narrow, peaky predictions can starve the search; a model with lower top-1 but better-calibrated diversity can feed it. Top-1 accuracy measures the former and ignores the latter — so the metric the whole field optimizes is misaligned with the task the field cares about. That is a deep problem, and it is why route-level benchmarks like USPTO-190 (hard targets, measured by solve rate and route quality) matter more than the single-step leaderboard everyone quotes.
"Round-trip" feasibility is graded by a model that's never seen failure
Even the smarter metric has a worm in it. Exact-match top-k is unfair — it marks a chemically valid alternative disconnection "wrong" for not matching the one recorded in the patent — so Schwaller introduced round-trip accuracy: take the predicted precursors, run them through a forward reaction-prediction model, and check whether they actually regenerate the target. [7] It's a real improvement, and transformers reach round-trip accuracy north of 80% with ~96% coverage.
But look at what it actually certifies. The forward model is trained on the same corpus of successful, published reactions — with no negative data. It has never seen a reaction that was tried and failed. So it can confirm that a proposed precursor set is consistent with reactions that worked; it cannot reliably tell you that a precursor set won't work, because infeasibility is exactly the signal absent from its training data. Round-trip accuracy grades one optimistic model with another optimistic model trained on the same biased manifold. The community that introduced it says as much — it's a proxy that still needs development before it can stand in for wet-lab feasibility. [4] When a vendor cites round-trip numbers, they are citing internal consistency, not the probability the reaction runs in a flask.
A route is not a procedure — and the planner doesn't even try
Now the commercially decisive limit. The output of all of this is a sequence of disconnections — bond-level edits — not an executable synthesis. It contains no conditions: no solvent, catalyst, base, temperature, stoichiometry, or order of addition. Those determine whether a "correct" disconnection runs at all, and they live in a separate model.
How good is that separate model? The best-known condition recommender (Gao et al.), trained on roughly 10 million Reaxys reactions, finds a close match to the recorded catalyst, solvent, and reagent within its top-10 predictions only 69.6% of the time, with per-species top-10 accuracy in the 80–90% range. [8] That is a useful coarse recommender and a long way from a procedure you'd hand a chemist unread. And note the gap it implies: the retrosynthesis model that produced the route never reasoned about conditions in the first place, so the disconnection it proposed may have no set of conditions that achieves the needed chemoselectivity, regioselectivity, or stereoselectivity. The planner optimizes for reachability to purchasable stock, not for whether the amide coupling it suggested will fire selectively in the presence of a competing free amine, or whether the reduction it proposed will spare the ketone three carbons over. Selectivity, protecting-group strategy, and scale-up are not weakly modeled; they are unmodeled.
The real ceiling is the data, not the model
Step back and the pattern is unmistakable. Every limit above traces to the same root: the training corpus is the patent and Reaxys record of reactions that worked, reported without yields in bulk, without conditions in the standard splits, and without a single example of what failed. A generative model trained on that manifold learns to propose transformations that resemble successful ones. That is genuinely valuable — it's a superb recall engine for plausible disconnections, far broader than any chemist's memory. But recall was never the hard part of synthesis. Feasibility was: will this specific step, on this specific substrate, run selectively, in acceptable yield, at the scale you need? That information is not in the data, so no amount of additional parameters puts it in the model. The field is asymptoting against a data ceiling.
Which is why the genuinely promising direction isn't a bigger transformer. It's data with the missing dimensions: high-throughput experimentation that records conditions and yields under controlled variation, electronic lab notebooks that capture the failures the patent literature buries, and autonomous "self-driving" labs that close the loop by generating balanced positive-and-negative reaction data on purpose. ASKCOS and its peers are already bolting condition recommendation, in-scope filtering, and outcome prediction around the retrosynthesis core for exactly this reason — the route is the cheap part; making it real is the expensive part the data doesn't yet cover. [1]
So what is an AI route actually worth?
It is an exceptional ideation and triage instrument: it enumerates a far wider funnel of plausible disconnections than a human, in seconds, and round-trip-style checks keep most of them internally consistent. It is not, in any honest sense, a synthesis planner of the kind a process chemist means — because the parts that decide whether a synthesis works (selectivity, conditions, yield, scale, and the knowledge of what fails) are the parts the data omits and the metrics don't measure. The chemist's veto isn't a transitional inconvenience that the next model removes. It persists precisely because the feasibility knowledge was never in the training set, and the leaderboard was never measuring it.
The next time you're shown a retrosynthesis accuracy number, ask the four questions the number is built to dodge: Single-step or multi-step? Was the atom-map leak closed? Is that exact-match or round-trip — and if round-trip, what graded it? And where, in any of this, are the conditions?