Algorithm · Offline black-box optimization

SPADE

A forward diffusion surrogate for offline black-box optimization that stays calibrated on the data manifold and conservative where the dataset does not support a design.

Inverse

p(x | y)

Generate a design from a target score. The map is one-to-many and often ill-posed, so the model can collapse or miss feasible modes.

SPADE takes Forward

Forward

p(y | x)

Score a candidate from a design. The map is many-to-one, but a point regressor still cannot say when the dataset no longer supports the query.

Module 01

Calibrated diffusion

A conditional diffusion model of p(y | x) is trained with moment matching and pairwise rank consistency, so the surrogate stays useful for search, not only for reconstructing the training scores.

Module 02

Support proximity

A kNN density proxy shrinks predicted means and inflates uncertainty away from the data manifold. The regularizer is first-order equivalent to a Bayesian design prior.

How the search stays conservative

Train the surrogateDiffusion estimation plus calibration and support-proximity losses on the static dataset only.
Score with LCBThe lower confidence bound uses both the predicted mean and the inflated OOD uncertainty.
Evolve candidatesAn evolutionary update proposes new designs without querying an online oracle.

Abstract

Offline black-box optimization aims to discover novel designs with high property scores using only a static dataset. The central failure mode is out-of-distribution extrapolation: a search procedure chases surrogate error in regions the data never supported.

SPADE models the forward likelihood with a conditional diffusion surrogate, then adds calibrated moment and ranking losses and a kNN support regularizer. It reaches state-of-the-art results on Design-Bench and an LLM data-mixture optimization benchmark. Code is public.