Abstract:We present a general strategy for turning generative models into candidate solution samplers for batch Bayesian optimization (BO). The use of generative models for BO enables large batch scaling as generative sampling, optimization of non-continuous design spaces, and high-dimensional and combinatorial design. Inspired by the success of direct preference optimization (DPO), we show that one can train a generative model with noisy, simple utility values directly computed from observations to then form proposal distributions whose densities are proportional to the expected utility, i.e., BO's acquisition function values. Furthermore, this approach is generalizable beyond preference-based feedback to general types of reward signals and loss functions. This perspective avoids the construction of surrogate (regression or classification) models, common in previous methods that have used generative models for black-box optimization. Theoretically, we show that the generative models within the BO process follow a sequence of distributions which asymptotically approximate an optimal target under certain conditions. We also evaluate the performance through experiments on challenging optimization problems involving large batches in high dimensions.
| Comments: | Published at ICLR 2026. Compared with the proceedings version on OpenReview, this version includes a minor revision to Section 3 |
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.25240 [stat.ML] |
| (or arXiv:2510.25240v3 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2510.25240 arXiv-issued DOI via DataCite |
|
| Journal reference: | The Fourteenth International Conference on Learning Representations (ICLR 2026) |
Submission history
From: Rafael Oliveira [view email]
[v1]
Wed, 29 Oct 2025 07:42:25 UTC (832 KB)
[v2]
Fri, 12 Dec 2025 08:45:56 UTC (1,230 KB)
[v3]
Thu, 14 May 2026 13:08:46 UTC (1,237 KB)