Abstract
Bayesian optimization (BO) is the method of choice for optimizing expensive black-box functions. At its core, an acquisition function decides where to query next by trading off exploration and exploitation. Information-theoretic acquisitions — such as Predictive Entropy Search and Joint Entropy Search — frame this decision as maximising the mutual information between the next query and the optimum.
In this talk I introduce Precision-Weighted Joint Entropy Search (PWJES), a new acquisition function that weights the joint entropy of the optimum location and its value by the precision of the posterior predictive distribution of the surrogate Gaussian Process. This yields an acquisition that is both more robust in regions of high epistemic uncertainty and more sample-efficient in well-explored regions, with favourable empirical behaviour across standard synthetic benchmarks and hyperparameter-tuning tasks.
The talk covers the derivation of the acquisition, its approximation via expectation propagation, and experimental comparisons with JES, MES and PES on both single- and multi-objective problems.