Surrogate module

References

• CKI Williams, and CE Rasmussen, Gaussian Processes for Machine Learning, 2nd ed. Cambridge, MA: MIT press, 2006.

• RB Gramacy, Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences, 1st ed. Boca Raton, FL: CRC press, 2020.

• J Gardner, G Pleiss, KQ Weinberger, D Bindel, and AG Wilson, “GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with Gpu Acceleration,” Advances in Neural Information Processing Systems, vol. 31, 2018.

Gaussian process

References

• J Snoek, H Larochelle, and RP Adams, “Practical Bayesian Optimization of Machine Learning Algorithms,” Advances in Neural Information Processing Systems, vol. 25, 2012.

• CKI Williams, and CE Rasmussen, Gaussian Processes for Machine Learning, 2nd ed. Cambridge, MA: MIT press, 2006.

• RB Gramacy, Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences, 1st ed. Boca Raton, FL: CRC press, 2020.

class nubo.models.gaussian_process.GaussianProcess(x_train: Tensor, y_train: Tensor, likelihood: Likelihood)

Bases: ExactGP

Gaussian process model with constant mean function and Matern 5/2 kernel.

Constant mean function:

\[\mu (\boldsymbol x) = c,\]

where constant \(c\) is estimated.

Matern 5/2 Kernel:

\[\Sigma_0 (\boldsymbol x, \boldsymbol x^\prime) = \sigma_K^2 \left(1 + \sqrt{5}r + \frac{5}{3}r^2 \right) \exp{\left(-\sqrt{5}r \right)},\]

where \(r = \sqrt{\sum_{m=1}^d \frac{(\boldsymbol x_m - \boldsymbol x^\prime_m)^2}{l^2_m}}\), \(l\) is the length-scale, \(\sigma_K^2\) is the outputscale, and \(m\) is the \(m\)-th dimension of the input points.

Attributes:

x_traintorch.Tensor

(size n x d) Training inputs.

y_traintorch.Tensor

(size n) Training outputs.

likelihoodgpytorch.likelihoods.Likelihood

Likelihood.

mean_modulegpytorch.means

Zero mean function.

covar_modulegpytorch.kernels

Automatic relevance determination Matern 5/2 covariance kernel.

Methods

forward(x)

Compute the mean vector and covariance matrix for some test points x and returns a multivariate normal distribution.

forward(x: Tensor) → MultivariateNormal

Compute the mean vector and covariance matrix for some test points x and returns a multivariate normal distribution.

Parameters:

xtorch.Tensor

(size n x d) Test points.

Returns:

gpytorch.distributions.MultivariateNormal

Predictice multivariate normal distribution.

Hyper-parameter estimation

References

• CKI Williams, and CE Rasmussen, Gaussian Processes for Machine Learning, 2nd ed. Cambridge, MA: MIT press, 2006.

• RB Gramacy, Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences, 1st ed. Boca Raton, FL: CRC press, 2020.

• DP Kingma and J Ba, “Adam: A Method for Stochastic Optimization,” Proceedings of the 3rd International Conference on Learning Representations, 2015.

nubo.models.fit.fit_gp(x: Tensor, y: Tensor, gp: GP, likelihood: Likelihood, lr: float | None = 0.1, steps: int | None = 200, **kwargs) → None

Estimate hyper-parameters of the Gaussian process gp by maximum likelihood estimation (MLE) using torch.optim.Adam algorithm.

Maximises the log marginal likelihood \(\log p(\boldsymbol y \mid \boldsymbol X)\).

Parameters:

xtorch.Tensor

(size n x d) Training inputs.

ytorch.Tensor

(size n) Training targets.

gpgpytorch.likelihoods.Likelihood

Gaussian Process model.

lrfloat, optional

Learning rate of torch.optim.Adam algorithm, default is 0.1.

stepsint, optional

Optimisation steps of torch.optim.Adam algorithm, default is 200.

**kwargsAny

Keyword argument passed to torch.optim.Adam.