optilab.functions.surrogate package

Submodules

optilab.functions.surrogate.knn_surrogate_objective_function module

Surrogate Objective function using FAISS for fast KNN-based regression.

class KNNSurrogateObjectiveFunction(num_neighbors: int, train_set: PointList | None = None)

Bases: SurrogateObjectiveFunction

Surrogate objective function using FAISS for fast KNN-based regression.

__init__(num_neighbors: int, train_set: PointList | None = None) → None

Class constructor.

Parameters:

• num_neighbors – Number of closest neighbors to use in regression.

• train_set – Training data for the model.

train(train_set: PointList) → None

Train the FAISS-based KNN Surrogate function with provided data.

Parameters:

train_set – Training data for the model.

__call__(point: Point) → Point

Estimate the function value at a given point using kNN regression.

Parameters:

point – Point to estimate.

Returns:

Estimated value of the function at the given point.

optilab.functions.surrogate.locally_weighted_polynomial_regression module

Surrogate function which estimates the objective function with polynomial regression. Points are weighted based on mahalanobis distance from query points.

biquadratic_kernel_function(x: float) → float

Biquadratic weighting function.

Parameters:

x – Distance between points.

Returns:

Weight value.

class LocallyWeightedPolynomialRegression(degree: int, num_neighbors: int, train_set: ~optilab.data_classes.point_list.PointList | None = None, covariance_matrix: ~numpy.ndarray | None = None, kernel_function: ~typing.Callable[[float], float] = <function biquadratic_kernel_function>)

Bases: SurrogateObjectiveFunction

Surrogate function which estimates the objective function with polynomial regression. Points are weighted based on mahalanobis distance from query points.

__init__(degree: int, num_neighbors: int, train_set: ~optilab.data_classes.point_list.PointList | None = None, covariance_matrix: ~numpy.ndarray | None = None, kernel_function: ~typing.Callable[[float], float] = <function biquadratic_kernel_function>) → None

Class constructor.

Parameters:

• degree – Degree of the polynomial used to approximate function.

• num_neighbors – Number of closest points to use in function approximation.

• train_set – Training set for the regressor, optional.

• covariance_matrix – Covariance class used in mahalanobis distance, optional. When no such matrix is provided an identity matrix is used.

• kernel_function – Function used to assign weights to points.

set_covariance_matrix(new_covariance_matrix: ndarray) → None

Setter for the covariance matrix.

Parameters:

new_covariance_matrix – New covariance matrix to use for mahalanobis distance.

train(train_set: PointList) → None

Build FAISS index and preprocess data to use Mahalanobis distance.

Parameters:

train_set – Training set for the function

__call__(point: Point) → Point

Estimate the value of a single point with the surrogate function. Since the surrogate model is built for each point independently, this is where the regressor is trained.

Parameters:

x – Point to estimate.

Raises:

ValueError – If dimensionality of x doesn’t match self.dim.

Returns:

Estimated point.

Return type:

Point

optilab.functions.surrogate.mlp_surrogate_objective_function module

Surrogate objective function using sklearn MLPRegressor.

class MLPSurrogateObjectiveFunction(hidden_layer_sizes: Tuple[int] = (32,), train_set: PointList | None = None, *, activation: str = 'relu', solver: str = 'adam', learning_rate_init: float = 0.001, l2_alpha: float = 0.0001, max_iter: int = 200, early_stopping: bool = True, random_seed: int | None = None)

Bases: SurrogateObjectiveFunction

Surrogate objective function using sklearn MLPRegressor.

__init__(hidden_layer_sizes: Tuple[int] = (32,), train_set: PointList | None = None, *, activation: str = 'relu', solver: str = 'adam', learning_rate_init: float = 0.001, l2_alpha: float = 0.0001, max_iter: int = 200, early_stopping: bool = True, random_seed: int | None = None) → None

Class constructor.

Parameters:

• hidden_layer_sizes – Shape of hidden layers.

• train_set – Training data for the model.

• activation – Activation function for hidden layers.

• solver – The solver for weight optimization.

• learning_rate_init – Initial learning rate used by optimizer.

• l2_alpha – L2 regularization weight.

• max_iter – Maximum number of optimization iterations.

• early_stopping – Whether to use validation-based early stopping.

• random_seed – Seed for reproducible initialization.

train(train_set: PointList) → None

Train the MLP surrogate function with provided data.

Parameters:

train_set – Training data for the model.

__call__(point: Point) → Point

Estimate the function value at a given point using MLP regression.

Parameters:

point – Point to estimate.

Returns:

Estimated value of the function at the given point.

optilab.functions.surrogate.polynomial_regression module

Surrogate objective function which approximates the function value with polynomial regression with interactions optimized using least squares.

class PolynomialRegression(degree: int, train_set: PointList | None = None)

Bases: SurrogateObjectiveFunction

Surrogate objective function which approximates the function value with polynomial regression with interactions optimized using least squares.

__init__(degree: int, train_set: PointList | None = None) → None

Class constructor.

Parameters:

• degree – Degree of the polynomial used for approximation.

• train_set – Training data for the model.

train(train_set: PointList) → None

Train the Surrogate function with provided data

Parameters:

train_set – Train data for the model.

__call__(point: Point) → Point

Estimate the value of a single point with the surrogate function.

Parameters:

point – Point to estimate.

Raises:

ValueError – If dimensionality of x doesn’t match self.dim.

Returns:

Estimated value of the function in the provided point.

optilab.functions.surrogate.surrogate_objective_function module

Abstract base class for surrogate objective functions.

class SurrogateObjectiveFunction(name: str, train_set: PointList | None = None, hyperparameters: Dict[str, Any] | None = None)

Bases: ObjectiveFunction

Abstract base class for surrogate objective functions.

__init__(name: str, train_set: PointList | None = None, hyperparameters: Dict[str, Any] | None = None) → None

Class constructor. The dimensionality is deduced from the training points.

Parameters:

• name – Name of the surrogate function.

• train_set – Training data for the model.

• hyperparameters – Dictionary with hyperparameters of the function.

train(train_set: PointList) → None

Train the Surrogate function with provided data.

Parameters:

train_set – Training data for the model.

Raises:

ValueError – If not all points are evaluated.

__call__(point: Point) → Point

Estimate the value of a single point with the surrogate function.

Parameters:

point – Point to estimate.

Raises:

ValueError – If dimensionality of x doesn’t match self.dim.

Returns:

Estimated value of the function in the provided point.

optilab.functions.surrogate.xgboost_surrogate_objective_function module

Surrogate objective function using XGBoost for gradient-boosted tree regression.

class XGBoostSurrogateObjectiveFunction(n_estimators: int = 100, max_depth: int = 6, learning_rate: float = 0.1, train_set: PointList | None = None)

Bases: SurrogateObjectiveFunction

Surrogate objective function using XGBoost for gradient-boosted tree regression.

__init__(n_estimators: int = 100, max_depth: int = 6, learning_rate: float = 0.1, train_set: PointList | None = None) → None

Class constructor.

Parameters:

• n_estimators – Number of boosting rounds.

• max_depth – Maximum depth of each tree.

• learning_rate – Step size shrinkage used to prevent overfitting.

• train_set – Training data for the model.

train(train_set: PointList) → None

Train the XGBoost surrogate function with provided data.

Parameters:

train_set – Training data for the model.

__call__(point: Point) → Point

Estimate the function value at a given point using XGBoost regression.

Parameters:

point – Point to estimate.

Returns:

Estimated value of the function at the given point.

Module contents

Surrogate objective functions, regressors used to estimate objective function values.