L2-boosting for high-dimensional linear models#
- class EarlyStopping.L2_boost(design, response, true_signal=None)#
[Source] L2-boosting algorithm for high dimensional linear models.
Parameters
design:
array. nxp-Design matrix of the linear model.response:
array. n-dim vector of the observed data in the linear model.true_signal:
array or None, default = None. For simulation purposes only. For simulated data the true signal can be included to compute theoretical quantities such as the bias and the mse alongside the boosting procedure.Attributes
sample_size:
int. Sample size of the linear model.parameter_size:
int. Parameter size of the linear model.iter:
int. Current boosting iteration of the algorithm.boost_estimate:
array. Boosting estimate at the current iteration for the data given in design.residuals:
array. Lists the sequence of the residual mean of squares betwean the data and the boosting estimator.bias2:
array. Only exists if true_signal was given. Lists the values of the squared bias up to current boosting iteration.stoch_error:
array. Only exists if true_signal was given. Lists the values of a stochastic error term up to current boosting iteration.mse:
array. Only exists if true_signal was given. Lists the values of the mean squared error betwean the boosting estimator and the true signal up to current boosting iteration.Methods
iterate(
number_of_iterations=1)Performs number of iterations of the boosting algorithm.
predict(
design_observation)Predicts the response based on the current boosting estimate.
discrepancy_stop(
crit, max_iter)Stops the boosting algorithm based on the discrepancy principle.
residual_ratio_stop(
max_iter, alpha=0.05, K=1.2)Stops the boosting algorithm based on the residual ratios.
get_noise_estimate(
K=1)Computes a noise estimate for the model via the scaled lasso.
get_aic_iteration(
K=2)Computes the minimizer of a high dimensional Akaike criterion.
boost_to_balanced_oracle()
Iterates the boosting algorithm up to the balanced oracle.
- L2_boost.iterate(number_of_iterations=1)#
Performs number_of_iterations iterations of the orthogonal boosting algorithm.
Parameters
number_of_iterations:
int. Number of boosting iterations to be performed.
- L2_boost.predict(design_observation)#
Predicts the output variable based on the current boosting estimate.
Parameters
input_variable:
array. The size of input_variable has to match parameter_size.
- L2_boost.discrepancy_stop(crit, max_iter)#
Early stopping for the boosting procedure based on the discrepancy principle. Procedure is stopped when the residuals go below crit or iteration max_iter is reached.
Parameters
crit:
float. Critical value for the early stopping procedure.max_iter:
int. Maximal number of iterations to be performed.
- L2_boost.residual_ratio_stop(max_iter, alpha=0.05, K=1.2)#
Stops the algorithm based on a residual ration criterion.
Parameters
alpha:
float. accuracy levelK:
float. Constant for the definition.max_iter:
int. Maximal number of iterations to be performed.
- L2_boost.get_noise_estimate(K=1)#
Computes an estimator for the noise level sigma^2 of the model via the scaled Lasso.
Parameters
K:
float. Constant in the definition. Defaults to 1, which is the choice from the scaled Lasso paper.
- L2_boost.get_aic_iteration(K=2)#
Computes the iteration index minimizing a high dimensional Akaike criterion.
Parameters
K:
float. Constant in the definition. Defaults to 2, which is common in the literature.
- L2_boost.boost_to_balanced_oracle()#
Performs iterations of the orthogonal boosting algorithm until the balanced oracle index at which the squared bias is smaller than the stochastic error is reached.