Conjugate gradients#
- class EarlyStopping.ConjugateGradients(design, response, initial_value=None, true_signal=None, true_noise_level=None, computation_threshold=1e-08)#
[Source] A class to perform estimation using the conjugate gradients algorithm for the normal equation.
Description
Consider the linear model
\[Y = Af + \delta Z,\]where \(Z\) is a \(n\)-dimensional standard normal distribution. The conjugate gradient estimate \(\hat{f}^{(m)}\) at the integer iteration index \(m\) is iteratively calculated by the conjugate gradients for the normal equation algorithm with initial value \(\hat{f}_0\), see Björck (1996, Algorithm 7.4.1).
Parameters
design:
ndarray. nxp design matrix of the linear model. ( \(A \in \mathbb{R}^{n \times p}\) )response:
ndarray. n-dim vector of the observed data in the linear model. ( \(Y \in \mathbb{R}^{n}\) )initial_value:
array, default = None. Determines the zeroth step of the iterative procedure. Default is zero. ( \(\hat{f}_0\) )true_signal:
ndarray, default = None. p-dim vector for simulation purposes only. For simulated data the true signal can be included to compute additional quantities alongside the iterative procedure. ( \(f \in \mathbb{R}^{p}\) )true_noise_level:
float, default = None. For simulation purposes only. Corresponds to the standard deviation of normally distributed noise contributing to the response variable. ( \(\delta \geq 0\) )computation_threshold:
float, default = 10 ** (-8). Threshold used to terminate the conjugate gradients algorithm.Attributes
sample_size:
int. Sample size of the linear model. ( \(n \in \mathbb{N}\) )parameter_size:
int. Parameter size of the linear model. ( \(p \in \mathbb{N}\) )iteration:
int. Current conjugate gradient iteration of the algorithm. ( \(m \in \mathbb{N}\) )conjugate_gradient_estimate:
ndarray. Conjugate gradient estimate at the current iteration for the data given in design and response. ( \(\hat{f}^{(m)}\) )conjugate_gradient_estimate_list:
list. List containing the conjugate gradient estimates at integer iteration indices up to the current conjugate gradient iteration.residuals:
ndarray. Lists the sequence of the squared residuals between the observed data and the conjugate gradient estimator.strong_empirical_risk:
ndarray. Only exists if true_signal was given. Lists the values of the strong empirical error between the conjugate gradient estimator and the true signal up to the current conjugate gradient iteration.weak_empirical_risk:
ndarray. Only exists if true_signal was given. Lists the values of the weak empirical error between the conjugate gradient estimator and the true signal up to the current conjugate gradient iteration.Methods
iterate(
number_of_iterations = 1)Performs a specified number of iterations of the conjugate gradients algorithm.
get_estimate(
iteration)Returns the conjugate gradient estimator at iteration.
get_discrepancy_stop(
critical_value,max_iteration,interpolation = False)Returns the early stopping index according to the discrepancy principle with emergency stop.
get_residual(
iteration)Returns the squared residual at iteration.
get_strong_empirical_risk(
iteration)Returns the strong empirical error at iteration.
get_weak_empirical_risk(
iteration)Returns the weak empirical error at iteration.
get_strong_empirical_oracle(
max_iteration,interpolation = False)Returns the strong empirical oracle up to max_iteration.
get_weak_empirical_oracle(
max_iteration,interpolation = False)Returns the weak empirical oracle up to max_iteration.
- ConjugateGradients.iterate(number_of_iterations=1)#
Performs number_of_iterations iterations of the conjugate gradients algorithm
Parameters
number_of_iterations:
int. The number of iterations to perform.
- ConjugateGradients.get_estimate(iteration)#
Returns the conjugate gradient estimate at a possibly noninteger iteration
Parameters
iteration:
int or float. The (possibly noninteger) iteration at which the conjugate gradient estimate is requested.Returns
conjugate_gradient_estimate:
ndarray. The conjugate gradient estimate at iteration.
- ConjugateGradients.get_discrepancy_stop(critical_value, max_iteration, interpolation=False)#
Returns early stopping index based on discrepancy principle up to max_iteration
Parameters
critical_value:
float. The critical value for the discrepancy principle. The algorithm stops when \(\Vert Y - A \hat{f}^{(m)} \Vert^{2} \leq \kappa^{2},\) where \(\kappa\) is the critical value.max_iteration:
int. The maximum number of total iterations to be considered.interpolation:
boolean, default = False. If interpolation is set toTrue, the early stopping index can be noninteger valued.Returns
early_stopping_index:
int or float. The first iteration at which the discrepancy principle is satisfied or \(\Vert A^{\top}(Y - A \hat{f}^{(m)}) \Vert^{2} \leq C,\) where \(C\) is the computation_threshold for the emergency stop. (Noneis returned if the stopping index is not found.)
- ConjugateGradients.get_residual(iteration)#
Returns the squared residual at a possibly noninteger iteration index
Parameters
iteration:
int or float. Iteration index where the squared residual should be calculated.Returns
residual:
float. The squared residual at the requested iteration index.
- ConjugateGradients.get_strong_empirical_risk(iteration)#
Returns the strong empirical error at a possibly noninteger iteration index
Parameters
iteration:
int or float. Iteration index where the error should be calculated.Returns
strong_empirical_risk:
float. The strong empirical error at the requested iteration index.
- ConjugateGradients.get_weak_empirical_risk(iteration)#
Returns the weak empirical error at a possibly noninteger iteration index
Parameters
iteration:
int or float. Iteration index where the error should be calculated.Returns
weak_empirical_risk:
float. The weak empirical error at the requested iteration index.
- ConjugateGradients.get_strong_empirical_oracle(max_iteration, interpolation=False)#
Returns the strong empirical oracle up to max_iteration
Parameters
max_iteration:
int. The maximum number of total iterations to be considered.interpolation:
boolean, default = False. If interpolation is set toTrue, the strong empirical oracle can be noninteger valued.Returns
strong_empirical_oracle:
int or float. The iteration index at which the strong empirical error is minimal along the iteration path up to max_iteration. If not unique, the smallest one is returned.
- ConjugateGradients.get_weak_empirical_oracle(max_iteration, interpolation=False)#
Returns the weak empirical oracle up to max_iteration
Parameters
max_iteration:
int. The maximum number of total iterations to be considered.interpolation:
boolean, default = False. If interpolation is set toTrue, the weak empirical oracle can be noninteger valued.Returns
weak_empirical_oracle:
int or float. The iteration index at which the weak empirical error is minimal along the iteration path up to max_iteration. If not unique, the smallest one is returned.