Landweber iteration#
- class EarlyStopping.Landweber(design, response, learning_rate=1, initial_value=None, true_signal=None, true_noise_level=None)#
[Source] A class to perform estimation using the Landweber iterative method.
Description
Consider the linear model
\[Y = Af + \delta Z,\]where \(Z\) is a \(D\)-dimensional normal distribution. The landweber iteration is defined through:
\[\hat{f}^{(0)}=\hat{f}_0, \quad \hat{f}^{(m+1)}= \hat{f}^{(m)} + A^{\top}(Y-A \hat{f}^{(m)}).\]Parameters
design:
array. design matrix of the linear model. ( \(A \in \mathbb{R}^{D \times p}\) )response:
array. n-dim vector of the observed data in the linear model. ( \(Y \in \mathbb{R}^{D}\) )initial_value:
array, default = None. Determines the zeroth step of the iterative procedure. Default is zero. ( \(\hat{f}_0\) )true_signal:
array, default = None. p-dim vector For simulation purposes only. For simulated data the true signal can be included to compute theoretical quantities such as the bias and the mse alongside the iterative procedure. ( \(f \in \mathbb{R}^{p}\) )true_noise_level:
float, default = NoneFor simulation purposes only. Corresponds to the standard deviation of normally distributed noise contributing to the response variable. Allows the analytic computation of the strong and weak variance. ( \(\delta \geq 0\) )Attributes
sample_size:
int. Sample size of the linear model ( \(D \in \mathbb{N}\) )parameter_size:
int. Parameter size of the linear model ( \(p \in \mathbb{N}\) )iteration:
int. Current Landweber iteration of the algorithm ( \(m \in \mathbb{N}\) )residuals:
array. Lists the sequence of the squared residuals between the observed data and the Landweber estimator.strong_bias2:
array. Only exists if true_signal was given. Lists the values of the strong squared bias up to the current Landweber iteration.\[B^{2}_{m} = \Vert (I-A^{\top}A)(f-\hat{f}_{m-1}) \Vert^{2}\]strong_variance:
array. Only exists if true_signal was given. Lists the values of the strong variance up to the current Landweber iteration.\[V_m = \delta^2 \mathrm{tr}((A^{\top}A)^{-1}(I-(I-A^{\top}A)^{m})^{2})\]strong_risk:
array. Only exists if true_signal was given. Lists the values of the strong norm error between the Landweber estimator and the true signal up to the current Landweber iteration.\[E[\Vert \hat{f}_{m} - f \Vert^2] = B^{2}_{m} + V_m\]weak_bias2:
array. Only exists if true_signal was given. Lists the values of the weak squared bias up to the current Landweber iteration.\[B^{2}_{m,A} = \Vert A(I-A^{\top}A)(f-\hat{f}_{m-1}) \Vert^{2}\]weak_variance:
array. Only exists if true_signal was given. Lists the values of the weak variance up to the current Landweber iteration.\[V_{m,A} = \delta^2 \mathrm{tr}((I-(I-A^{\top}A)^{m})^{2})\]weak_risk:
array. Only exists if true_signal was given. Lists the values of the weak norm error between the Landweber estimator and the true signal up to the current Landweber iteration.\[E[\Vert \hat{f}_{m} - f \Vert_A^2] = B^{2}_{m,A} + V_{m,A}\]Methods
iterate(
number_of_iterations=1)Performs a specified number of iterations of the Landweber algorithm.
landweber_to_early_stop(
max_iter)Applies early stopping to the Landweber iterative procedure.
get_estimate(
iteration)Returns the landweber estimator at iteration.
get_discrepancy_stop(
critical_value, max_iteration)Returns the early stopping index according to the discrepancy principle.
get_weak_balanced_oracle(
max_iteration)Returns the weak balanced oracle if found up to max_iteration.
get_strong_balanced_oracle(
max_iteration)Returns the strong balanced oracle if found up to max_iteration.
- get_discrepancy_stop(critical_value, max_iteration)#
Returns early stopping index based on discrepancy principle up to max_iteration
Parameters
critical_value:
float. The critical value for the discrepancy principle. The algorithm stops when :math: Vert Y - A hat{f}^{(m)} Vert^{2} leq kappa^{2}, where :math: kappa is the critical value.max_iteration:
int. The maximum number of total iterations to be considered.Returns
early_stopping_index:
int. The first iteration at which the discrepancy principle is satisfied. (None is returned if the stopping index is not found.)
- get_estimate(iteration)#
Returns the Landweber estimate at iteration
Parameters
iteration:
int. The iteration at which the Landweber estimate is requested.Returns
landweber_estimate:
ndarray. The Landweber estimate at iteration.
- get_strong_balanced_oracle(max_iteration)#
Returns strong balanced oracle if found up to max_iteration.
Parameters
max_iteration:
int. The maximum number of total iterations to be considered.Returns
strong_balanced_oracle:
int. The first iteration at which the strong bias is smaller than the strong variance.
- get_weak_balanced_oracle(max_iteration)#
Returns weak balanced oracle if found up to max_iteration.
Parameters
max_iteration:
int. The maximum number of total iterations to be considered.Returns
weak_balanced_oracle:
int. The first iteration at which the weak bias is smaller than the weak variance.
- iterate(number_of_iterations)#
Performs number_of_iterations iterations of the Landweber algorithm
Parameters
number_of_iterations:
int. The number of iterations to perform.