Developer Guide and Reference

  • 2021.2
  • 03/26/2021
  • Public Content

Iterative Solver

The iterative solver provides an iterative method to minimize an objective function that can be represented as a sum of functions in composite form
LaTex Math image.
  • LaTex Math image., LaTex Math image., where LaTex Math image. is a convex, continuously differentiable LaTex Math image. (smooth) functions,
    i = 1, …, n
  • LaTex Math image. is a convex, non-differentiable (non-smooth) function
The Algorithmic Framework of an Iterative Solver
All solvers presented in the library follow a common algorithmic framework. Let LaTex Math image. be a set of intrinsic parameters of the iterative solver for updating the argument of the objective function. This set is the algorithm-specific and can be empty. The solver determines the choice of LaTex Math image..
To do the computations, iterate
until LaTex Math image.:
  1. Choose a set of indices without replacement LaTex Math image., LaTex Math image.,
    j = 1, ldots, b
    , where
    is the batch size.
  2. Compute the gradient LaTex Math image. where LaTex Math image.
  3. Convergence check:
    Stop if LaTex Math image. where
    is an algorithm-specific vector (argument or gradient) and d is an algorithm-specific power of Lebesgue space
  4. Compute LaTex Math image. using the algorithm-specific transformation
    that updates the function’s argument:
    LaTex Math image.
  5. Update LaTex Math image. where
    is an algorithm-specific update of the set of intrinsic parameters.
The result of the solver is the argument LaTex Math image. and a set of parameters LaTex Math image. after the exit from the loop.
You can resume the computations to get a more precise estimate of the objective function minimum. To do this, pass to the algorithm the results LaTex Math image. and LaTex Math image. of the previous run of the optimization solver. By default, the solver does not return the set of intrinsic parameters. If you need it, set the
flag for the algorithm.

Product and Performance Information


Performance varies by use, configuration and other factors. Learn more at