Developer Guide

Contents

Computation

The stochastic average gradient (SAGA) algorithm is a special case of an iterative solver. For parameters, input, and output of iterative solvers, see
Iterative Solver
Computation
.

Algorithm Input

In addition to the input of the iterative solver, the SAGA optimization solver has the following optional input:
OptionalDataID
Default Value
Description
gradientTable
NULL
Numeric table of size
n
x
p
which represents
G
0
matrix that contains gradients of
at the initial point
This input is optional: if the user does not provide the table of gradients for
, the library will compute it inside the SAGA algorithm.
This parameter can be an object of any class derived from
NumericTable
, except for
PackedTriangularMatrix
,
PackedSymmetricMatrix
, and
CSRNumericTable
.

Algorithm Parameters

In addition to parameters of the iterative solver, the SAGA optimization solver has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be float or double.
method
defaultDense
Performance-oriented computation method.
batchIndices
1
Numeric table of size
nIterations
x 1 with 32-bit integer indices of terms in the objective function. If no indices are provided, the implementation generates random index on each iteration.
This parameter can be an object of any class derived from
NumericTable
, except for
PackedTriangularMatrix
,
PackedSymmetricMatrix
, and
CSRNumericTable
.
learningRateSequence
NULL
The numeric table of size 1 x
nIterations
or 1 x 1 that contains learning rate for each iterations is first case, otherwise constant step length will be used for all iterations. It`s recommended to set diminishing learning rate sequence.
This parameter can be an object of any class derived from
NumericTable
, except for
PackedTriangularMatrix
,
PackedSymmetricMatrix
, and
CSRNumericTable
.
If
learningRateSequence
is not provided, the learning rate will be computed automatically via
constantOfLipschitz
result-id
.
engine
SharedPtr<engines::mt19937::Batch<>
Pointer to the random number generator engine that is used internally for generation of 32-bit integer index of term in the objective function.

Algorithm Output

In addition to the output of the iterative solver, the SAGA optimization solver calculates the following optional result:
OptionalDataID
Default Value
Description
gradientTable
NULL
Numeric table of size
n
x
p
which represents updated after all iterations matrix
G
t
This parameter can be an object of any class derived from
NumericTable
, except for
PackedTriangularMatrix
,
PackedSymmetricMatrix
, and
CSRNumericTable
.

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804