Computation

The stochastic average gradient (SAGA) algorithm is a special case of an iterative solver. For parameters, input, and output of iterative solvers, see Iterative Solver > Computation .

Algorithm Input

In addition to the input of the iterative solver, the SAGA optimization solver has the following optional input:

OptionalDataID

Default Value

Description

gradientTable

NULL

Numeric table of size n x p which represents G 0 matrix that contains gradients of at the initial point

This input is optional: if the user does not provide the table of gradients for , the library will compute it inside the SAGA algorithm.

This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable.

Algorithm Parameters

In addition to parameters of the iterative solver, the SAGA optimization solver has the following parameters:

Parameter

Default Value

Description

algorithmFPType

float

The floating-point type that the algorithm uses for intermediate computations. Can be float or double.

method

defaultDense

Performance-oriented computation method.

batchIndices

1

Numeric table of size nIterations x 1 with 32-bit integer indices of terms in the objective function. If no indices are provided, the implementation generates random index on each iteration.

This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable.

learningRateSequence

NULL

The numeric table of size 1 x nIterations or 1 x 1 that contains learning rate for each iterations is first case, otherwise constant step length will be used for all iterations. It`s recommended to set diminishing learning rate sequence.

This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable.

If learningRateSequence is not provided, the learning rate will be computed automatically via constantOfLipschitz result-id.

engine

SharedPtr<engines::mt19937::Batch<>

Pointer to the random number generator engine that is used internally for generation of 32-bit integer index of term in the objective function.

Algorithm Output

In addition to the output of the iterative solver, the SAGA optimization solver calculates the following optional result:

OptionalDataID

Default Value

Description

gradientTable

NULL

Numeric table of size n x p which represents updated after all iterations matrix G t

This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable.

Examples

C++:

  • saga_dense_batch.cpp
  • saga_log_loss_dense_batch.cpp

Java*:

  • SagaDenseBatch.java
  • SagaLogLossDenseBatch.java

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)