The default method (defaultDense) of SAGA algorithm is a particular case of the iterative solver method with the batch size b=1.
Algorithmic-specific transformation T, set of intrinsic parameters S t defined for the learning η rate, and algorithm-specific vector U and power d of Lebesgue space are defined as follows:
S t - a matrix of the gradients of smooth terms at point , where
Update of the set of intrinsic parameters S t: .
NoteThe algorithm enables automatic step-length selection if learning rate η was not provided by the user. Automatic step-length will be computed as , where L - Lipschitz constant returned by objective function. If the objective function returns nullptr to numeric table with lipschitzConstantresult-id, the library will use default step size '0.01'.