Developer Guide and Reference

  • 2021.1
  • 12/04/2020
  • Public Content
Contents

Principal Component Analysis

Principal Component Analysis (PCA) is a method for exploratory data analysis. PCA transforms a set of observations of possibly correlated variables to a new set of uncorrelated variables, called principal components. Principal components are the directions of the largest variance, that is, the directions where the data is mostly spread out.
Because all principal components are orthogonal to each other, there is no redundant information. This is a way of replacing a group of variables with a smaller set of new variables. PCA is one of powerful techniques for dimension reduction.

Details

Given a set LaTex Math image. of
p
-dimensional feature vectors or a LaTex Math image. correlation matrix and the number of principal components LaTex Math image. , the problem is to compute LaTex Math image. principal directions (eigenvectors) for the data set. The library returns the transformation matrix
T
of size LaTex Math image. , which contains eigenvectors in the row-major order and a vector of respective eigenvalues in descending order.
oneDAL provides two methods for running PCA:
  • SVD
  • Correlation
Eigenvectors computed by PCA are not uniquely defined due to sign ambiguity. PCA supports fast ad-hoc “sign flip” technique described in the paper [Bro07]. It modifies the signs of eigenvectors shown below:
LaTex Math image.
where
T
-transformation matrix is computed by PCA, LaTex Math image. -
i
-th row in the matrix,
j
- column number,
sgn
- signum function:
LaTex Math image.
You can provide these types of input data to the PCA algorithms of the library:
  • Original, non-normalized data set
  • Normalized data set, where each feature has the zero mean and unit variance
  • Correlation matrix

Computation

The following computation modes are available:

Examples

oneAPI DPC++
Batch Processing:
oneAPI C++
Batch Processing:
Python* with DPC++ support
Batch Processing:
Python*
Batch Processing:
Distributed Processing:

Performance Considerations

To get the best overall performance of the PCA algorithm:
  • If input data is homogeneous, provide the input data and store results in homogeneous numeric tables of the same type as specified in the algorithmFPType class template parameter.
  • If input data is non-homogeneous, use AOS layout rather than SOA layout.
PCA computation using the correlation method involves the correlation and variance-covariance matrices algorithm. Depending on the method of this algorithm, the performance of PCA computations may vary. For sparse data sets, use the methods of this algorithm for sparse data.
Batch Processing
Because the PCA in the batch processing mode performs normalization for data passed as Input ID, to achieve the best performance, normalize the input data set. To inform the algorithm that the data is normalized, set the normalization flag for the input numeric table that represents your data set by calling the setNormalizationFlag() method of the NumericTableIface class.
Because the PCA with the correlation method (defaultDense) in the batch processing mode is based on the computation of the correlation matrix, to achieve the best performance, precompute the correlation matrix. To pass the precomputed correlation matrix to the algorithm, use correlation as Input ID.
Online Processing
PCA with the SVD method (svdDense) in the online processing mode is at least as computationally complex as in the batch processing mode and has high memory requirements for storing auxiliary data between calls to compute(). On the other hand, the online version of the PCA with the SVD method may enable you to hide the latency of reading data from a slow data source. To do this, implement load prefetching of the next data block in parallel with the compute() method for the current block.
Distributed Processing
PCA with the SVD method (svdDense) in the distributed processing mode requires gathering local-node LaTex Math image. numeric tables on the master node. When the amount of local-node work is small, that is, when the local-node data set is small, the network data transfer may become a bottleneck. To avoid this situation, ensure that local nodes have a sufficient amount of work. For example, distribute the input data set across a smaller number of nodes.
Optimization Notice
Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.
Notice revision #20110804

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.