Principal Component Analysis
Principal Component Analysis (PCA) is a method for exploratory data
analysis. PCA transforms a set of observations of possibly correlated
variables to a new set of uncorrelated variables, called principal
components. Principal components are the directions of the largest
variance, that is, the directions where the data is mostly spread
out.
Because all principal components are orthogonal to each other, there
is no redundant information. This is a way of replacing a group of
variables with a smaller set of new variables. PCA is one of powerful
techniques for dimension reduction.
Details
Given a set
of 
correlation matrix and the number of principal components
,
the problem is to compute
principal directions
(eigenvectors) for the data set. The library returns the
transformation matrix 
, which contains
eigenvectors in the row-major order and a vector of respective
eigenvalues in descending order.
of p
-dimensional
feature vectors or a
correlation matrix and the number of principal components
,
the problem is to compute
principal directions
(eigenvectors) for the data set. The library returns the
transformation matrix T
of size
, which contains
eigenvectors in the row-major order and a vector of respective
eigenvalues in descending order.oneDAL provides two methods for running PCA:
- SVD
- Correlation
Eigenvectors computed by PCA are not uniquely defined due to sign
ambiguity. PCA supports fast ad-hoc “sign flip” technique described
in the paper [Bro07]. It modifies the signs of eigenvectors shown below:
where 
-
T
-transformation matrix is computed by PCA,
- i
-th
row in the matrix, j
- column number, sgn
- signum function:
You can provide these types of input data to the PCA algorithms of
the library:
- Original, non-normalized data set
- Normalized data set, where each feature has the zero mean and unit variance
- Correlation matrix
Computation
The following computation modes are available:
Examples
oneAPI DPC++
Batch Processing:
oneAPI C++
Batch Processing:
C++ (CPU)
Batch Processing:
Online Processing:
Distributed Processing:
Java*
There is no support for Java on GPU.
Batch Processing:
Online Processing:
Distributed Processing:
Python* with DPC++ support
Batch Processing:
Python*
Performance Considerations
To get the best overall performance of the PCA algorithm:
- If input data is homogeneous, provide the input data and store results in homogeneous numeric tables of the same type as specified in the algorithmFPType class template parameter.
- If input data is non-homogeneous, use AOS layout rather than SOA layout.
PCA computation using the correlation method involves the correlation
and variance-covariance matrices algorithm. Depending on the method
of this algorithm, the performance of PCA computations may vary. For
sparse data sets, use the methods of this algorithm for sparse data.
Batch Processing
Because the PCA in the batch processing mode performs
normalization for data passed as Input ID, to achieve the best
performance, normalize the input data set. To inform the algorithm
that the data is normalized, set the normalization flag for the
input numeric table that represents your data set by calling the
setNormalizationFlag() method of the NumericTableIface class.
Because the PCA with the correlation method (defaultDense) in the
batch processing mode is based on the computation of the
correlation matrix, to achieve the best performance, precompute
the correlation matrix. To pass the precomputed correlation matrix
to the algorithm, use correlation as Input ID.
Online Processing
PCA with the SVD method (svdDense) in the online processing mode
is at least as computationally complex as in the batch processing
mode and has high memory requirements for storing auxiliary data
between calls to compute(). On the other hand, the online version
of the PCA with the SVD method may enable you to hide the latency
of reading data from a slow data source. To do this, implement
load prefetching of the next data block in parallel with the
compute() method for the current block.
Distributed Processing
PCA with the SVD method (svdDense) in the distributed processing
mode requires gathering local-node
numeric tables on the
master node. When the amount of local-node work is small, that is,
when the local-node data set is small, the network data transfer
may become a bottleneck. To avoid this situation, ensure that
local nodes have a sufficient amount of work. For example,
distribute the input data set across a smaller number of nodes.
numeric tables on the
master node. When the amount of local-node work is small, that is,
when the local-node data set is small, the network data transfer
may become a bottleneck. To avoid this situation, ensure that
local nodes have a sufficient amount of work. For example,
distribute the input data set across a smaller number of nodes.Optimization Notice |
|---|
Intel’s compilers may or may not optimize to the same degree for
non-Intel microprocessors for optimizations that are not unique to
Intel microprocessors. These optimizations include SSE2, SSE3, and
SSSE3 instruction sets and other optimizations. Intel does not
guarantee the availability, functionality, or effectiveness of any
optimization on microprocessors not manufactured by Intel.
Microprocessor-dependent optimizations in this product are intended
for use with Intel microprocessors. Certain optimizations not
specific to Intel microarchitecture are reserved for Intel
microprocessors. Please refer to the applicable product User and
Reference Guides for more information regarding the specific
instruction sets covered by this notice. Notice revision #20110804 |