DAAL4PY has different results with sklearn PCA

DAAL4PY has different results with sklearn PCA

Hi guys:

I was using daal4py for large data set. It is super fast, but the result looks wrong.

I had already searched the web and I know that DAAL4py will normalize data before compute PCA. But still, I normalized data and pass to sklearn, the eigen values and vectors varying a lot!

Attached the code and data, I like to see a consist data between them. Otherwise, we are not convinced we could use DAAL in our production. Please note that preprocessing data before passing to sklearn is acceptable. But I tried both minmax/zscore method. The results are quite different from intel PCA results.

test.zip has the testing data, which is 10000 * 512 tensor.

Run the script on my windows I got the following result.

Intel engvals = [136.30274983  85.45575273  51.9877961 ]
Sklearn engvals = [213.9291753  102.09328516  76.81426116]

AttachmentSize
Downloadtext/plain validation.py.txt613 bytes
Downloadapplication/zip test.zip4.97 MB
2 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

I figured out the issue. Intel PCA always normalization the data before computation. While, you have to set doScale = True and then pass to sklearn, and expected_variance is the eigen values instead of singular values.

Compare with that results, they are pretty similar. 

No issues on DAAL side.

Leave a Comment

Please sign in to add a comment. Not a member? Join today