MKL PARDISO - Large Scale Problem

MKL PARDISO - Large Scale Problem

Hi,

I am solving [A][X] = [B] where both X and B are matrices. I
am solving for X. Size of [A] is 1,500,000 x 1,500,000. Aproximate non
zeros are 60,000,000. I am able to run my code for the lower size
problem. But for this size it does not run.

System configuration:

SGI - 8 processor with 32 GB memory.
Compiler - Intel 10.1.012.

I am using 10.0.1.014 intel mkl

When I am using iparm(60) = 2 it kill the process with -9 error

***************************FOLLOWING WAS ERROR******************************

The file ./pardiso_ooc.cfg was not opened
*** error PARDISO ( not enough mem. for OOC) error_num= -9

================ PARDISO: solving a symmetric indef. system ================

Summary PARDISO: ( reorder to reorder )
================

Times:
======

Time fulladj: 1.360909 s
Time reorder: 29.476320 s
Time symbfct: 34.058441 s
Time malloc : 0.000549 s
Time total : 64.896383 s total - sum: 0.000164 s

Statistics:
===========
< Parallel Direct Factorization with #processors: > 1
< Numerical Factorization with Level-3 BLAS performance >

< Linear system Ax = b>
#equations: 1460346
#non-zeros in A: 58022798
non-zeros in A (%): 0.002721
#right-hand sides: 1

< Factors L and U >
#columns for each panel: 10
#independent subgraphs: 0
< Preprocessing with state of the art partitioning metis>
#supernodes: 231589
size of largest supernode: 20746
number of nonzeros in L -905604445
number of nonzeros in U 1
number of nonzeros in L+U -905604444
test
check point 1
check point 2
check point 3
check point 5
check point 7
58022799
Reordering completed ...
The following ERROR was detected: -9

*****************************************************************************
Questions:

1) Is it possible to solve using current resources (8-Processors with 32GB RAM)?
2)
If I want to run this size problem then what is the maximum RAM on
single processor will I need? As we are planning to buy a system that
has sufficient memory for this type of operations.

Thank you,

Mital Mistry.

7 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hi Mital,I would suggest you to try to use MKL version 10.3 first of all. A lot of significant improvements has been implemented since MKL 10.0 in PARDISO.Regards,Konstantin

Thanks Konstantin,

I will do that and also, I tried following,

export MKL_PARDISO_OOC_MAX_CORE_SIZE=16000

And the process did not stop but it kept running for almost 2 days and did nothing.

Also, the process is not possible by in-core right?

Regards,

Mital

Konstantin,

We have IA-64 architecture. So is there any special needs for updating to MKL 10.3 version?

Regards,
Mital

Hi, the workstation of mine is with 32G memery and two 6-core XEON E5-2630 cpu.

I wonder a sparse matrix with about 2million unknowns (roughly 20 million non-zero elements in the matrix) can be solved by the PARDISO  within reasonable time.

Thanks a lot! 

Yanpu:

I'm afraid that the answer is "It depends". Factorization of a highly sparse matrix (yours has a density of 0.000005) can be accompanied by substantial fill-in, and your system is far short of the memory that a dense matrix would take for the same number of unknowns.

You could try smaller problems and run tests to measure how the calculation scales, and project the results to the full size to obtain an estimate the run time.

Hi yanpu.

You can try to use pardiso with iparm(60) is equal to 1. In such case if pardiso need to use more than 32Gb of RAM OOC algorithm will be used. Just set environment variable MKL_PARDISO_OOC_MAX_CORE_SIZE to 30000 to provide information about existence RAM to pardiso before PARDISO call.

With best regards,

Alexander Kalinkin

Leave a Comment

Please sign in to add a comment. Not a member? Join today