Indexing an array of size between 2^31 and 2^32-1 with LP64?

Indexing an array of size between 2^31 and 2^32-1 with LP64?

Ritratto di Yu S.

I am working with spblas. My matrix dimension is about 300k, nnz is between 2^31 and 2^32-1. To keep the memory consumption as small as possible. I would like to use 32bit unsigned integer to index my element. Is it possible to do so with LP64 by defining MKL_INT as uint32_t? I tried it, but my program crashed with a segmentation fault when calling mkl_scsrmv.

6 post / 0 new
Ultimo contenuto
Per informazioni complete sulle ottimizzazioni del compilatore, consultare l'Avviso sull'ottimizzazione
Ritratto di Henrik Arlinghaus

Changing the definition on your end will not change what MKL is expecting with LP64 defined.

Ritratto di Tim Prince

Quote:

Henrik Arlinghaus wrote:

Changing the definition on your end will not change what MKL is expecting with LP64 defined.

I agree with this.  Simply changing your include file doesn't modify the libraries or change the basic mechanism, which doesn't support unsigned indexing in a useful manner.

Ritratto di Sergey Kostrov

>>...To keep the memory consumption as small as possible. I would like to use 32bit unsigned integer to index
>>my element...

Your input matrix is already big and I don't think that you will be able to save a significant amount of memory. Modern operating systems use Virtual Memory ( VM ) and why wouldn't you increase VM settings ( I mean Initial and Maximum sizes ) instead? It is Not clear on what platform and hardware you're going to do processing and please provide details if it is possible.

Ritratto di Yu S.

Quote:

TimP (Intel) wrote:

Quote:

Henrik Arlinghauswrote:

Changing the definition on your end will not change what MKL is expecting with LP64 defined.

I agree with this.  Simply changing your include file doesn't modify the libraries or change the basic mechanism, which doesn't support unsigned indexing in a useful manner.

Thanks for the reply! So is there a way to get around this without making the indexing array to be 64bit because it is going to double the size? I tried compiling my program with ilp64 and define MKL_INT as uint32_t, but it did work either.

Ritratto di Yu S.

Quote:

Sergey Kostrov wrote:

>>...To keep the memory consumption as small as possible. I would like to use 32bit unsigned integer to index
>>my element...

Your input matrix is already big and I don't think that you will be able to save a significant amount of memory. Modern operating systems use Virtual Memory ( VM ) and why wouldn't you increase VM settings ( I mean Initial and Maximum sizes ) instead? It is Not clear on what platform and hardware you're going to do processing and please provide details if it is possible.

Sorry for the confusion. I am working on a Linux 64bit machine. If I make the array element a type of 64bit integer, it will double the memory consumption and exceed my physical memory allowance.

Accedere per lasciare un commento.