MPI_Init crash after setting I_MPI_STATS

MPI_Init crash after setting I_MPI_STATS

After setting I_MPI_STATS=4, Intel MPI 4.1.3 on Linux crashes at MPI_Init

*** glibc detected ***  free(): invalid pointer: 0x0000000002933384 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3193e7164f]
/lib64/libc.so.6(cfree+0x4b)[0x3193e7587b]
[0x231e318]
MPIR_Init_thread+0x62d)[0x231f7bd]
(MPI_Init+0x218)[0x2313598]

 

 

 

10 Beiträge / 0 neu
Letzter Beitrag
Nähere Informationen zur Compiler-Optimierung finden Sie in unserem Optimierungshinweis.

Interestingly, I found that by setting I_MPI_STATS_FILE the crash went away...

Very odd.  Are you doing anything with stdout or stderr?

Nothing special with stdout/stderr. The code being executed is your typical  FORTRAN executable.

By setting I_MPI_STATS_FILE everything works as expected.

What do you have I_MPI_STATS_BUCKETS set to?

It was not set. But this crash is happening at MPI_Init

What I mean is, no statistics have been gathered , so I do not see how any other I_MPI_STATS_XXXX settings would affect the issue.

If I_MPI_STATS_BUCKETS is not set, I_MPI_STATS=4 shouldn't give anything.  I'm not able to reproduce the problem, is it only with a specific code, or with any code?

That is not the case, I only set I_MPI_STATS and I_MPI_STATS_FILE , and I get meaningful information in the file as expected.

I don't have any other MPI software to test with, so cannot comment.

Here is the I_MPI_XXX dump.

[0] MPI startup(): I_MPI_DEBUG=100
[0] MPI startup(): I_MPI_FABRICS=shm:tcp
[0] MPI startup(): I_MPI_INFO_BRAND=Intel(R) Xeon(R)
[0] MPI startup(): I_MPI_INFO_CACHE1=0,1,2,3
[0] MPI startup(): I_MPI_INFO_CACHE2=0,1,2,3
[0] MPI startup(): I_MPI_INFO_CACHE3=0,0,0,0
[0] MPI startup(): I_MPI_INFO_CACHES=3
[0] MPI startup(): I_MPI_INFO_CACHE_SHARE=2,2,16
[0] MPI startup(): I_MPI_INFO_CACHE_SIZE=32768,262144,6291456
[0] MPI startup(): I_MPI_INFO_CORE=0,1,2,3
[0] MPI startup(): I_MPI_INFO_C_NAME=Wolfdale
[0] MPI startup(): I_MPI_INFO_DESC=1342208505
[0] MPI startup(): I_MPI_INFO_FLGB=0
[0] MPI startup(): I_MPI_INFO_FLGC=398124031
[0] MPI startup(): I_MPI_INFO_FLGD=-1075053569
[0] MPI startup(): I_MPI_INFO_LCPU=4
[0] MPI startup(): I_MPI_INFO_MODE=263
[0] MPI startup(): I_MPI_INFO_PACK=0,0,0,0
[0] MPI startup(): I_MPI_INFO_SERIAL=E31225
[0] MPI startup(): I_MPI_INFO_SIGN=132775
[0] MPI startup(): I_MPI_INFO_STATE=0
[0] MPI startup(): I_MPI_INFO_THREAD=0,0,0,0
[0] MPI startup(): I_MPI_INFO_VEND=1
[0] MPI startup(): I_MPI_PIN_INFO=x0,1,2,3
[0] MPI startup(): I_MPI_PIN_MAPPING=1:0 0
[0] MPI startup(): I_MPI_STATS=4
[0] MPI startup(): I_MPI_STATS_FILE=/home/andrewc/mpi_stats.txt

 

Can you send me the code?  Maybe there is something specific to the code causing the problem.

Kommentar hinterlassen

Bitte anmelden, um einen Kommentar hinzuzufügen. Sie sind noch nicht Mitglied? Jetzt teilnehmen