Hello, I am writting a hybrid openmp/MPI program which will end up calling SCLAPACK routines from threaded regions (to solve independent problems similtanously) , but am unable to use the BLACS send/recieves from a parallel region without a high (>90%) crash rate. I have successfully used the thread safe INTEL MPI library to do multiple threaded MPI send/recieves. Three type of error comes up for the same code.
- Stall in the 'parallel blacs' region
- segfault in libmkl_blacs ... + libpthread ( see snippet 1)
- MPI error (see snippet 2)
xxx@yyy:~/scalapack> ... PARALLEL BLACS : 1 forrtl: severe (174): SIGSEGV, segmentation fault occurred Image PC Routine Line Source libpthread.so.0 00002ADF44483C10 Unknown Unknown Unknown libmkl_blacs_inte 00002ADF4425C6BB Unknown Unknown Unknown
... PARALLEL BLACS : 1 Fatal error in MPI_Testall: Invalid MPI_Request, error stack: MPI_Testall(261): MPI_Testall(count=2, req_array=0x6107b0, flag=0x7fff4ce1cd80, status_array=0x60fe80) failed MPI_Testall(123): The supplied request in array element 1 was invalid (kind=15) APPLICATION TERMINATED WITH THE EXIT STRING: Hangup (signal 1)
The source code is attached, with the compiler line and the relevent parts of the PBS submission as a footer.
I am using the Intel Cluster Studio 2012, ifort 12.1.0, Intel MPI version 4.0, update 3 - i Am unsure of which MKL i have (came with cluster studio). I suspect this is a threading issue since the program works with OMP_NUM_THREADS=1.
Any help would be greatly appreciated!