I am trying to get my head around a nasty segmentation fault. It occurs in a third party library, only after our program has finished. Running the software in gdb gives the following stacktrace:
#0 0x00007ffff01fbf40 in pthread_mutex_destroy () from /lib64/libpthread.so.0
#1 0x0000000000f30041 in kill_resource ()
#2 0x0000000000f2ffa1 in reentrancy_cleanup ()
#3 0x0000000000f2feae in for__reentrancy_cleanup ()
#4 0x00007ffff15ae8f5 in for__exit_handler () from /u/username/wrk/software/petsc-3.9.3_intel16/lib/libpetsc.so.3.9
#5 0x00000000004135a3 in main ()
#6 0x00007fffefe7dd1d in __libc_start_main () from /lib64/libc.so.6
#7 0x00000000004134a9 in _start ()
We are compiling with ifort 18.104.22.168. However, when compiling the same software (our application, but also libpetsc, the third party library) with ifort 22.214.171.124, we didn't experience this problem. Moreover, when checking the symbols in the library with nm, the older version does not even show routines such as for__reentrancy_cleanup() or kill_resource().
We have been trying very hard to isolate the error and prepare a minimal example, but so far this wasn't successful. The hints that we have are
- the problem did not occur when building with version 14.0
- it seems to be related to multi-threading/reentrancy
- In particular, dropping the compiler option '-fopenmp' gets rid of the segmentation fault, but ideally we would like to keep this
Does anyone have a clue what is going on here? Has some default changed between 126.96.36.199 to 188.8.131.52 that could explain the above behavior? What generates the 'for__reentrancy_cleanup()'?
It would be great if someone can help out here and maybe give some pointers. Also, please let me know if you would need more information.