static linking of multi-threaded MKL, OMP: Error #18

static linking of multi-threaded MKL, OMP: Error #18

Greetings,

My application is crashing with this error:

OMP: Error #18: Setting environment variable "__KMP_REGISTERED_LIB_2119" failed:
OMP: Hint: Seems application required too much memory.

Linking command:

/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/ifort -static -Wl,-E -w -assume byterecl -DVAR_IFORT -g -traceback -static-libgcc -static-intel -O0 CMakeFiles/dirac.x.dir/src/main/main.F90.o -o dirac.x -i_dynamic -L/home/milias/work/qch/dirac_git/working_trunk/external/lib -L/home/milias/work/qch/dirac_git/working_trunk/external/gen1int/external/lib lib/libdirac.a lib/libxcfun.a -lstdc++ -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_lapack95_lp64.a -Wl,--end-group -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_core.a /opt/intel/mkl/lib/intel64/libmkl_intel_thread.a /usr/lib/libpthread.a /usr/lib/libm.a /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a -liomp5 -Wl,--end-group ../external/lib/libgen1int.a -ldecimal -lirc -Wl,-rpath,/home/milias/work/qch/dirac_git/working_trunk/external/lib:/home/milias/work/qch/dirac_git/working_trunk/external/gen1int/external/lib:

ifort (IFORT) 12.1.3 20120212 with MKL
on Linux grafix 2.6.32-28-generic #55-Ubuntu SMP Mon Jan 10 23:42:43 UTC 2011 x86_64 GNU/Linux

GNU debugger shows:

Program received signal SIGABRT, Aborted.
0x0000000002a9de8b in raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
42 ../nptl/sysdeps/unix/sysv/linux/pt-raise.c: No such file or directory.
in ../nptl/sysdeps/unix/sysv/linux/pt-raise.c
(gdb) where
#0 0x0000000002a9de8b in raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
#1 0x0000000002c91020 in abort ()
#2 0x00000000021bf6e1 in __kmp_abort_process ()
#3 0x00000000021a6726 in __kmp_msg ()
#4 0x0000000002203fa9 in __kmp_env_set ()
#5 0x00000000021be660 in __kmp_middle_initialize ()
#6 0x00000000021a5a7e in omp_get_num_procs ()
#7 0x0000000002186243 in MKL_get_N_Cores ()
#8 0x0000000002187431 in mkl_serv_mkl_domain_get_max_threads ()
#9 0x00000000021859b4 in mkl_blas_dcopy ()
#10 0x0000000001f7fafe in dcopy_ ()
#11 0x0000000000a6d756 in pr1dr1 (soint=..., work=..., lwork=63982623, npoint=1, labint=..., inttyp=1, intrep=...,
noptyp=1, nbast=169, nelmnt=14365, anti=.FALSE., iorder=0, doatom=..., weight=..., abscis=..., triang=4294967295,
natom=1, intadr=..., nshint=1, square=.FALSE., doint=..., iprint=1, .tmp.LABINT.len_V$fc=8)
at /home/milias/work/qch/dirac_git/working_trunk/src/abacus/her1int.F:706

Intel debugger output:

Info: Optimized variables show as when no location is allocated.
raise (sig=) at /build/buildd/eglibc-2.11.1/nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
No source file named /build/buildd/eglibc-2.11.1/nptl/sysdeps/unix/sysv/linux/pt-raise.c.
(idb) where
#0 0x0000000002a9de8b in raise (sig=) at /build/buildd/eglibc-2.11.1/nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
#1 0x0000000002c91020 in atoi () in /tmp/milias/DIRAC_ci_F_2373/dirac.x
#2 0x00000000021bf6e1 in __kmp_abort_process () in /tmp/milias/DIRAC_ci_F_2373/dirac.x
#3 0x00000000021a6726 in __kmp_msg () in /tmp/milias/DIRAC_ci_F_2373/dirac.x
#4 0x0000000002203fa9 in __kmp_env_set () in /tmp/milias/DIRAC_ci_F_2373/dirac.x
#5 0x00000000021be660 in __kmp_middle_initialize () in /tmp/milias/DIRAC_ci_F_2373/dirac.x
#6 0x00000000021a5a7e in omp_get_num_procs () in /tmp/milias/DIRAC_ci_F_2373/dirac.x
#7 0x0000000002186243 in MKL_get_N_Cores () in /tmp/milias/DIRAC_ci_F_2373/dirac.x

set environment variables:
export MKL_NUM_THREADS="1"
export MKL_DYNAMIC="FALSE"
export OMP_NUM_THREADS="1"

Any help please ?

Miro

6 Beiträge / 0 neu
Letzter Beitrag
Nähere Informationen zur Compiler-Optimierung finden Sie in unserem Optimierungshinweis.

Hi,

Statically linking in general and especially with pthread library is not a good idea. Please try using --whole-archive linker option.

Why do you need fully-static executable? It may have serious problems when run on different machines...

Thanks,
-- Victor

Dear Victor,

yes, I added the "--whole-archive" option to the linking:

milias@grafix.fpv.umb.sk:~/work/qch/dirac_git/working_trunk/build_intel_static/./opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/ifort -static -Wl,-E -w -assume byterecl -DVAR_IFORT -g -traceback -static-libgcc -static-intel -i8 -O0 CMakeFiles/dirac.x.dir/src/main/main.F90.o -o dirac.x -i_dynamic lib/libdirac.a lib/libxcfun.a -lstdc++ -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_lapack95_ilp64.a -Wl,--end-group -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_core.a /opt/intel/mkl/lib/intel64/libmkl_intel_thread.a /usr/lib/libpthread.a /usr/lib/libm.a /opt/intel/mkl/lib/intel64/libmkl_intel_ilp64.a -liomp5 -Wl,--end-group -ldecimal -lirc --whole-archive

but still the same error...

Also I couldn't find this flag among ifort's options.

I need static executable to be able to run on many grid server, which are not equipped with Intel compilers/libraries.

Best, Miro

You have no hope if you can't depend on your servers to have a compatible dynamic pthreads library installed (and dynamic pthreads is more likely to be supported than static). That library comes with the OS; it is not an Intel library. Since you are linking against libiomp5.so, you will need to distribute at least that library, and that's another reason why you can't link static pthreads library anywhere in your build.
Your grouping of libraries in --start-group --end-group directives looks fishy. Please follow the model in the link advisor tool posted at the top of this forum and in your MKL installation.
-static-intel (not -static) option is frequently employed for the purpose of not requiring distribution of all the Intel shared libraries.

Quoting miro_ilias...yes, I added the "--whole-archive" option to the linking:
but still the same error...

Also I couldn't find this flag among ifort's options.

Please use -Wl,--whole-archive to pass this option to linker

Thanks,
-- Victor

Dear Victor,

unfortunately, adding " -Wl, --whole-archive" to the end did not solve the problem:

milias@grafix.fpv.umb.sk:~/work/qch/dirac_git/working_trunk/build/./opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/ifort -static -Wl,--no-export-dynamic -Wl,-E -w -assume byterecl -DVAR_IFORT -g -traceback -static-libgcc -static-intel -O0 CMakeFiles/dirac.x.dir/src/main/main.F90.o -o dirac.x -i_dynamic lib/libdirac.a lib/libxcfun.a -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_lapack95_lp64.a /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a -openmp -Wl,--end-group -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_core.a /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/mkl/lib/intel64/libmkl_intel_thread.a /usr/lib/libpthread.a /usr/lib/libm.a -openmp -Wl,--end-group -Wl,--no-export-dynamic /opt/intel/mkl/lib/intel64/libmkl_core.a /opt/intel/mkl/lib/intel64/libmkl_intel_thread.a /usr/lib/libpthread.a /usr/lib/libm.a -ldecimal -lirc -ldecimal -lstdc++ -lirc -Wl, --whole-archive

Any help, please ?

Kommentar hinterlassen

Bitte anmelden, um einen Kommentar hinzuzufügen. Sie sind noch nicht Mitglied? Jetzt teilnehmen