Segmentation fault on shutdown in safer_scalable_free when using libtbbmalloc_proxy.so.2

Segmentation fault on shutdown in safer_scalable_free when using libtbbmalloc_proxy.so.2

Hi,

I am from Openet, we have a support account, but it is disabled at the moment.

We stared using tbbmalloc recently, but we had a problem with the process core dumping on shutdown fairly consistently. There are a few examples of stack traces, they all have various originating frames, but the top of the stack is consistent (see below). I would like to paste some full stack traces but would rather do so privately. I can provide serial number etc. Any help would be appeciated.

We are running a c++ application on RHEL6.3, on VM's.

Program terminated with signal 11, Segmentation fault.
#0  0x00000031c440f3cb in raise () from /lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6_4.2.x86_64 jdk-1.7.0_45-fcs.x86_64 libaio-0.3.107-10.el6.x86_64 libgcc-4.4.7-3.el6.x86_64 libstdc++-4.4.6-4.el6.x86_64 lksctp-tools-1.0.10-5.el6.x86_64 numactl-2.0.7-3.el6.x86_64 zlib-1.2.3-27.el6.x86_64
(gdb) bt
#0  0x00000031c440f3cb in raise () from /lib64/libpthread.so.0
#1  0x00007f41da9ac130 in skgesigOSCrash () from /u01/oracle/product/12.1.0.1/client/lib/libclntsh.so.12.1
#2  0x00007f41dac7566e in kpeDbgSignalHandler () from /u01/oracle/product/12.1.0.1/client/lib/libclntsh.so.12.1
#3  0x00007f41da9ac310 in skgesig_sigactionHandler () from /u01/oracle/product/12.1.0.1/client/lib/libclntsh.so.12.1
#4  <signal handler called>
#5  0x00007f41e98a5ee0 in safer_scalable_free (object=0x193d040, original_free=0x31c3c7b6d0 <free>) at ../../src/tbbmalloc/frontend.cpp:2574
#6  0x00007f41e98a1e00 in rml::internal::__TBB_internal_free (object=0x193d040) at ../../src/tbbmalloc/tbbmalloc.cpp:201
#7  0x00007f41ef49f196 in free (object=0x193d040) at ../../src/tbbmalloc/proxy.cpp:88
#8  0x00000031c3811219 in _dl_deallocate_tls () from /lib64/ld-linux-x86-64.so.2
#9  0x00000031c44065dd in __free_stacks () from /lib64/libpthread.so.0
#10 0x00000031c440776a in __free_tcb () from /lib64/libpthread.so.0
#11 0x00000031c4408074 in pthread_join () from /lib64/libpthread.so.0
10 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

hi Karl,

Do you use proxy via LD_PRELOAD, right?

--Vladimir

 

Yes:

 

if [ "$USING_TBB" = "yes" ]; then
        export LD_PRELOAD=$FUSIONWORKS_PROD/lib/libtbbmalloc_proxy.so.2
    fi

Just in case, are you using fresh version of libtbbmalloc? Can you reproduce it with debug version libtbbmalloc_proxy_debug.so.2?

we have fixed similar problem in TBB 4.1 update 4 about year ago. So it would be good to know which version you use.

--Vladimir

Version Info

strings">strings libtbb.so.2 | grep -i version
TBB_runtime_interface_version
_ZTVN3rml16versioned_objectE
_ZTIN3rml16versioned_objectE
_ZTSN3rml16versioned_objectE
__itt_api_version
ITT-API-Version 0.0.0 (20111111)
TBB: VERSION            4.1
TBB: INTERFACE VERSION  6101
TBB: BUILD_GCC          gcc version 4.1.0 (SUSE Linux)
TBB: BUILD_COMPILER     ICC: Intel(R) C Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 12.1.5.339 Build 20120612
TBB: BUILD_LD           GNU ld version 2.16.91.0.5 20051219 (SUSE Linux)
@TBB_VERSION
 Windows XP 64-bit Edition Version 2003 or newer should be used.
Conversion from text file %s to binary %s completed
N3rml16versioned_objectE
barryg@dub-eng-vm23>

Hello,

This is TBB 4.1 update 1. As I have written we have fixed something similar in TBB 4.1 update 4. You can check this with the latest update (TBB 4.2 update 5). For fast check you can take OSS release from https://www.threadingbuildingblocks.org/download#stable-releases site. Just copy libraries from tbb42_20140601oss/lib/intel64/gcc4.1 folder on top of your libraries. Re-compilation is not needed.

--Vladimir

Thanks very much Vladimir, I will try that out and get back to you within a couple of days to let you know the results.

Regards,

Karl.

Looks like this has resolved the issue we were seeing. We put the new version of the TBB libs into our test harness last night and could not reproduce the problem.

Thanks for your help.

Karl.

good to see this. Thank you, Karl.

--Vladimir

Leave a Comment

Please sign in to add a comment. Not a member? Join today