vtssrun seg fault

Accedi per lasciare commenti

Lun, 27/10/2008 - 11:25

Hi,

First of all thank you very much for the detailed problem description.

Unfortunately this problem is not something known for us.

The only thing I could say is that it is hard to imagine that application or libraries it depends on affect the vtssrun tool.

It is a different process and it uses the standard version of malloc if you did not hack libc.

Ifit is possible for you to create some test case that does not contain intellectual property/confidential informationof you corporation please provide it to us and we will see what can be done about it.

Thank you in advance.

K

Accedi per lasciare commenti

Konstantin Lupach (Intel)

Lun, 27/10/2008 - 11:28

To provide you information you can also use the channel described here by David Levinthal: http://software.intel.com/en-us/forums/showthread.php?t=61310

Accedi per lasciare commenti

levinth

Mar, 28/10/2008 - 09:12

Quoting - klf@yahoo-inc.com
We've been using vtss successfully for a while, and recently used it with a new version of our app. In this case, vtssrun seg faults (while the application continues running).

This is the backtrace form a core file (this is representative, though I've seen different backtraces from other cores):

#0 0x0000003dfaa723d4 in memcpy () from /lib64/tls/libc.so.6
(gdb) bt
#0 0x0000003dfaa723d4 in memcpy () from /lib64/tls/libc.so.6
#1 0x0000002a95be2d81 in std::string::assign (this=0x4820a980,
__s=0x2a96979028 , __n=262087)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/char_traits.h:270
#2 0x0000002a95bdee17 in std::basic_stringbuf, std::allocator >::overflow (
this=0x94cbd0, __c=99)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/basic_string.h:1465
#3 0x0000002a95be09df in std::basic_streambuf >::xsputn (this=0x94cbd0,
__s=0x959718 "clientHandlerThread", __n=19)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/char_traits.h:284
#4 0x0000002a95bd7b54 in std::ostream::write (this=0x94cf60, __s=0x959718 "clientHandlerThread", __n=19)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/streambuf:421
#5 0x0000002a95a3fc49 in DTLU_3_1::Log::start ()
from /export/crawlspace/ptu31_003_lin_intel64/bin/../lib/libdtlu_trace_assert.so
#6 0x0000002a95a4012e in _ZN8DTLU_3_15TraceC9EPNS_3LogERKSsS4_b ()
from /export/crawlspace/ptu31_003_lin_intel64/bin/../lib/libdtlu_trace_assert.so
#7 0x0000002a95a40370 in DTLU_3_1::Trace::Trace ()
from /export/crawlspace/ptu31_003_lin_intel64/bin/../lib/libdtlu_trace_assert.so
#8 0x0000000000417555 in main ()

One suspicion we have is a custom malloc library we are using in our app may conflict with vtssrun's allocations.

Other info:

Vtss: version 3.1 build 31

Processor: (2) 2GHz Clovertown (L5335)

OS: Rhel4u6 Linux 2.6.9-67.0.15.ELsmp

Compiler: gcc4.1.1

Binary: x86_64

if you are comfortable sending us a binary with symbols we can debug this...but there is little we can do without that.

If you submit an issue to https://premier.intel.com to the vtune product and point out that I asked you to do this and have the case assigned to me..we can use the secure file exchange to get the test case in house and resolve this...

thanks you

Dave L.

Accedi per lasciare commenti

levinth

Mar, 28/10/2008 - 09:14

Quoting - klf@yahoo-inc.com
We've been using vtss successfully for a while, and recently used it with a new version of our app. In this case, vtssrun seg faults (while the application continues running).

This is the backtrace form a core file (this is representative, though I've seen different backtraces from other cores):

#0 0x0000003dfaa723d4 in memcpy () from /lib64/tls/libc.so.6
(gdb) bt
#0 0x0000003dfaa723d4 in memcpy () from /lib64/tls/libc.so.6
#1 0x0000002a95be2d81 in std::string::assign (this=0x4820a980,
__s=0x2a96979028 , __n=262087)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/char_traits.h:270
#2 0x0000002a95bdee17 in std::basic_stringbuf, std::allocator >::overflow (
this=0x94cbd0, __c=99)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/basic_string.h:1465
#3 0x0000002a95be09df in std::basic_streambuf >::xsputn (this=0x94cbd0,
__s=0x959718 "clientHandlerThread", __n=19)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/char_traits.h:284
#4 0x0000002a95bd7b54 in std::ostream::write (this=0x94cf60, __s=0x959718 "clientHandlerThread", __n=19)
at /tmp/gcc/gcc-3.4.6/x86_64-unknown-linux-gnu/libstdc++-v3/include/streambuf:421
#5 0x0000002a95a3fc49 in DTLU_3_1::Log::start ()
from /export/crawlspace/ptu31_003_lin_intel64/bin/../lib/libdtlu_trace_assert.so
#6 0x0000002a95a4012e in _ZN8DTLU_3_15TraceC9EPNS_3LogERKSsS4_b ()
from /export/crawlspace/ptu31_003_lin_intel64/bin/../lib/libdtlu_trace_assert.so
#7 0x0000002a95a40370 in DTLU_3_1::Trace::Trace ()
from /export/crawlspace/ptu31_003_lin_intel64/bin/../lib/libdtlu_trace_assert.so
#8 0x0000000000417555 in main ()

One suspicion we have is a custom malloc library we are using in our app may conflict with vtssrun's allocations.

Other info:

Vtss: version 3.1 build 31

Processor: (2) 2GHz Clovertown (L5335)

OS: Rhel4u6 Linux 2.6.9-67.0.15.ELsmp

Compiler: gcc4.1.1

Binary: x86_64

also

Could you please set LD_DEBUG=all and LD_DEBUG_OUTPUT=/tmp/vtssrun and run vtssrun again? Then find the biggest file in /tmp/vtssrun* and send it to me.

dave L

Accedi per lasciare commenti

klf@yahoo-inc.com

Mar, 28/10/2008 - 10:36

Quoting - klupach
Hi,

First of all thank you very much for the detailed problem description.

Unfortunately this problem is not something known for us.

The only thing I could say is that it is hard to imagine that application or libraries it depends on affect the vtssrun tool.

It is a different process and it uses the standard version of malloc if you did not hack libc.

Ifit is possible for you to create some test case that does not contain intellectual property/confidential informationof you corporation please provide it to us and we will see what can be done about it.

Thank you in advance.

K

Thanks for your reply. Unfortunately I have not been able to reproduce this behavior with a simple test case, and we cannot send this application outside the company. Otoh we have been able to collect callgraphs with vtssrun by not using the custom malloc library and not using mlock. I have sent David the file he requested in his post.

Accedi per lasciare commenti

Vitaly Slobodskoy (Intel)

Ven, 31/10/2008 - 08:58

Hello!

I didn't find any criminal in LD_DEBUG output files - all symbols are bind into correct libraries. There is no runtime conflict in vtssrun process.

But I found that libc symbols (e.g. 'malloc') in libvtssagent.so are bind into libxmalloc.so library which is seems your custom malloc library (correct?). I don't know how this library works, but theoretically it may change behavior of vtssagent. It gives me another idea - something wrong happens in libvtssagent.so and as result listener thread in vtssrun crashes while receiving corrupted data.

To check that I'd like to ask you to provide stacks of crash for other threads in vtssrun process ("thread " command in gdb and then "bt" as usual, to get the list of threads use "info threads").

Thanks!

Accedere per lasciare un commento.