Error message before machine crashes

Error message before machine crashes

Ritratto di jburri
jburri
Gio, 10/01/2013 - 15:28

We have been having problems with our machine crashing with no clear casue.  However, right before the machine crashes you see the following messages:

Jan  9 15:57:34 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:57:44 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:57:54 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:58:04 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:58:14 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:58:24 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:58:34 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:58:44 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:58:54 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:59:04 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:59:14 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:59:24 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:59:34 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:59:44 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 15:59:54 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 16:00:04 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 16:00:14 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 16:00:24 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 16:00:34 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 16:00:44 tacopie intelremotemond[12460]: Consumer: flush data to db
Jan  9 16:00:44 tacopie intelremotemond[12460]: Consumer: timeout, exiting...
Jan  9 16:00:44 tacopie intelremotemond[12601]: SendDataToBackend: path to agent is /opt/intel/ism/bin/intel64/intelremotemonagent
Jan  9 16:00:44 tacopie intelremotemond[12601]: SendDataToBackend: start agent
Jan  9 16:00:44 tacopie intelremotemond[12460]: Main: try to delete semaphore, status 0
Jan  9 16:00:44 tacopie intelremotemond[12460]: Main: daemon stopped, force is 0

I have not had much luck understanding what these message mean.  Does anyone have any insight?

I should add that you can compile and run the same code dozens of times without issue.  We are using icpc (ICC) 12.1.0 20111011 and CentOS release 6.3.

-Jeremy

RSS In cima
11 post / 0 new
Ultimo contenuto
Per informazioni complete sulle ottimizzazioni del compilatore, consultare l'Avviso sull'ottimizzazione
Ritratto di Sergey Kostrov
Sergey Kostrov
Sab, 12/01/2013 - 21:04

Hi Jeremy,

It is not clear what your computer is doing just before it crashes. Do you compile some C/C++ project with Intel C/C++ compiler? Please specify.

Also, Looking at these four log-messages:

>>...
>>Jan 9 16:00:44 tacopie intelremotemond[12601]: SendDataToBackend: path to agent is /opt/intel/ism/bin/intel64/intelremotemonagent
>>Jan 9 16:00:44 tacopie intelremotemond[12601]: SendDataToBackend: start agent
>>Jan 9 16:00:44 tacopie intelremotemond[12460]: Main: try to delete semaphore, status 0
>>Jan 9 16:00:44 tacopie intelremotemond[12460]: Main: daemon stopped, force is 0
>>...

I see that some processing was done in Intel Remote Monitor Agent.

In cima
Ritratto di jburri
jburri
Mer, 16/01/2013 - 08:27

Hi Sergey,

The system is currently being used for the development of a couple of C++ program (one using only CPUs and another that is using CPUs and a GPU).  We are using icpc (ICC) 12.1.0 20111011.  Do you know anything about "Intel Remote Monitor Agent"? An initial goolge search didn't turn up anything useful for me.  We recently installed and started using VTune, but there does not seem to be a correlation between the running of VTune and the crashing of the system.  A lot of times the machine will become unresponsive in the middle of the night when no one is using.  

It has been a real head scratcher for us.

-Jeremy

In cima
Ritratto di Sergey Kostrov
Sergey Kostrov
Mer, 16/01/2013 - 20:20

>>... We recently installed and started using VTune, but there does not seem to be a correlation between the
>>running of VTune and the crashing of the system.

Did VTune work when the system crashed? If Yes, I would try to uninstall VTune in order to verify that it is not involved in these craches.

In cima
Ritratto di Sergey Kostrov
Sergey Kostrov
Mer, 16/01/2013 - 20:25

And one more thing:

>>...SendDataToBackend: path to agent is /opt/intel/ism/bin/intel64/intelremotemonagent

I wouldn't say that there is no relation with some Intel software.

In cima
Ritratto di iliyapolak
iliyapolak
Mer, 16/01/2013 - 21:23

@Jeremy

Can you enumerate all active daemons on your compute before the crashr?

>>>Jan 9 16:00:44 tacopie intelremotemond[12460]: Main: try to delete semaphore, status 0>>>

Could that be somehow relate to synchronization issue?

In cima
Ritratto di Brandon Hewitt (Intel)
Brandon Hewitt (Intel)
Ven, 18/01/2013 - 11:48

Hi jburri,

This intelremotemonagent service is part of the Intel(R) Software Improvement program functionality.  I've had one of the developers of this take a look and he doesn't see anything unusual about the logging messages your provide.  We're working on what more information could help us determine if there is anything unusual here - there is a log (intelremotemonagent.<uid>.log) but it's in /tmp which would get wiped after reboot.  I'll let you know as soon as we have anything further.

Brandon Hewitt Technical Consulting Engineer Tools Knowledge Base: http://software.intel.com/en-us/articles/tools Software Product support info: http://www.intel.com/software/support
In cima
Ritratto di Brandon Hewitt (Intel)
Brandon Hewitt (Intel)
Lun, 21/01/2013 - 11:04

It sounds like you can configure /tmp in your OS to not have all files in it deleted after reboot.  If you could do that, and if you get another crash, send us the log file, that would be helpful.

Brandon Hewitt Technical Consulting Engineer Tools Knowledge Base: http://software.intel.com/en-us/articles/tools Software Product support info: http://www.intel.com/software/support
In cima
Ritratto di Sergey Kostrov
Sergey Kostrov
Lun, 21/01/2013 - 16:58

>>...This intelremotemonagent service is part of the Intel(R) Software Improvement program functionality...

Brandon, Here is a question:

How a user can disable or turn it off to verify that all the rest software is not creating that problem?`

In cima
Ritratto di a.kaliazin
a.kaliazin
Lun, 06/05/2013 - 14:51

Hi all

It looks like the Sergey's last plea was left unanswered, so I'd like to raise this question again - how can I turn off completely this annoying service which we never have asked for? For me it only fills up the syslog without improving anything. VTune keeps crashing, then suggests to send a report and crashes at that as well.

In cima
Ritratto di mark-sabahi (Intel)
mark-sabahi (Intel)
Gio, 09/05/2013 - 13:20

You can disable the service by running ISM in /opt/intel/ism or ${HOME}/intel/ism folder, go to settings, click "Custoemr Feedback Options" button and choose "Manual launch only".  The executable name is "ism".

 --mark

In cima

Accedere per lasciare un commento.