• 2019 Update 7
  • 03/31/2020
Contents

Error Message: Bad Termination

Intel® MPI Library Developer Guide for Linux* OS
Note
The values in the tables below may not reflect the exact node or MPI process where a failure can occur.

Case 1

Error Message

=================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = RANK 1 PID 27494 RUNNING AT node1 = KILLED BY SIGNAL: 11 (Segmentation fault) ===================================================================================
or:
=================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = RANK 1 PID 27494 RUNNING AT node1 = KILLED BY SIGNAL: 8 (Floating point exception) ===================================================================================

Cause

One of MPI processes is terminated by a signal (for example,
Segmentation fault
or
Floating point exception
) on the
node01
.

Solution

Find the reason of the MPI process termination. It can be the out-of-memory issue in case of
Segmentation fault
or division by zero in case of
Floating point exception
.

Case 2

Error Message

================================================================================ = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = RANK 1 PID 20066 RUNNING AT node01 = KILLED BY SIGNAL: 9 (Killed) ================================================================================

Cause

One of MPI processes is terminated by a signal (for example,
SIGTERM
or
SIGKILL
) on the
node01
due to:
  • the host reboot;
  • an unexpected signal received;
  • out-of-memory manager (OOM) errors;
  • killing by the process manager (if another process was terminated before the current process);
  • job termination by the Job Scheduler (PBS Pro*,  SLURM*) in case of resources limitation (for example, walltime or cputime limitation).

Solution

  1. Check the system log files.
  2. Try to find the reason of the MPI process termination and fix the issue.

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804