zero CPU usage with vtune on Xeon Platinum 8160

zero CPU usage with vtune on Xeon Platinum 8160

Hi,

I'm trying to analyze the performance of a hybrid MPI & OpenMP code using intel vtune. I'm compiling my code using intel/18.0.2 & impi/18.0.2 & vtune/18.0.2 The OS is rhel 3.10.0-693.17.1.el7.x86_64 and CPU's are Intel Xeon Platinum 8160 (Skylake). I compile my code using "mpicc -xCORE-AVX512 -qopenmp mycode.c -g -O3" (I tried -O0 but didn't help), and I submit the code using "amplxe-cl -collect hpc-performance -r r000hpc ./a.out"

The code doesn't run at all and just jumps out (without vtune it runs without any problem). I get the following error message:

amplxe: Peak bandwidth measurement started.
amplxe: Peak bandwidth measurement finished.
amplxe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: amplxe-cl -r /work/vtune/r000hpc -command stop.
amplxe: Collection stopped.
amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 19 % Resolving information for `libittnotify_collecto
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libnss_files-2.17.so'.
amplxe: Executing actions 19 % Resolving information for `libpsm2.so.2.1'      
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/ld-2.17.so'.
amplxe: Warning: Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.2.0.551022/lib64/runtime/libittnotify_collector.so'.
amplxe: Executing actions 19 % Resolving information for `libc-2.17.so'        
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libpsm2.so.2.1'.
amplxe: Warning: Cannot locate file `igb.ko'.
amplxe: Executing actions 19 % Resolving information for `igb'                 
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libc-2.17.so'.
amplxe: Executing actions 21 % Resolving information for `vmlinux'             
amplxe: Warning: Cannot locate file `dm_mod.ko'.
amplxe: Executing actions 21 % Resolving information for `dm_mod'              
amplxe: Warning: Cannot locate file `sep4_1.ko'.
amplxe: Executing actions 22 % Resolving information for `sep4_1'              
amplxe: Warning: Cannot locate debugging information for the Linux kernel. Source-level analysis will not be possible. Function-level analysis will be limited to kernel symbol tables. See the Enabling Linux Kernel Analysis topic in the product online help for instructions.
amplxe: Executing actions 100 % done

and when I use "amplxe-cl -report summary -r r000hpc -report-knob show-issues=false" I get a zero usage result like this:

amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 75 % Generating a report                             Elapsed Time: 0.355s
SP GFLOPS: 0.000
Effective Physical Core Utilization: 0.3% (0.157 out of 48)
    Effective Logical Core Utilization: 0.2% (0.209 out of 96)
Memory Bound: 1.2% of Pipeline Slots
    Cache Bound: 9.9% of Clockticks
    DRAM Bound: 0.0% of Clockticks
        DRAM Bandwidth Bound: 0.0% of Elapsed Time
    NUMA: % of Remote Accesses: 0.0%
FPU Utilization: 0.0%
    SP FLOPs per Cycle: 0.000 Out of 64
    Vector Capacity Usage: 0.0%
    FP Instruction Mix
        % of Packed FP Instr.: 0.0%
            % of 128-bit: 0.0%
            % of 256-bit: 0.0%
            % of 512-bit: 0.0%
        % of Scalar FP Instr.: 0.0%
    FP Arith/Mem Rd Instr. Ratio: 0.000
    FP Arith/Mem Wr Instr. Ratio: 0.000
Collection and Platform Info
    Application Command Line: ./a.out
    User Name: ######## (I erased it)
    Operating System: 3.10.0-693.17.1.el7.x86_64 NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"  CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"
    Computer Name: ######### (I erased it)
    Result Size: 6 MB
    Collection start time: 18:36:37 09/07/2018 UTC
    Collection stop time: 18:36:38 09/07/2018 UTC
    Collector Type: Event-based sampling driver
    CPU
        Name: Intel(R) Xeon(R) Processor code named Skylake
        Frequency: 2.095 GHz
        Logical CPU Count: 96
amplxe: Executing actions 100 % done

Do you know what I am doing wrong?

Thank you very much!

AR

4 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hi,

please send the whole command line you used to run the collection. Also it will be helpful if you can set an absolute path to your application, repeat your experiment and upload here a failed result.

BR,

Vladimir

Hi Vladimir,

I compile the code using "mpicc -xCORE-AVX512 -qopenmp mycode.c -g -O3"

I submit mycode using a batch script, with this line inside the script: "ibrun amplxe-cl -collect hpc-performance -r /work/vtune/r000hpc ./a.out"

and I run vtune afterwards, using "amplxe-cl -report summary -r /work/vtune/r000hpc/ -report-knob show-issues=false"

The result looks like:

amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 75 % Generating a report                             Elapsed Time: 0.287s
SP GFLOPS: 0.000
Effective Physical Core Utilization: 0.3% (0.129 out of 48)
    Effective Logical Core Utilization: 0.3% (0.259 out of 96)
Memory Bound: 0.0% of Pipeline Slots
    Cache Bound
    DRAM Bound
        DRAM Bandwidth Bound: 0.0% of Elapsed Time
    NUMA: % of Remote Accesses: 0.0%
FPU Utilization: 0.0%
    SP FLOPs per Cycle: 0.000 Out of 64
    Vector Capacity Usage: 0.0%
    FP Instruction Mix
        % of Packed FP Instr.: 0.0%
            % of 128-bit: 0.0%
            % of 256-bit: 0.0%
            % of 512-bit: 0.0%
        % of Scalar FP Instr.: 0.0%
    FP Arith/Mem Rd Instr. Ratio: 0.000
    FP Arith/Mem Wr Instr. Ratio: 0.000
Collection and Platform Info
    Application Command Line: ./a.out
    User Name: ########(I erased this)
    Operating System: 3.10.0-693.17.1.el7.x86_64 NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"  CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"
    Computer Name: ##########(I erased this)
    Result Size: 5 MB
    Collection start time: 16:13:33 10/07/2018 UTC
    Collection stop time: 16:13:33 10/07/2018 UTC
    Collector Type: Event-based sampling driver
    CPU
        Name: Intel(R) Xeon(R) Processor code named Skylake
        Frequency: 2.095 GHz
        Logical CPU Count: 96

 

 

Here is the error file:

Currently Loaded Modules:
  1) git/2.9.0       3) xalt/2.0.7   5) intel/18.0.2      7) impi/18.0.2
  2) autotools/1.1   4) ####(I erased this)         6) libfabric/1.6.1   8) vtune/18.0.2

Inactive Modules:
  1) python

amplxe: Analyzing data in the node-wide mode. The hostname (######) will be added to the result path/name.
amplxe: Peak bandwidth measurement started.
amplxe: Peak bandwidth measurement finished.
amplxe: Collection started.
amplxe: Collection stopped.
amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 19 % Resolving information for `libc-2.17.so'        
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libnss_files-2.17.so'.
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/ld-2.17.so'.
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libpsm2.so.2.1'.
amplxe: Warning: Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.2.0.551022/lib64/runtime/libittnotify_collector.so'.
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libc-2.17.so'.
amplxe: Warning: Cannot locate file `dm_mod.ko'.
amplxe: Executing actions 20 % Resolving information for `librdmacm.so.1.0.13'
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/librdmacm.so.1.0.13'.
amplxe: Executing actions 20 % Resolving information for `libmpi.so.12.0'      
amplxe: Warning: Cannot locate file `hfi1.ko'.
amplxe: Executing actions 21 % Resolving information for `libiomp5.so'         
amplxe: Warning: Cannot locate file `xfs.ko'.
amplxe: Executing actions 22 % Resolving information for `xfs'                 
amplxe: Warning: Cannot locate debugging information for the Linux kernel. Source-level analysis will not be possible. Function-level analysis will be limited to kernel symbol tables. See the Enabling Linux Kernel Analysis topic in the product online help for instructions.
amplxe: Executing actions 100 % done

 

What are these warnings about?

 

Hi AR,

This is resolving warnings that means that a collector gathered some info but the tool cannot find symbol information for a particular module to attribute info to some function. So as you see these warnings and collection has ended I can make a conclusion that VTune has launched something that ended very quickly.

Currently I cannot say why it is so.  I think that I can say more based on the failed result, under the failed result I mean the r000hpc or any other result collected by VTune in this scenario. And as I said above it will be helpful if you change your script to use a full absolute path to your application.

But if it isn't possible we can try few additional checks to localize the problem. Please try the following:

1. try to collect hotspots and advanced-hotspots results

2. try to analyze another existing standard binary, for example ls or printenv. It will be something like ibrun amplxe-cl -collect hpc-performance -r /work/vtune/r001hpc  printenv

3. try to collect something without ibrun

BR,

Vladimir

 

 

Leave a Comment

Please sign in to add a comment. Not a member? Join today