Intel® VTune™ Profiler User Guide

User Guide

Contents

GPU Offload Analysis

Explore code execution on various CPU and GPU cores on your platform, correlate CPU and GPU activity, and identify whether your application is GPU or CPU bound.
Run the GPU Offload analysis for applications that use a Graphics Processing Unit (GPU) for rendering, video processing, and computations with explicit support of DPC++, Intel® Media SDK and OpenCL™ software technology.
The tool infrastructure automatically aligns clocks across all cores in the entire system so that you can analyze some CPU-based workloads together with GPU-based workloads within a unified time domain.
This analysis enables you to:
  • Identify how effectively your application uses DPC++ or OpenCL kernels and explore them further with GPU Compute/Media Hotspots analysis
  • Analyze execution of Intel Media SDK tasks over time (for Linux targets only)
  • Explore GPU usage and analyze a software queue for GPU engines at each moment of time
For the GPU Offload analysis, Intel® VTune™ Profiler instruments your code executing both on CPU and GPU. Depending on your configuration settings, VTune Profiler provides performance metrics that serve as insights on the efficiency of GPU hardware use as well as next steps in your analysis.

Configure and Run Analysis

By default, the GPU Offload analysis enables the
GPU Utilization
 option to explore GPU busyness over time and understand whether your application is CPU or GPU bound. As a result of such an analysis configuration, if you explore the Timeline view in the
Graphics
 window and see that the GPU is busy most of the time and having small idle gaps between busy intervals and the GPU software queue is rarely decreased to zero, you can conclude that your application is GPU bound. If the gaps between busy intervals are big and the CPU is busy during these gaps, your application is CPU bound. But such obvious situations are rare and you need a detailed analysis to understand all dependencies. For example, an application may be mistakenly considered GPU bound when GPU engines usage is serialized (for example, when GPU engines responsible for video processing and for rendering are loaded in turns). In this case, an ineffective scheduling on the GPU results from the application code running on the CPU.
On Windows systems, to monitor general GPU usage over time, run
VTune
Profiler
 as an Administrator.
To configure and run the GPU Offload analysis:
Prerequisites:
  1. Click the (standalone GUI)/ (Visual Studio IDE)
    Configure Analysis
     toolbar button to open the
    Configure Analysis
     window .
  2. Click anywhere in the title bar of the
    HOW
     pane. Open the Analysis Tree and select
    GPU Offload (Preview)
     analysis from the
    Accelerators
     group.
    The GPU Offload analysis is pre-configured to collect GPU usage data and collect Processor Graphics hardware events (Compute Basic preset).
  3. (Optional)
     If you are employing multiple Intel GPUs,
    VTune
    Profiler
     identifies all of these adapters in the
    Target GPU
     pulldown menu. Use this menu to specify the device you want to profile. If you do not select a GPU,
    VTune
    Profiler
     selects the most recent device family in the list by default.
    The
    Target GPU
     pulldown menu displays only when
    VTune
    Profiler
     detects multiple GPUs running on the system. The menu then displays the name of each GPU with the bus/device/function (BDF) of its adapter. You can also find this information on your Windows (see Task Manager) or Linux (run
    lspci
    ) system.
  4. Configure these GPU analysis options:
    • Use the
      Trace GPU programming APIs
       option to analyze DPC++, OpenCL™ and Intel Media SDK programs running on Intel Processor Graphics. This option may affect the performance of your application on the CPU side.
    • Use the
      Analyze memory bandwidth
       option to collect the data required to compute memory bandwidth. This type of analysis requires Intel sampling drivers to be installed.
    • Use the
      Collect CPU-side stacks
       option to analyze call stacks executed on the CPU and identify critical paths.
    • Use the
      Show GPU performance insights
       to get metrics (based on the analysis of Processor Graphics events) that help you estimate the efficiency of hardware usage and learn next steps. The following Insights metrics are collected:
      • The
        EU Array
         metric shows the breakdown of GPU core array cycles, where:
        • Active
          : The normalized sum of all cycles on all cores spent actively executing instructions. Formula:
        • Stalled
          : The normalized sum of all cycles on all cores spent stalled. At least one thread is loaded, but the core is stalled for some reason. Formula:
        • Idle
          : The normalized sum of all cycles on all cores when no threads were scheduled on a core. Formula:
      • The
        EU Threads Occupancy
         metric shows the normalized sum of all cycles on all cores and thread slots when a slot has a thread scheduled.
      • The
        Computing Threads Started
         metric shows the number of threads started across all EUs for compute work.
  5. Click
    Start
     to run the analysis.
To run the GPU Offload analysis from the command line, enter:
$
vtune
 -collect gpu-offload [-knob <
knob_name
=
knob_option
>] -- <
target
> [
target_options
]
To generate the command line for any analysis configuration, use the
Command Line...
 button at the bottom of the interface.
Once the GPU Offload Analysis completes data collection, the
Summary
 window displays several metrics that describe:
  • GPU usage
  • GPU idle time
  • The most active computing tasks that ran on the GPU
Summary window of GPU Offload Analysis You also see
Recommendations
 and guidance for next steps.

Analyze Data Transfer Between Host and Device

To understand the efficiency of data transfer between the CPU host and GPU device, see metrics in the Summary and Graphics windows.
The Summary window displays the total time spent on computing tasks as well as the execution time per task. The difference indicates the amount of time spent on data transfers between host and device. If the execution time is lower than the data transfer time, this indicates that your offload schema could benefit from optimization.
In the Summary window, look for offload cost metrics including
Host-to-Device Transfer
 and
Device-to-Host Transfer
. These metrics can help you locate unnecessary memory transfers that reduce performance.
In the Graphics window, see the
Total Time by Device Operation Type
 column, which displays the total time for each computation task.
Data Transfer Metrics
The total time is broken down into:
  • Allocation time
  • Time for data transfer from host to device
  • Execution time
  • Time for data transfer from device to host
This breakdown can help you understand better the balance between data transfer and GPU execution time. The Graphics window also displays in the
Transfer Size
 section, the size of the data transfer between host and device per computation task.
Computation tasks with sub-optimal offload schemas are highlighted in the table with details to help you improve those schemes.

Support for DPC++ Applications using oneAPI Level Zero API

This section describes support in the GPU Offload analysis for DPC++ applications that run OpenCL or oneAPI Level Zero API in the back end.
VTune
Profiler
 supports version 1.0.4 of the oneAPI Level Zero API.
Support Aspect
DPC++ application with OpenCL as back end
DPC++ application with Level Zero as back end
Operating System
Linux OS
Windows OS
Linux OS
Windows OS
Data collection
VTune
Profiler
 collects and shows GPU computing tasks and the GPU computing queue.
VTune
Profiler
 collects and shows GPU computing tasks and the GPU computing queue.
Data display
VTune
Profiler
 maps the collected GPU HW metrics to specific kernels and displays them on a diagram.
VTune
Profiler
 maps the collected GPU HW metrics to specific kernels and displays them on a diagram.
Display Host side API calls
Yes
Yes
Source Assembler for computing tasks
Yes
Yes

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.