Welcome to GTPin

========================================================================================

Introduction

========================================================================================
Binary Instrumentation Technology is widely used in the world of CPUs, in software and hardware development for dynamic performance analysis, emulation of future instructions, tracing, modeling, and more. The Binary Instrumentation technology for x86, named the Pin project, is the underlying technology for many internal and external tools. However, in the world of Graphics Processing Units (GPU), and specifically in Intel® GPU (GEN) architecture, the profiling and performance analysis capabilities are limited, or even simply absent.

The GTPin framework is a unique platform, and the only SW platform available for profiling the GEN Execution Units (EUs). GTPin provides a binary instrumentation engine for Intel GPUs EUs, along with an API for developing analysis tools, and many sample tools. GTPin allows you to capture a range of dynamic profiling data at the finest granularity of the specific GPU EU instruction. GTPin supports both compute and graphics workloads. It operates on regular, real-world GPU applications, as well as on precaptured API streams. The technology enables fast and accurate dynamic analysis of the code that is executing on the GPU EUs. GTPin opens up new opportunities to perform dynamic, low level workload and HW analysis on an Intel GPU, with greater efficiency than other current solutions. Some of the GTPin capabilities are integrated into Intel® VTune™ Profiler, Intel® Advisor and the Intel® Graphics Performance Analyzers (Intel® GPA).

GTPin is available, along with a set of analysis tools based on the GTPin framework. It also enables more advanced users to develop their own analysis tools. GTPin can analyze any GPU application. It also collects dynamic profiling data which the application executes on the GPU.

The rest of this guide describes GTPin capabilities, describes how to develop a profiling tool on top of GTPin, describes its API, and shows several examples how to use it.

Tutorial sections:

• High Level Architecture
• Capabilities
• Installation
• How To Run GTPin
• GTPin Configuration
• GTPin Sample Tools
• GTReplay

Reference sections:

• GTPin: Getting Started
• GTPin: API Reference

========================================================================================

High Level Architecture

========================================================================================
The picture below shows a software stack schematic of an original graphics or GP-GPU application that exploits a GPU device, and how profiling flow goes. GTPin framework is located beneath the application. The original application is not aware of being instrumented, and all the process happens in a completely transparent manner. GTPin receives the original binary kernel from the driver and instruments it. All the original instructions of the kernel, their order and flow remain intact. In addition, some extra instructions are added to the kernel to perform the profiling functionality. The new binary kernel - which is the new instrumented kernel - is sent for the execution on the device. During the execution the profiling data is collected within a memory buffer. In the end of the process GTPin retrieves the gathered profiling data from the memory and passes it back to the user for further analysis.

To perform any instrumentation work GTPin must be instructed by a Profiling Tool (or Profiling Application). The latter drives the instrumentation process, decides what kind of the instrumentation to perform, what kind of profiling data to collect, and how to process the gathered data. Profiling Tool communicates with GTPin framework via the "Tool API" interface. The rest of this tutorial describes the Tool API and how to create a new Profiling Tool based on existing examples.

========================================================================================

Capabilities

========================================================================================

• Supported operating systems
• Supported Gfx and GP-GPU APIs
• Supported GEN hardware
• Profiling capabilities

Supported operating systems

GTPin supports the following operating systems:

• Microsoft® Windows™ 10
• Microsoft® Windows™ 11
• The Linux Foundation™ Linux - tested on:
  • Canonical® Ubuntu™ 22.04
  • Canonical® Ubuntu™ 20.04
  • SUSE® SLES 15 SP4

GTPin supports 64-bit and 32-bit applications.

Supported Gfx and GP-GPU APIs

GTPin supports the following graphics and GP-GPU interfaces:

• Intel® oneAPI Level Zero (L0)
• OpenCL™
• Microsoft® DirectX™ 11
• Microsoft® DirectX™ 12
  • Including DirectX™ Raytracing (DXR)
• Khronos Group® Vulkan™

Supported GEN hardware

GTPin supports the following HW platforms:

Intel Integrated Graphics:

• 9th-13th Gen Intel® Core™ processors with Intel® UHD Graphics or Intel® Iris® Plus Graphics

Intel Discrete Graphics:

• Intel® Iris® Xe MAX Graphics

Profiling capabilities

GTPin is unique as the only framework that allows performing binary instrumentation of the Intel GPU (GEN) kernels for profiling purposes.
GTPin provides a set of ready-to-use analysis tools, and also allows advanced users to develop their own tools. Some of the available GTPin analysis tools include:

• Funtime and basic block (BBL) Latency sample tools allow you to perform Performance Profiling and Hot-spot Detection.
  The Funtime sample tool collects and aggregates EU cycles at the function (kernel) granularity. The BBL latency sample tool collects and aggregates EU cycles at the BBL granularity.
  Using these sample tools, you can easily find the most time-consuming blocks of code within any application.
  
• OpcodeProf sample tool allows you to collect an instructions mix. This analysis tool generates a histogram of the opcodes being executed by the program.
  Examples of questions answered by the Opcode Profiling include: How many multiply and add instructions have been executed? Is the systolic array a significant part of this workload? And so on.
  
• SIMDProf sample tool provides the number of active SIMD vector elements at the execution of each instruction within the kernel.
  This instrumentation allows accurate calculation of the number of operations executed, while taking SIMD instructions and SIMD masks into consideration.
  
• Traces are a powerful tool in the development of next-generation CPUs. They are also critical in GPU development.
  Traces are usually used to feed performance simulators, in order to check how architectural changes affect the performance of the workload.
  The GTPin tracing tool generates an instructions trace and a memory addresses trace, partitioned to HW threads, EUs, slices, etc. It is possible to add additional information to these traces.
  • Thread Level Occupancy. This tool gives you the ability to analyze GPU EU busy and idle times, and see how the application`s execution is balanced across the EUs.
    This tool is based on tracing time-stamp counters at the beginning and end of any SW thread dispatch to any of the HW threads.
  • Memory Contention Analysis. By analyzing the memory address tracing, you can build memory access patterns of the application, check their balance and efficiency.
    This is currently being prototyped by the VTune(TM) Profiler team to support the next generation of Intel discrete graphics products.

GTPin can capture any architecturally available data at the lowest possible level: the single EU assembly instruction.
More details on the existing tools can be found in GTPin Sample Tools.

Profiling granularity

GTPin collects profiling data separately, for:

• Each kernel/shader
• Each Enqueue/Draw command
• Each Execution Unit (EU)
• Each HW thread

Selective profiling

Profiling of a GPU workload can be limited to an area of interest by specifying instrumentation filters. These filters define the instrumentation and analysis scope in terms of

• Specific kernels, shaders and Enqueue/Draw commands (see GTPin: Filtering Instrumented Kernels)
• Specific hardware components, like Execution Units and HW threads (see GTPin: Defining the HW Profiling Scope)
• Specific software objects, like Thread Group IDs (see GTPin: Defining the Profiled Thread Group IDs)

========================================================================================

Installation

========================================================================================
To install GTPin you must unzip the release package that is provided as a zipped archive.

What's included within the package

When opened, the GTPin package has the following directory structure:

Profilers
|--Bin
|--Docs
|--Examples
|--GTReplay
|--Include
|--Lib
|--Pin
|--Scripts

• Bin. Contains the main gtpin executable.
• Docs. Contains the GTPin User Guide.
• Examples. Contains GTPin sample tools sources and corresponding DLLs.
• GTReplay. Contains GTReplay binaries and sample tools sources and corresponding DLLs.
• Include. Contains all header files required to build GTPin tools.
• Lib. Contains GTPin libraries.
• Pin. Contains Pin executables (an external component).
• Scripts. Contains Python Software Foundation Python* scripts for uncompressing GTPin traces.

In addition, the package contains the GTPin license and required external licenses.

========================================================================================

How To Run GTPin

========================================================================================
The following command runs application under GTPin and profiles GPU kernels with the tool toolname:

Profilers/Bin/gtpin -t toolname [GTPin arguments] -- app [application arguments]

Refer to the section GTPin Configuration for the information on available GTPin arguments.

========================================================================================

GTPin Configuration

========================================================================================
GTPin supports several configuration arguments. The most useful arguments are:

• -t, –toolname
  Specifies the name of the tool (if it is an existing GTPin sample tool), or it provides the path to the tool’s DLL
• –profile_dir
  Specifies directory to store profiling results. The default is ./GTPIN_PROFILE_{tool_name}{ordinal_number}
• –dump_isa
  Dumps all ISA files into profile_dir/ISA
• –hw_profile_scope
  Sets the HW scope to perform profiling (see details in GTPin: Defining the HW Profiling Scope)
• –thread_group_scope
  Sets the Thread Group IDs to be profiled (see details in GTPin: Defining the Profiled Thread Group IDs)
• –filter
  Specifies the kernels to be instrumented or the kernels not to be instrumented (see details in GTPin: Filtering Instrumented Kernels)
• -h, –help
  Displays usage information and lists all the supported configuration options

To see all the supported arguments, run the following command:

Profilers/Bin/gtpin --help

========================================================================================

GTPin Sample Tools

========================================================================================
The list of the ready-to-use sample tools:

• Funtime
  Counts the cycles it takes for each kernel to execute from the beginning to the end (see GTPin: Funtime Sample Tool)
• Latency
  Counts the cycles it takes for the basic block (BBL) of each kernel to execute from the beginning to the end (see GTPin: Latency Sample Tool)
• Opcodeprof
  Provides the dynamic frequencies of each of the kernel instructions ("instruction mix"), in the form of opcode histograms (see GTPin: Opcodeprof Sample Tool)
• Simdprof
  Counts the effective number of SIMD operations executed by the kernel (see GTPin: Simdprof Sample Tool)
• Itrace
  Generates an instruction (control-flow) trace (see GTPin: Itrace Sample Tool)
• Memorytrace
  Generates a memory-address trace (see GTPin: Memorytrace Sample Tool)
• Perftrace
  Traces kernel latencies (see GTPin: Perftrace Sample Tool)