In order to further improve performance of small-matrix multiplication, Intel MKL provides just-in-time (JIT) code generation for sgemm and dgemm on Intel® Xeon® processor Intel® Advanced Vector Extensions 2 (Intel® AVX2) and Intel® Advanced Vector Extensions 512 (Intel® AVX-512) architectures. Using JIT code generation enables you to use a GEMM kernel tailored to specific parameters given via input (for example, matrix sizes); thus it significantly increases performance of small-matrix multiplication.

The JIT ?gemm feature provided as part of MKL_DIRECT_CALL requires no code change from the user. If the size is small enough (M, N, K 16), a standard ?gemm call might invoke JIT for some ?gemm kernels.


In order to further improve performance, a dedicated JIT API has been introduced. In addition to enabling benefits from tailored GEMM kernels, this API enables you to call directly the generated kernel and remove any library overhead. For more information see the JIT API documentation.

To enable JIT code generation for ?gemm, compile your C or Fortran code with the preprocessor macro shown depending on whether a threaded or sequential mode of Intel MKL is required:

Intel MKL mode Macro Compiler option

For Fortran applications:

  • Enable the preprocessor by using the -fpp option for the Intel® Fortran Compiler or the -Mpreprocess option for PGI* compilers.

  • Include the Intel MKL Fortran include-file


  • Just-in-time code generation introduces a runtime overhead at the first call of ?gemm for a given set of input parameters: layout (parameter for C only), transa, transb, m, n, k, alpha, lda, ldb, beta, and ldc. To benefit from JIT code generation, use this feature when you need to call the same GEMM kernel (same set of input parameters - layout (parameter for C only), transa, transb, m, n, k, alpha, lda, ldb, beta, and ldc) many times (for example, several hundred calls).

  • If MKL_DIRECT_CALL_JIT is enabled, every call to ?gemm might generate a kernel that is stored by Intel MKL. The memory used to store those kernels cannot be freed by the user and will be freed only if mkl_finalize is called or if Intel MKL is unloaded. To limit the memory footprint of the feature, the number of stored kernels is limited to 1024 on IA-32 and 4096 on Intel® 64.

Optimization Notice

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804

Orange (only for download buttons)