Background

Advice

Performance

Usage Guidelines

Additional Resources

1: You do not create threads in your application but specify the parallel mode within the FFT module of Intel MKL.

Example:  Using Intel MKL Internal Threading Mode

#include "mkl_dfti.h"

void main () {

float x[200][100];

DFTI_DESCRIPTOR_HANDLE my_desc1_handle;

MKL_LONG status, len[2];

//...put input data into x[j][k] 0<=j<=199, 0<=k<=99

len[0] = 200; len[1] = 100;

status = DftiCreateDescriptor( &my_desc1_handle, DFTI_SINGLE,DFTI_REAL, 2,len);

status = DftiCommitDescriptor(my_desc1_handle);

status = DftiComputeForward(my_desc1_handle, x);

status = DftiFreeDescriptor(&my_desc1_handle);

}

See Intel® MKL 10.0 threading for more information on how to do this.

2. You create threads in the application yourself and have each thread perform all stages of FFT implementation, including descriptor initialization, FFT computation, and descriptor deallocation.

In this case, each descriptor is used only within its corresponding thread. It is recommended to set single-threaded mode for Intel MKL.

Specify the number of threads as below:

set MKL_NUM_THREADS = 1 for Intel MKL to work in the single-threaded mode (recommended) or use
mkl_set_num_threads( 1 ) threading control function.

set OMP_NUM_THREADS = n where n is the number of cores for the customer program to work in the multi-threaded mode if it is threaded using Open MP.

The configuration parameter DFTI_NUMBER_OF_USER_THREADS must have its default value of 1.

Example: Using Parallel Mode with Multiple Descriptors Initialized in a Parallel Region

#include "mkl_dfti.h"

#include "omp.h"
void main () {

MKL_LONG len[2];

//...put input data into x[j][k] 0<=j<=199, 0<=k<=99

len[0] = 50; len[1] = 100;

#pragma omp parallel {

DFTI_DESCRIPTOR_HANDLE my_desc_handle;

MKL_LONG myStatus;

int myID = omp_get_thread_num ();

myStatus = DftiCreateDescriptor (&my_desc_handle, DFTI_SINGLE,

DFTI_COMPLEX, 2, len);

myStatus = DftiCommitDescriptor (my_desc_handle);

myStatus = DftiComputeForward (my_desc_handle, &x [myID * len[0]] [0] );

} /* End OpenMP parallel region */

}

3. You create threads in the application yourself after initializing all FFT descriptors.

This implies that threading is employed for parallel FFT computation only, and the descriptors are released upon return from the parallel region.

In this case, each descriptor is used only within its corresponding thread. It is obligatory to explicitly set the single-threaded mode for Intel MKL, otherwise, the actual number of threads may differ from one, because the DftiCommitDescriptor function is not in a parallel region.

Example: Using Parallel Mode with Multiple Descriptors Initialized in One Thread

#include "mkl_dfti.h"

#include "omp.h"

void main (){

float _Complex x[200][100];

MKL_LONG len[2];

MKL_LONG i;

//...put input data into x[j][k] 0<=j<=199, 0<=k<=99

len[0] = 50; len[1] = 100;

DFTI_DESCRIPTOR_HANDLE my_desc_handle[4];

MKL_LONG myStatus;

for (i=0;i<3;i++) myStatus = DftiCreateDescriptor &my_desc_handle[i], DFTI_SINGLE, DFTI_COMPLEX, 2, len);

// each thread calculates real FFT for matrix (50*100)

#pragma omp parallel {

int myID = omp_get_thread_num ();

myStatus = DftiCommitDescriptor (my_desc_handle[myID]);

myStatus = DftiComputeForward (my_desc_handle[myID], &x [myID * len[0]] [0] );

} /* End OpenMP parallel region */

for (i=0;i<3;i++) myStatus = DftiFreeDescriptor (&my_desc_handle[i]);

}

Specify the number of threads as:

set MKL_NUM_THREADS = 1 for Intel MKL to work in the single-threaded mode (obligatory) or use
mkl_set_num_threads( 1 ) threading control function.

set OMP_NUM_THREADS = n where n is the number of cores for the customer program to work in the multi-threaded mode if you are using Open MP for threading.

The configuration parameter DFTI_NUMBER_OF_USER_THREADS must have the default value of 1.

4. You create threads in the application yourself using OpenMP after initializing the only FFT descriptor.

This implies that threading is employed for parallel FFT computation only, and the descriptor is released upon return from the parallel region. In this case, each thread uses the same descriptor.

The following example illustrates a parallel user program with a common descriptor used in several threads.

Example: Using Parallel Mode with a Common Descriptor

// set number of threads inside Intel MKL:

// since one-threaded mode for Intel MKL is forced automatically

// set OMP_NUM_THREADS = 4 - multi-threaded mode for customer

#include "mkl_dfti.h"

#include "omp.h"

void main (){

float _Complex x[200][100];

MKL_LONG status;

DFTI_DESCRIPTOR_HANDLE desc_handle;

int nThread = omp_get_max_threads ();

MKL_LONG len[2];

//...put input data into x[j][k] 0<=j<=199, 0<=k<=99

len[0] = 50; len[1] = 100;

status = DftiSetValue (desc_handle, DFTI_NUMBER_OF_USER_THREADS, nThread);

status = DftiCommitDescriptor (desc_handle);

// each thread calculates real FFT for matrix (50*100)

#pragma omp parallel num_threads(nThread){

MKL_LONG myStatus;

int myID = omp_get_thread_num ();

myStatus = DftiComputeForward (desc_handle,  &x [myID * len[0]] [0] );

} /* End OpenMP parallel region */

status = DftiFreeDescriptor (&desc_handle);

}

In this case, the number of threads, as well as any other configuration parameter, must not be changed after FFT initialization by the DftiCommitDescriptor() function is done.

In cases "1", "2", and "3", listed above, set the parameter DFTI_NUMBER_OF_USER_THREADS to 1 (its default value), since each particular descriptor instance is used only in a single thread.

In case "4", you must use the DftiSetValue() function to set the DFTI_NUMBER_OF_USER_THREADS to the actual number of FFT computation threads, because multiple threads will be using the same descriptor. If this setting is not done, your program will work incorrectly or fail, since the descriptor contains individual data for each thread.

Warning:

• It is not recommended to simultaneously parallelize your program and employ the Intel MKL internal threading because this will slow down the performance. Note that in case "4" above, FFT computation is automatically initiated in a single-threading mode.

• You must not change the number of threads after the DftiCommitDescriptor() function completed FFT initialization.

• New Features
  • Introduced the single precision complex absolute value function SCABS1
  • Introduced the solver ?DTSVB for diagonally dominant tri-diagonal systems which is up to 2x faster than the general solver with partial pivoting (?GTSV)
    • Added routines for factorization (?DTTRFB) and the forward/backward substitution (?DTTRSB) of the diagonally dominant tri-diagonal systems
• Performance improvements
  • FFTs
    • Enhanced performance for transforms which are a multiple of 8 or 13
    • Optimized 1D complex cluster FFTs for non-power-of-2 vector lengths
  • VSL
    • Convolution and Correlation computations that require decimation show significant improvements
• Bug fixes (see fixes list)

New in Intel® MKL 10.2 Update 3:

• Performance improvements
  • BLAS: Several Level 1 & 2 BLAS functions newly threaded; Improved scaling for DGEMM for skinny matrices
  • LAPACK: Improved scalability for LAPACK functions: ?POTRF, ?GEBRD, ?SYTRD, ?HETRD, and ?STEDC
  • FFTs: Extended threading to small-size multi-dimensional transforms and other cases
  • VML: Further optimizations: v(s,d)Asin, v(s,d)Acos, v(s,d)Ln, v(s,d)Log10, vsLog1p, v(s/d)Hypot
  • VSL: Improved performance for viRngPoisson and viRngPoissonV random number generators
• Usability/Interface improvements
  • Improved example programs for uBLAS, Java, FFTW3, LAPACK95, and BLAS95
  • New 64-bit integer (ILP64) fftw_mpi interfaces for cluster FFTs
• Bug fixes (see fixes list)

New in Intel® MKL 10.2 Update 2:

• Performance improvements
  • Many improvements in BLAS functions for Intel® Core™ i7 processors, and Intel® Xeon® processor 5300, 5400, and 5500 series
  • Improved scalability of the following LAPACK functions: ?POTRF, ?GEBRD, ?SYTRD, ?HETRD, and ?STEDC divide and conquer eigensolvers
  • PARDISO OOC performance has improved significantly for symmetric positive definite matrices
  • Improved performance for the double precision Sobol generator for dimensions >= 16
  • Improvements in many VML functions for Intel® Xeon® processor 5500 series and others: v(s,d)Pow, v(s,d)Ceil/Trunc/Floor, vsSin/Cos/SinCos, and vdSin/Cos/SinCos
  • Improved scalability of 1D, single precision, complex FFTs and improved performance for small 3D transforms
• Usability/Interface improvements
  • Support for 64-bit integer parameters in FFTW wrappers
  • Intel MKL is now compatible with the representation of logical values in GCC 4.4.0
  • All transpose functions now have a Fortran interface
• Bug fixes (see fixes list)

New in Intel® MKL 10.2:

• New features
  • LAPACK 3.2
  • Introduced implementation of the DZGEMM Extended BLAS function (as described at http://www.netlib.org/blas/blast-forum/). See the description of the ?gemm family of functions in the BLAS section of the reference manual.
  • PARDISO now supports real and complex, single precision data
• Usability/Interface improvements
  • Sparse matrix format conversion routines which convert between the following formats:
    • CSR (3-array variation) <-> CSC (3-array variation)
    • CSR (3-array variation) <-> diagonal format
    • CSR (3-array variation) <-> skyline
  • Fortran95 BLAS and LAPACK mod files are now included
    • Modules are pre-built with the Intel compiler and located in the include directory (see Intel® MKL User's Guide for full path)
    • Source is still included for use with other compilers
    • Documentation for these interfaces can be found in the Intel® MKL User's Guide
  • The FFTW3 interface is now integrated directly into the main libraries
    • Source code is still included to create wrappers for use with compilers not compatible with the default Intel® Fortran compiler convention for name decoration
    • See Appendix G of the Reference Manual for information
  • DFTI_DESCRIPTOR_HANDLE now represents a true type name and can now be referenced as a type in user programs
  • Added parameter to Jacobi matrix calculation routine in the optimization solver domain to allow access to user data (see the description of the djacobix function in the reference manual for more information)
  • Added an interface mapping calls to single precision BLAS functions in Intel® MKL (functions with 's' or 'c' initial letter) to 64-bit floating point precision functions has been added on 64-bit architectures (See 'sp2dp' in the Intel® MKL User Guide for more information)
  • Compatibility libraries (also known as "dummy" libraries) have been removed from this version of the library
• Performance improvements
  • Further threading in BLAS level 1 and 2 functions for Intel® 64 architecture
    • Level 1 functions (vector-vector): (CS,ZD,S,D)ROT, (C,Z,S,D)COPY, and (C,Z,S,D)SWAP
      • Increase in performance by up to 1.7-4.7 times over version 10.1 Update 1 on 4-core Intel® Core™ i7 processor depending on data location in cache
      • Increase in performance by up to 14-130 times over version 10.1 Update 1 on 24-core Intel® Xeon® processor 7400 series system, depending on data location in cache
    • Level 2 functions (matrix-vector): (C,Z,S,D)TRMV, (S,D)SYMV, (S,D)SYR, and (S,D)SYR2
      • Increase in performance by up to 1.9-2.9 times over version 10.1 Update 1 on 4-core Intel® Core™ i7 processor, depending on data location in cache
      • Increase in performance by up to 16-40 times over version 10.1 Update 1 on 24-core Intel® Xeon® processor 7400 series system, depending on data location in cache
  • Introduced recursive algorithm in 32-bit sequential version of DSYRK for up to 20% performance improvement on Intel® Core™ i7 processors and Intel® Xeon® processors in 5300, 5400, and 7400 series.
  • Improved LU factorization (DGETRF) by 25% over Intel MKL 10.1 Update 1 for large sizes on the Intel® Xeon® 7460 Processor; small sizes are also dramatically improved
  • BLAS *TBMV/*TBSV functions now use level 1 BLAS functions to improve performance by up to 3% on Intel® Core™ i7 processors and up to 10% on Intel® Core™2 processor 5300 and 5400 series.
  • Improved threading algorithms to increase DGEMM performance
    • up to 7% improvement on 8 threads and up to 50% on 3,5,7 threads on the Intel® Core™ i7 processor
    • up to 50% improvement on 3 threads on Intel® Xeon® processor 7400 series.
  • Threaded 1D complex-to-complex FFTs for non-prime sizes
  • New algorithms for 3D complex-to-complex transforms deliver better performance for small sizes (up to 64x64x64) on 1 or 2 threads
  • Implemented high-level parallelization of out-of-core (OOC) PARDISO when operating on symmetric positive definite matrices.
  • Reduced memory use by PARDISO for both in-core and out-of-core on all matrix types
    • PARDISO OOC now uses less than half the memory previously used in Intel MKL 10.1 for real symmetric, complex Hermitian, or complex symmetric matrices
  • Parallelized Reordering and Symbolic factorization stage in PARDISO/DSS
  • Up to 2 times better performance (30% improvement on average) on Intel® Core™ i7 and Intel® Core™2 processors for the following VML functions: v(s,d)Round, v(s,d)Inv, v(s,d)Div, v(s,d)Sqrt, v(s,d)Exp, v(s,d)Ln, v(s,d)Atan, v(s,d)Atan2
  • Optimized versions of the following functions available for Intel® Advanced Vector Extensions (Intel® AVX)
    • BLAS: DGEMM
    • FFTs
    • VML: exp, log, and pow
    • See important information in the Intel® MKL User's Guide regarding the mkl_enable_instructions() function for access to these functions

Note 1:  A number of other LAPACK routines, which are based on threaded LAPACK or BLAS routines, make effective use of parallelism: *gesv, *posv, *gels, *gesvd,*syev, *heev, etc.

Note 2: Level1 BLAS  and Level2 BLAS are threaded only for:
1) Intel® 64 architecture
2) Intel® Core^TM2 Duo and Intel® Core^TM i7 processors

Note 3: all data provided into this article are relevant to the latest MKL version 10.2.

Note 4: The list of threaded functions into the current version of MKL you can find into MKL User's Guide ( see chapter 6 - "Using Intel(R) MKL Parallelism")

Intel® MKL 10.2 has early support of Intel® AVX and includes special optimizations for several BLAS, DFT and VML functions. Intel® AVX code in Intel® MKL corresponds to the programmer's reference 319433-004 available on the Intel® AVX page. To prevent this code from running on actual Intel® AVX hardware, which may correspond to updated version of the specification, Intel® MKL blocks dispatching of Intel® AVX code unless it requested explicitly via an enabling function call. This document provides a step-by-step guide for enabling the optimizations in Intel® MKL.

Functions with Intel® AVX optimizations

Domain	Functions
BLAS	dgemm
FFT	all
VML	exp, log, pow

Building an Intel® AVX enabled application with Intel® MKL

To enable Intel® AVX optimizations in an Intel® MKL application, add a call to the "mkl_enable_instructions" function before any other Intel® MKL call. Then the application will benefit from Intel® AVX optimizations when executed on Intel® SDE emulator.

The "mkl_enable_instructions" function is declared in mkl_service.fi for FORTRAN 77 interface and in mkl_service.h for C interface and has the following syntax:

irc = mkl_enable_instructions(MKL_AVX_ENABLE)

The function returns ‘0' if the function have failed, for instance when it has been called after another Intel® MKL function. Otherwise the return value is ‘1'.

With the call added compile the application and link it with Intel® MKL in usual way. See Intel® MKL User's Guide for details.

Running an Intel® AVX enabled application

Exploring Intel® AVX starts with downloading Intel® SDE emulator from WhatIf site http://whatif.intel.com/. Intel® SDE does not require installation, download and unpack a package suiting your system (Intel® 64 and Linux* or Windows* OS are supported by Intel® MKL 10.2).

With Intel® SDE in place, run your Intel® AVX enabled application via the emulator to get the new instructions working.

Linux*:

<path to SDE>/sde -- <application> [args]

Windows*:

<path to SDE>\sde.exe -- <application> [args]

Refer to Intel® SDE page for detailed information on its how to use Intel® SDE for application analysis.

Code sample

This sample C code demonstrates multiplication of two 8x8 matrices using Intel® AVX optimized DGEMM from Intel® MKL 10.2.

#include <stdio.h>

#include "mkl_service.h"

#include "mkl_cblas.h"

#include "mkl_types.h"

int main(void)

{

    MKLVersion ver;

    const MKL_INT n = 8;

    double        alpha = 1.0, beta = 1.0;

    double        a[n*n], b[n*n], c[n*n];

    int i;

// Enable Intel® AVX optimizations in Intel® MKL

     if ( MKL_Enable_Instructions(MKL_AVX_ENABLE) ) {

           puts("Intel(R)  AVX optimizations are enabled.");

     } else {

           puts("Intel(R)  AVX optimizations are not enabled. MKL_Enable_Instructions was called after another MKL function.");

     }

// Print information on CPU optimization in effect

     MKLGetVersion(&ver);

     printf("Processor optimization: %s\n",ver.Processor);

// Generate matrices

    for (i=0;i < n*n;i++) {

           a[i] = i;

           b[i] = n*n-i;

     }

// Call Intel® MKL DGEMM function

     cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans, n, n, n, alpha, a, n, b, n, beta, c, n);

}

When linked with Intel® MKL 10.2 (see User's Guide and MKL release examples for linking instructions) and executed on Intel® SDE this code produces the following output:

Intel(R) AVX optimizations are enabled.

Processor optimization: Intel(R) Advanced Vector Extensions (Intel(R) AVX) Enabled Processor.

• if at least one prime factor is larger than 2²⁹, the transform is not supported

An Intel MKL 1D FFT of size K=N*M (where N and M are chosen somewhat arbitrarily) is computed using the Cooley-Tukey factorization algorithm in 2 stages. During the first stage, N transforms of size M are performed in parallel with necessary permutation. For the second stage, multiplication by twiddle factors and M transforms of size N are done in parallel. Care is taken to avoid cache line splits between the threads for both stages. Additionally, the size of the twiddle factor table is reduced by a binary split.

When running on an Intel® Xeon® Processor X5492 (2 x 4-core, 3.4 GHz) running a 64-bit operating system FFT performance improved by up to 5 times for 8 threads, when compared to Intel MKL 10.1 Update 2.

Customers just need to upgrade to the Intel MKL 10.2 in order to take advantage of this new feature for multi-threaded applications. No code changes are required.

For more information the Intel MKL FFTs, please refer to Chapter 11 of the Reference Manual. Also refer to Intel MKL Linking Advisor for details on how to link to the FFT functions.

• Extra Precise Iterative Refinement
• Non-Negative Diagonals from Householder QR
• High Performance QR and Householder Reflections on Low-Profile Matrices
• New fast and accurate Jacobi SVD
• Routines for Rectangular Full Packed format
• Pivoted Cholesky
• Mixed precision iterative refinement routines for exploiting fast single precision hardware
• Some new variants added for the one sided factorization
• More robust DQDS algorithm