Intel® Math Kernel Library: Custom Static Linkage

Audience

People who wish to link statically against Intel MKL while using a legacy build system might experience difficulties when linking Intel MKL components due to circular dependencies. This issue only exists when the legacy build system is not able to form a group of libraries during linkage i.e., the start group and end group options cannot be supplied.

Solutions

There are two solutions possible:

  1. Supply the different libraries multiple times to the linker until all symbols are resolved.
  2. Link dynamically, in particular against the Single Dynamic Layer (SDL) library.

Background

Intel MKL Link Line Advisor suggests to form a group of libraries when linking against Intel MKL. Linking libraries as a group instructs the linker to search the set of enclosed libraries until there are no further symbols resolved. While this increases the link time of the build process, it is the proper way to resolve circular dependencies between libraries. A typical link line may look like below example; the group options carried to the linker are given by -Wl,--start-group and -Wl,--end-group:

-Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_lp64.a $MKLROOT/lib/intel64/libmkl_sequential.a $MKLROOT/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm

(static sequential library using 32-bit index space)

Intel MKL consists of multiple components that can be combined in a flexible manner. For example, Intel MKL supports a variety of compilers along with their threading runtimes, different index sizes (LP64 vs. ILP64), and many more choices. To find out more about the possible choices, please consult the Intel MKL Link Line Advisor. According to the component choices, Intel MKL is assembled from multiple components that correspond to different libraries.

Although a valid set of components have been selected, it might be possible that the corresponding libraries expose circular dependencies i.e., components might need symbols from other components such that no particular order of libraries is able to resolve all symbols. Because of the complex choices, circular dependencies between Intel MKL components cannot be avoided, and they are not considered a bug that needs to be resolved. Instead, a group of libraries can be formed that are linked with proper awareness.

A Non-Solution

The mechanics here rely on unique translation names (object files). Therefore the presented "solution" is not reliable as it depends on things that can be well-considered MKL-internals. Anyhow, a given selection of objects (or archives) can be re-archived into a custom static library. The bash script below creates a single consolidated static library (called mkl.a) that corresponds to the above example selection (static sequential library using 32-bit index space).


	#!/bin/bash


	NAME=mkl


	AR="xiar -qipo"

	MV="mv"

	RM="rm -rf"


	MKD="mkdir"

	TMP=$(mktemp -u)

	CWD=$(pwd)


	$MKD $TMP; cd $TMP

	$AR x $MKLROOT/lib/intel64/libmkl_intel_lp64.a

	$AR x $MKLROOT/lib/intel64/libmkl_sequential.a

	$AR x $MKLROOT/lib/intel64/libmkl_core.a

	$AR rcs $NAME.a *.o

	$MV $NAME.a $CWD


	cd $CWD

	$RM $TMP

	

The script uses the Intel (bin-)tool chain (which are wrappers around the GNU* tools) e.g., xiar (instead of ar). This way it is even possible to preserve the Intermediate Language (IL) information that might have been generated using the Intel Compiler. This is useful whenever at least one translation unit of the project was compiled using the InterProcedural Optimization (IPO) option of the Intel Compiler. On the other hand, using xiar rather than ar does not harm the usability of the resulting archive. An archive created in this way can be used by all compatible compilers i.e., the archive can be consumed by the GNU* GCC tool chain (Intel Compiler and the platform's main compiler are always binary/link-compatible including language options such as OpenMP*). Note, that in the above example, the question about the threading runtime is not even raised because the sequential library component was selected.

Comments

As usual: do not run a script without understanding what it does! Also, the above script could be more sophisticated up to and including to call the offline version of the Intel MKL Link Line Advisor in order to take a certain selection right away. Anyhow, keep in mind that the script consumes quite some temporary storage space (e.g., 700 MB) as well as running more than a few seconds (e.g., 2 Minutes).

Important: re-archiving MKL's static libraries depends on unique names of the internal translation units of Intel MKL, hence the above non-solution may overwrite intermediate objects using the same file names, and when linking the resulting static library one may experience unresolved symbols.

Related

Einzelheiten zur Compiler-Optimierung finden Sie in unserem Optimierungshinweis.