Vish Viswanathan, Karthik Kumar, Thomas Willhalm, Patrick Lu, Blazej Filipiak, Sri Sakthivelu
An important factor in determining application performance is the time required for the application to fetch data from the processor’s cache hierarchy and from the memory subsystem. In a multi-socket system where Non-Uniform Memory Access (NUMA) is enabled, local memory latencies and cross-socket memory latencies will vary significantly. Besides latency, bandwidth (b/w) also plays a big role in determining performance. So, measuring these latencies and b/w is important to establish a baseline for the system under test, and for performance analysis.
Intel® Memory Latency Checker (Intel® MLC) is a tool used to measure memory latencies and b/w, and how they change with increasing load on the system. It also provides several options for more fine-grained investigation where b/w and latencies from a specific set of cores to caches or memory can be measured as well.
Intel® MLC supports both Linux and Windows.
Previous releases of MLC s/w provided two sets of binaries (mlc and mlc_avx512). mlc_avx512 was compiled with newer tool chain to support AVX512 instructions while mlc binary supported SSE2 and AVX2 instructions. With MLC v3.7 release onwards, only one binary is provided which supports SSE2, AVX2 and AVX512 instructions. By default AVX512 instructions won’t be used whether the processor supports it or not unless -Z argument is added explicitly to the command line.
It is challenging to accurately measure memory latencies on modern Intel processors as they have sophisticated h/w prefetchers. Intel® MLC automatically disables these prefetchers while measuring the latencies and restores them to their previous state on completion. The prefetcher control is exposed through MSR (Disclosure of Hardware Prefetcher Control on Some Intel® Processors) and MSR access requires root level permission. So, Intel® MLC needs to be run as ‘root’ on Linux. On Windows, we have provided a signed driver that is used for this MSR access. If Intel® MLC can’t be run with root permissions, please consult the readme.pdf that can be found in the download package.
When the tool is launched without any argument, it automatically identifies the system topology and measures the following four types of information. A screen shot is shown for each.
1. A matrix of idle memory latencies for requests originating from each of the sockets and addressed to each of the available sockets
2. Peak memory b/w measured (assuming all accesses are to local memory) for requests with varying amounts of reads and writes
3. A matrix of memory b/w values for requests originating from each of the sockets and addressed to each of the available sockets
4. Latencies at different b/w points
It also measures cache-to-cache data transfer latencies
Intel® MLC also provides command line arguments for fine grained control over latencies and b/w that are measured.
Here are some of the things that are possible with command line arguments:
One of the main features of Intel® MLC is measuring how latency changes as b/w demand increases. To facilitate this, it creates several threads where the number of threads matches the number of logical CPUs minus 1. These threads are used to generate the load (henceforth, these threads will be referred to as load-generation threads). The primary purpose of the load-generation threads is to generate as many memory references as possible. While the system is loaded like this, the remaining one CPU (that is not being used for load generation) runs a thread that is used to measure the latency. This thread is known as the latency thread and issues dependent reads. Basically, this thread traverses an array of pointers where each pointer is pointing to the next one, thereby creating a dependency in reads. The average time taken for each of these reads provides the latency. Depending on the load generated by the load-generation threads, this latency will vary. Every few seconds the load-generation threads automatically throttle the load generated by injecting delays, thus measuring the latency under various load conditions. Please refer to the readme file in the package that you download for more details
Launching Intel® MLC without any parameters measures several things as stated earlier. However, with command line arguments, each of the following specific actions can be performed in sequence:
prints a matrix of local and cross-socket memory latencies
prints a matrix of local and cross-socket memory b/w
prints peak memory b/w (core generates requests at fastest possible rate) for various read-write ratios with all local accesses
prints maximum memory b/w (by automatically varying load injection rates) for various read-write ratios with all local accesses
prints the idle memory latency of the platform
prints the loaded memory latency of the platform
prints the cache-to-cache transfer latencies of the platform
do not modify prefetcher settings
There are more options for each of the commands above. Those are documented in the readme file in more detail and can be downloaded
Both Linux and Windows versions of Intel® MLC are included in the download.
Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.
Notice revision #20110804