Intel Software Network articles Feed

Improving the Compute Performance of Video Processing Software Using AVX (Advanced Vector Extensions) Instructions (by Eli Hernandez and Larry Moore)

Wed, 08 Feb 2012 00:00:00 -0800

Download Article

Download Improving the Compute Performance of Video Processing Software Using AVX (Advanced Vector Extensions) Instructions [PDF 311KB]

Abstract

Modern x86 CPUs permit instruction level parallelism (e.g. SIMD) on register vectors at most 128-bits. Second Generation Intel® Core™ Processors include the first generation of AVX (256-bit operators), which permits increased parallel processing. This paper outlines a case study in which AVX instructions are used to improve the compute performance of a de-saturation algorithm. The paper also discusses how future integer based AVX instructions might be used to further enhance SIMD optimizations and achieve even greater performance benefits on video processing algorithms.

1. Introduction

Modern x86 CPUs permit Instruction Level Parallelism (ILP), such as Single Instruction Multiple Data (SIMD), on vectors at most 128-bit. These register vectors can be used to process multiple data elements with fewer instructions. Second Generation Intel® Core™ Processors (codenamed Sandy Bridge) included the first generation of AVX, which is a 256-bit instruction set extension to the Intel® Streaming SIMD Extensions (Intel® SSE).

The first generation of AVX included a wide range of instructions designed primarily to accelerate compute intensive algorithms performing arithmetic operations on floating point data. However, even if an algorithm is integer based, using AVX instructions could potentially increase an algorithm’s performance without sacrificing accuracy of the results. In video processing algorithms, the pixel channels are often stored as 8-bit unsigned integers (bytes) and processed as 32-bit or larger format integer values. Therefore, most video algorithms require conversion of pixels to and from a format. Wider bit widths are used for calculation accuracy and smaller formats are used to save space. Typically, floating-point units are not used because the extra conversion costs do not significantly improve accuracy. However, AVX is capable of greatly improving the runtime performance of video processing software and a vast number of other software applications by the increased parallelism.

This paper describes a case study in which AVX instructions are used to enhance the performance of a de-saturation algorithm (a common video filter). The case study takes the algorithm from a non-SIMD state to AVX based SIMD. The paper also discusses how future generations of AVX may be able to further aid performance optimization and enable greater performance of video processing.

2. Intel SIMD Overview

On Intel SIMD architectures, a vector register can store a group of data elements of a single data type (e.g. floats or integers). The vector registers of Sandy Bridge are 256 bits wide whereas all other processors since Intel® Pentium III were 128 bits wide. Each vector (called YMM in Sandy Bridge) register can store 8 floats, 8 32-bit integers, 32 chars, etc. AVX instructions operate on the full 256 bits, but SSE can only operate on 128 bits.

A SIMD enabled-processor can execute a single operation on multiple data. An operation performed simultaneously on multiple data elements is a vector process. SIMD vectorization is the process of converting an algorithm from a scalar to a vector implementation. The multiply function in sample code below is used to illustrate the difference between the scalar and SIMD vector process.

Figure 1: This illustrates the difference between scalar and vector processes. The scalar version would have 16 loads, 8 multiplications and 8 stores. SSE can potentially have 4 loads, 2 vector multiplications and 2 stores. AVX would use 2 loads, 1 large vector multiplication and 1 store. The labels with VMUL were shortened to hide the distinction between various versions of vector multiplication instructions. VMUL performs multiplication on vectors A and B for each element pair and stores the results in another vector. Let us suppose for simplicity that loads and stores cost 3 cycles, all multiplication costs 1 cycle and we are ignoring pipelining. Then the scalar version spends 80 cycles to compute 8 elements while the AVX version spends 10 cycles, yielding a theoretical speedup of 8x. This clearly illustrates why SIMD vectorization has become a very important aspect to optimize application performance. Also given observed performance benefits with SIMD, automatic SIMD vectorization has become as keystone feature in advanced compilers.

3. Video Processing Code

Typical video processing algorithms calculate pixel values using a triple for-loop (for each frame, for each X, for each Y). This typically is seen as an area of high CPU utilization (i.e. hotspot). Video processing application hotspots are excellent candidates to optimize with AVX.

A simple approach to optimize using SIMD involves taking advantage of the latest processor technology features, such as AVX. The following sections describe the optimization process using AVX instructions to enhance the performance of a de-saturation algorithm. The serial code implementation is briefly discussed and AVX-based SIMD instructions are used to optimize the de-saturation algorithm. Finally, this chapter ends with our performance results of the optimize code.

3.1. Desaturation - Sample Code

The typical implementation of the Desaturation algorithm uses the incoming pixel values to compute a luminance value. The luminance value is applied to all outgoing pixels to de-saturate images as part of processing video for output.

As you can see in the sample code below the algorithm traverses row by row to get pixel data, which channel values (blue, green, and red) are used to calculate the luminance value. In other to achieve high accuracy the algorithm converts the one-byte channel values to single precision floating point. The floating-point values are used in a dot product type of operation to compute the luminance value. The Desaturation sample algorithm uses the fLuminace(…) function to convert pixel channel values from byte to float. The conversion to float is achieved implicitly by typecasting each channel value to float and with weights as constants for Red, Green and Red, the fLuminance(…) function uses the float values to compute luminance which value is applied to the video output.

Note that the conversion of channel data from byte to float occurs implicitly by typecasting to float. Although the scalar code looks simple and trivial, the assembly code generated by the compiler is much more complex. In analysis of the generated assembly code, the implicit byte to float conversion can be performed with fewer instructions by using the more efficient AVX instructions. As we have observed, the serial code calculates one channel and one pixel at a time. Nothing is computed in parallel (ignoring pipelining and reordering). Refer to Appendix A for the assembly code.

3.2. Desaturation - Optimization with AVX

This section outlines the transformation of the serial code and describes how AVX, SSE4.1 and SSE2 instructions optimize the de-saturation algorithm. As illustrated in Chapter 2 with SIMD, we can work on many items at once. Therefore, the load, store, conversion and math operations can be done in parallel. The algorithm below describes how we can use instruction level parallelism (via AVX instructions) to significantly improve performance. Note that the algorithm is written with the restriction that we could only use available instructions, not idealistic for future instructions as we discuss later. Therefore, lines 19, 20 and 21 involve an intermediate step to convert 32-bit integers back down to 8-bit unsigned and etcetera.

Figure 2. De-saturation algorithm

With the Figure 2 as the backbone of de-saturate, we can implement the real code. The motivations for using a procedure similar to Figures 2 and 3 are that:

AVX provides greater throughput for parallel processing of single-precision floating- point units than any past Intel SIMD x86 extension (MMX, SSE, SSE2, SSE3, SSE4.1, SSE4.2).
The cost (SIMD) to cast byte (8-bit unsigned char) to integer (32-bit signed integer) to single precision floating point (32-bit float) and back is less than using multiple calls of the equivalent code (scalar) using just bytes or integers.
Using byte based SIMD with this procedure gives poor precision. Parallel performance is not considered.
Using integer based SIMD with this procedure gives acceptable precision. Current AVX instructions for integer arithmetic do not exist and therefore cannot take full advantage of the 256-bit registers.
Using float based SIMD with this procedure gives very good precision and offers higher performance than those described above.

Figure 3. De-saturation code optimzed AVX¹

The algorithm and AVX code shown in Figures 2 and 3 convey the same exact process line-for-line. Notice that only lines 9 and 16 involve doing the real work. Theses lines each process 8 single precision floating point multiplications in parallel, totalling 16 multiplications for 2 instructions versus 16 individual multiplication instructions. Everything else is unnecessary overhead to make use of the parallel instructions or to increase precision.

Despite the overhead, this code still improves performance by 1.45x . If integer based instructions existed with equivalent parallelism to that of single precision floating point, we could further increase performance. In such case, lines 6, 8, 10, 11, 14, 15, 17 and 18 could be eliminated. Lines 9 and 16 would operate on integers instead. Lines 19, 20 and 21 could require a single pack instruction (integer to byte). Of course, there are other hypothetical instructions that could be introduced with future AVX generations. The potential performance gain is left as an exercise for the reader. For assembly instructions generated by intrinsic functions used in the inner [ix] loop, refer to Appendix B.

3.3 Desaturation - Performance Test Results³

Performance assessment of the de-saturation algorithm optimized with AVX in this study observed a 1.45x speedup when compared to the serial code. To gather performance data the de-saturation algorithm was applied to a 1440x1080 image and was looped 100 times. Performance was measured in elapsed time (milliseconds) taken to de-saturate the image, the following performance numbers were consistently observed:

Serial Code:	1264 milliseconds
Code with AVX:	873 milliseconds
Performance Scaling:	1.45x or 1264ms/873ms

A kernel (small application program) was used to run the algorithm. A kernel with a 1.45x scaling typically translates to a performance improvement of 10% to 15% when measured at the workload level. However, for video processing this rule of thumb does not apply. Consider you are applying the de-saturation algorithm to a one-minute or longer video clip. In that case, there will be more than 100 frames (images) to process. In theory, since more data has be processed, the performance boost potential could be more than 1.45x especially if or when processing full High-Definition (e.i.,1920x1080) video.

4. Packed Integer Conversion Instructions

Since our optimized de-saturation algorithm uses one of the SSE4.1 instructions, we will give an overview of SSE4.1 because other SSE4.1 instructions may be applicable for the optimization of other video processing algorithms. The Packed Integer Conversion instruction set contains 12 instructions for packed integer bit width conversions. Any of which can be utilized to optimize code where bit width is to be increased for integer data.

The table in Figure 5 lists the SSE4.1 instructions for packed integer conversions. The instructions support sign extension and zero extension conversions of byte to word, byte to double word, byte to quad-word, word to double word, word to quad-word, and double word to quad-word. Additionally, the chart shows a comparison of SSE2 vs SSE4.1 instructions needed to convert four (4) one-byte integers to four (4) 32-bit integers.

The pmovzxbd (byte to double word) instruction was utilized a total of four (4) times in the de-saturate optimization. When/if these instructions include support for full 256-bit register, the use of this instruction in the optimized algorithm will be reduced to two (2). Thereby further improving the loop performance.

Figure 4. Instructions for bit width conversions of packed integers

The source operand to packed integer conversion instructions is from either an XMM register or memory. The destination is always an XMM register. When accessing memory, no alignment is required, unless alignment checking is enabled. In which case, all conversions must be aligned to the width of the memory being referenced. The number of elements that can be converted and width of memory reference is illustrated in Figure 5. The alignment requirement is shown in parenthesis.

Figure 5. Number of elements to process. P is Packed. MOV is Move (copy register). ZX is Zero Extend. SX is Sign Extend. B is Byte. W is Word. D is Double Word. Q is Quad-Word.

5. Conclusion

This paper has discussed how Second Generation Intel® Core™ Processors could increase parallel processing via AVX instructions and 256 bit registers. This paper outlined a case study in which AVX instructions were used to improve the compute performance of a de-saturation algorithm. The paper also discussed how future integer based AVX instructions could be used to further enhance SIMD optimizations and achieve even greater performance benefits on video processing algorithms. The procedure described demonstrated how AVX instructions or their intrinsic functions could be utilized to improve the runtime performance of video processing applications. The paper documented that despite some overhead incurred to setup for SIMD processing, the video de-saturation still achieved excellent performance benefits.

About the Authors

Eli Hernandez is an Application Engineer in the Consumer Client and Power Enabling Group at Intel Corporation where he works with customers to optimize their software for power efficiency and to run best on Intel hardware and software technologies. Eli joined Intel in August of 2007 with over 12 years of experience in software development for the telecom and the chemical industry. He received his B.S. in Electrical Engineering in 1989 and completed Master Studies in Computer Science in 1991-1992 from the DePaul University of Chicago.

In 2008, Larry Moore graduated from Saint Petersburg College with Honors. He received a Who's Who Among Students Award and was a member of Phi Theta Kappa Honor Society. In 2011, he spent 8 months at Intel as an application engineer intern, in DuPont, Washington. Currently, he is attending the University of South Florida at Tampa, Florida in an accelerated graduate program, pursuing both a Bachelor of Science and Master of Science in Computer Engineering. His current research involves computer aided verification of real-time systems and model checking. Larry is also a member of IEEE Computer Society.

Appendix A: Inner loop quivalent assembly of the serial code

Roughly 45 instructions to proccess an iteration of the algorithm inner loop. With throughput of 1 pixels processed per iteration.

Appendix B: Equivalent assembly of inner loop optimized with AVX

Roughly 30 instructions to proccess an iteration of the algorithm inner loop. With throughput of 4 pixels processed per iteration.

¹ Load and store operations on optimized code assumes data is aligned.
² Please see footnote 3 and section 3.3.
³ The performance measurements in this section are the actual numbers from real tests. However, we do not guarantee you will achieve as good of a performance.

02-08-2012

Tech Articles

http://software.intel.com/file/41303

This paper describes a case study in which AVX instructions are used to enhance the performance of a de-saturation algorithm (a common video filter). The case study takes the algorithm from a non-SIMD state to AVX based SIMD. The paper also discusses how future generations of AVX may be able to further aid performance optimization and enable greater performance of video processing.

Introduction to Intel® Advanced Vector Extensions

Tue, 21 Jun 2011 00:00:00 -0700

By Chris Lomont

Download Article

Download Introduction to Intel® Advanced Vector Extensions [PDF 1.4MB]

Intel® Advanced Vector Extensions (Intel® AVX) is a set of instructions for doing Single Instruction Multiple Data (SIMD) operations on Intel® architecture CPUs. These instructions extend previous SIMD offerings (MMX™ instructions and Intel® Streaming SIMD Extensions (Intel® SSE)) by adding the following new features:

The 128-bit SIMD registers have been expanded to 256 bits. Intel® AVX is designed to support 512 or 1024 bits in the future.
Three-operand, nondestructive operations have been added. Previous two-operand instructions performed operations such as A = A + B, which overwrites a source operand; the new operands can perform operations like A = B + C, leaving the original source operands unchanged.
A few instructions take four-register operands, allowing smaller and faster code by removing unnecessary instructions.
Memory alignment requirements for operands are relaxed.
A new extension coding scheme (VEX) has been designed to make future additions easier as well as making coding of instructions smaller and faster to execute.

Closely related to these advances are the new Fused-Multiply-Add (FMA) instructions, which allow faster and more accurate specialized operations such as single instruction A = A * B + C. The FMA instructions should be available in the second-generation Intel® Core™ CPU. Other features include new instructions for dealing with Advanced Encryption Standard (AES) encryption and decryption, a packed carry-less multiplication operation (PCLMULQDQ) useful for certain encryption primitives, and some reserved slots for future instructions, such as a hardware random number generator.

Instruction Set Overview

The new instructions are encoded using what Intel calls a VEX prefix, which is a two- or three-byte prefix designed to clean up the complexity of current and future x86/x64 instruction encoding. The two new VEX prefixes are formed from two obsolete 32-bit instructions-Load Pointer Using DS (LDS-0xC4, 3-byte form) and Load Pointer Using ES (LES-0xC5, two-byte form)-which load the DS and ES segment registers in 32-bit mode. In 64-bit mode, opcodes LDS and LES generate an invalid-opcode exception, but under Intel® AVX, these opcodes are repurposed for encoding new instruction prefixes. As a result, the VEX instructions can only be used when running in 64-bit mode. The prefixes allow encoding more registers than previous x86 instructions and are required for accessing the new 256-bit SIMD registers or using the three- and four-operand syntax. As a user, you do not need to worry about this (unless you're writing assemblers or disassemblers).

Note: The rest of this article assumes operation in 64-bit mode.

SIMD instructions allow processing of multiple pieces of data in a single step, speeding up throughput for many tasks, from video encoding and decoding to image processing to data analysis to physics simulations. Intel® AVX instructions work on Institute of Electrical and Electronics Engineers (IEEE)-754 floating-point values in 32-bit length (called single precision) and in 64-bit length (called double precision). IEEE-754 is the standard defining reproducible, robust floating-point operation and is the standard for most mainstream numerical computations.

The older, related Intel® SSE instructions also support various signed and unsigned integer sizes, including signed and unsigned byte (B, 8-bit), word (W, 16-bit), doubleword (DW, 32-bit), quadword (QW, 64-bit), and doublequadword (DQ, 128-bit) lengths. Not all instructions are available in all size combinations; for details, see the links provided in "For More Information." See Figure 2 later in this article for a graphical representation of the data types.

The hardware supporting Intel® AVX (and FMA) consists of the 16 256-bit YMM registers YMM0-YMM15 and a 32-bit control/status register called MXCSR. The YMM registers are aliased over the older 128-bit XMM registers used for Intel SSE, treating the XMM registers as the lower half of the corresponding YMM register, as shown in Figure 1.

Bits 0-5 of MXCSR indicate SIMD floating-point exceptions with "sticky" bits-after being set, they remain set until cleared using LDMXCSR or FXRSTOR. Bits 7-12 mask individual exceptions when set, initially set by a power-up or reset. Bits 0-5 represent invalid operation, denormal, divide by zero, overflow, underflow, and precision, respectively. For details, see the links "For More Information."

Figure 1. XMM registers overlay the YMM registers.

Figure 2 illustrates the data types used in the Intel® SSE and Intel® AVX instructions. Roughly, for Intel AVX, any multiple of 32-bit or 64-bit floating-point type that adds to 128 or 256 bits is allowed as well as multiples of any integer type that adds to 128 bits.

Figure 2. Intel® AVX and Intel® SSE data types

Instructions often come in scalar and vector versions, as illustrated in Figure 3. Vector versions operate by treating data in the registers in parallel "SIMD" mode; the scalar version only operates on one entry in each register. This distinction allows less data movement for some algorithms, providing better overall throughput.

Figure 3. SIMD versus scalar operations

Data is memory aligned when the data to be operated upon as an n-byte chunk is stored on an n-byte memory boundary. For example, when loading 256-bit data into YMM registers, if the data source is 256-bit aligned, the data is called aligned.

For Intel® SSE operations, memory alignment was required unless explicitly stated. For example, under Intel SSE, there were specific instructions for memory-aligned and memory-unaligned operations, such as the MOVAPD (move-aligned packed double) and MOVUPD (move-unaligned packed double) instructions. Instructions not split in two like this required aligned accesses.

Intel® AVX has relaxed some memory alignment requirements, so now Intel AVX by default allows unaligned access; however, this access may come at a performance slowdown, so the old rule of designing your data to be memory aligned is still good practice (16-byte aligned for 128-bit access and 32-byte aligned for 256-bit access). The main exceptions are the VEX-extended versions of the SSE instructions that explicitly required memory-aligned data: These instructions still require aligned data. Other specific instructions requiring aligned access are listed in Table 2.4 of the Intel® Advanced Vector Extensions Programming Reference (see "For More Information" for a link).

Another performance concern besides unaligned data issues is that mixing legacy XMM-only instructions and newer Intel AVX instructions causes delays, so minimize transitions between VEX-encoded instructions and legacy Intel SSE code. Said another way, do not mix VEX-prefixed instructions and non-VEX-prefixed instructions for optimal throughput. If you must do so, minimize transitions between the two by grouping instructions of the same VEX/non-VEX class. Alternatively, there is no transition penalty if the upper YMM bits are set to zero via VZEROUPPER or VZEROALL, which compilers should automatically insert. This insertion requires an extra instruction, so profiling is recommended.

Intel® AVX Instruction Classes

As mentioned, Intel® AVX adds support for many new instructions and extends current Intel SSE instructions to the new 256-bit registers, with most old Intel SSE instructions having a V-prefixed Intel AVX version for accessing new register sizes and three-operand forms. Depending on how instructions are counted, there are up to a few hundred new Intel AVX instructions.

For example, the old two-operand Intel SSE instruction ADDPS xmm1, xmm2/m128 can now be expressed in three-operand syntax as VADDPS xmm1, xmm2, xmm3/m128 or the 256-bit register using the form VADDPS ymm1, ymm2, ymm3/m256. A few instructions allow four operands, such as VBLENDVPS ymm1, ymm2, ymm3/m256, ymm4, which conditionally copies single-precision floating-point values from ymm2 or ymm3/m256 to ymm1 based on masks in ymm4. This is an improvement on the previous form, where xmm0 was implicitly needed, requiring compilers to free up xmm0. Now, with all registers explicit, there is more freedom for register allocation. Here, m128 is a 128-bit memory location, xmm1 is the 128-bit register, and so on.

Some new instructions are VEX only (not Intel SSE extensions), including many ways to move data into and out of the YMM registers. Examples are the useful VBROADCASTS[S/D], which loads a single value into all elements of an XMM or YMM register, and ways to shuffle data around in a register using VPERMILP[S/D]. (The bracket notation is explained in the Appendix A.)

Intel® AVX adds arithmetic instructions for variants of add, subtract, multiply, divide, square root, compare, min, max, and round on single- and double-precision packed and scalar floating-point data. Many new conditional predicates are also useful for 128-bit Intel SSE, giving 32 comparison types. Intel® AVX also includes instructions promoted from previous SIMD covering logical, blend, convert, test, pack, unpack, shuffle, load, and store. The toolset adds new instructions, as well, including non-strided fetching (broadcast of single or multiple data into a 256-bit destination, masked-move primitives for conditional load and store), insert and extract multiple-SIMD data to and from 256-bit SIMD registers, permute primitives to manipulate data within a register, branch handling, and packed testing instructions.

Future Additions
The Intel® AVX manual also lists some proposed future instructions, covered here for completeness. This is not a guarantee that these instructions will materialize as written.

Two instructions (VCVTPH2PS and VCVTPS2PH) are reserved for supporting 16-bit floating-point conversions to and from single- and double-floating-point types. The 16-bit format is called half-precision and has a 10-bit mantissa (with an implied leading 1 for non-denormalized numbers, resulting in 11-bit precision), 5-bit exponent (biased by 15), and 1-bit sign.

The proposed RDRAND instruction uses a cryptographically secure hardware digital random bit generator to generate random numbers for 16- 32- , and 64-bit registers. On success, the carry flag is set to 1 (CF=1). If not enough entropy is available, the carry flag is cleared (CF=0).

Finally, there are four instructions (RDFDBASE, RDGSBASE, WRFSBASE, and WRGSBASE) to read and write FS and GS registers at all privilege levels in 64-bit mode.

Another future addition is the FMA instructions, which perform operations similar to A = + A * B + C, where either of the plus signs (+) on the right can be changed to a minus sign (?) and the three operands on the right can be in any order. There are also forms for interleaved addition and subtraction. Packed FMA instructions can perform eight single-precision FMA operations or four double-precision FMA operations with 256-bit vectors.

FMA operations such as A = A * B + C are better than performing one step at a time, because intermediate results are treated as infinite precision, with rounding done on store, and thus are more accurate for computation. This single rounding is what gives the "fused" prefix. They are also faster than performing the computation in steps.

Each instruction comes in three forms for the ordering of the operands A, B, and C, with the ordering corresponding to a three-digit extension: form 132 does A = AC + B, form 213 does A = BA + C, and form 231 does A = BC + A. The ordering number is just the order of the operands on the right side of the expression.

Availability and Support

Detecting availability of the Intel® AVX features in hardware requires using the CPUID instruction to query support in the CPU and in the operating system, as detailed later. Second-generation Intel® Core™ processors (Intel® microarchitecture code name Sandy Bridge), released in Q1, 2011, are the first from Intel supporting Intel® AVX technology. These processors will not have the new FMA instructions. For development and testing without hardware support, the free Intel® Software Development Emulator (see "For More Information" for a link) includes support for all these features, including Intel AVX, FMA, PCLMULQDQ, and AES instructions.

To use the Intel AVX extensions reliably in most settings, the operating system must support saving and loading the new registers (with XSAVE/XRSTOR) on thread context switches to prevent data corruption. To help avoid such errors, operating systems supporting Intel AVX-aware context switches explicitly set a CPU bit enabling the new instructions; otherwise, an undefined opcode (#UD) exception is generated when Intel AVX instructions are used.

Microsoft Windows* 7 with Service Pack 1 (SP1) and Microsoft Windows* Server 2008 R2 with SP1-both 32- and 64-bit versions-and later versions of Windows* support Intel AVX save and restore in thread and process switches. Linux* kernels from 2.6.30 (June 2009) and later support Intel AVX, as well.

Detecting Availability and Support
Detection of support for the four areas-Intel® AVX, FMA, AES, and PCLMULQDQ-are similar and require similar steps consisting of checking for hardware and operating system support for the desired feature (see Table 1). These steps are (counting bits starting at bit 0):

Verify that the operating system supports XGETBV using CPUID.1:ECX.OSXSAVE bit 27 = 1.
At the same time, verify that CPUID.1:ECX bit 28=1 (Intel AVX supported) and/or bit 25=1 (AES supported) and/or bit 12=1 (FMA supported) and/or bit 1=1 (PCLMULQDQ) are supported.
Issue XGETBV, and verify that the feature-enabled mask at bits 1 and 2 are 11b (XMM state and YMM state enabled by the operating system).

Table 1. Feature-detection Masks

Feature	Bits to check	Constant
Intel® AVX	28, 27	`018000000H`
VAES	28, 27, and 25	`01A000000H`
VPCLMULQDQ	28, 27, and 1	`018000002H`
FMA	28, 27, and 12	`018001000H`

Example code implementing this process is provided in Listing 1, where the CONSTANT is the value from Table 1. A Microsoft* Visual Studio* C++ intrinsic version is given later.

Listing 1. Feature Detection

INT Supports_Feature()
   { 
   ; result returned in eax
   mov eax, 1
   cpuid
   and ecx, CONSTANT
   cmp ecx, CONSTANT; check desired feature flags
   jne not_supported 
   ; processor supports features
   mov ecx, 0; specify 0 for XFEATURE_ENABLED_MASK register
   XGETBV; result in EDX:EAX
   and eax, 06H
   cmp eax, 06H; check OS has enabled both XMM and YMM state support
   jne not_supported
   mov eax, 1; mark as supported
   jmp done
   NOT_SUPPORTED:
   mov eax, 0 ; // mark as not supported
   done:
   }

Usage

At the lowest programming level, most common x86 assemblers now support Intel® AVX, FMA, AES, and the VPCLMULQDQ instructions, including Microsoft MASM* (Microsoft Visual Studio* 2010 version), NASM*, FASM*, and YASM*. See their respective documentation for details.

For language compilers, Intel® C++ Compiler version 11.1 and later and Intel® Fortran Compilers support Intel® AVX through compiler switches, and both compilers support automatic vectorization of floating-point loops. The Intel C++ Compiler supports Intel AVX intrinsics (use #include to access intrinsics) and inline assembly and even supports Intel AVX intrinsics emulation using #include "avxintrin_emu.h".

Microsoft Visual Studio* C++ 2010 with SP1 and later has support for Intel AVX (see "For More Information") when compiling 64-bit code (use the /arch:AVX compiler switch). It supports intrinsics using the header but not inline assembly. Intel AVX support is also in MASM*, the disassembly view of code, and the debugger views of registers (giving full YMM support).

In the GNU Compiler Collection* (GCC*), version 4.4 supports Intel AVX intrinsics through the same header, . Other GNU toolchain support is found in Binutils 2.20.51.0.1 and later, gdb 6.8.50.20090915 and later, recent GNU Assembler (GAS) versions, and objdump. If your compiler does not support Intel AVX, you can emit the required bytes under many circumstances, but first-class support makes your life easier.

Each of the three C++ compilers mentioned supports the same intrinsic operations to simplify using Intel® AVX from C or C++ code. Intrinsics are functions that the compiler replaces with the proper assembly instructions. Most Intel AVX intrinsic names follow the following format:

_mm256_op_suffix(data_type param1, data_type param2, data_type param3)

where _mm256 is the prefix for working on the new 256-bit registers; _op is the operation, like add for addition or sub for subtraction; and _suffix denotes the type of data to operate on, with the first letters denoting packed (p), extended packed (ep), or scalar (s). The remaining letters are the types in Table 2.

Table 2. Intel® AVX Suffix Markings

Marking	Meaning
`[s/d]`	Single- or double-precision floating point
`[i/u]nnn`	Signed or unsigned integer of bit size nnn, where nnn is 128, 64, 32, 16, or 8
`[ps/pd/sd]`	Packed single, packed double, or scalar double
`epi32`	Extended packed 32-bit signed integer
`si256`	Scalar 256-bit integer

Data types are in Table 3. The first two parameters are source registers, and the third parameter (when present) is an integer mask, selector, or offset value.

Table 3. Intel® AVX Intrinsics Data Types

Type	Meaning
`__m256`	256-bit as eight single-precision floating-point values, representing a YMM register or memory location
`__m256d`	256-bit as four double-precision floating-point values, representing a YMM register or memory location
`__m256i`	256-bit as integers, (bytes, words, etc.)
`__m128`	128-bit single precision floating-point (32 bits each)
`__m128d`	128-bit double precision floating-point (64 bits each)

Some intrinsics are in other headers, such as the AES and PCLMULQDQ being in . Consult your compiler documentation or the web to track down where various intrinsics live.

Microsoft Visual Studio* 2010
For conciseness, the rest of this article uses Microsoft Visual Studio* 2010 with SP1; similar code should work on the Intel® compiler or GCC*. Microsoft Visual Studio* 2010 with SP1 can automatically generate Intel® AVX code if you click Project Properties > Configuration > Code Generation, select Not Set under Enable Enhanced Instruction Set, and then manually add /arch:AVX to the command line under the Command Line entry. As an example of using intrinsics, Listing 2 offers an intrinsic-based Intel AVX feature-detection routine.

Listing 2. Intrinsic-based Feature Detection

// get AVX intrinsics
#include 
// get CPUID capability
#include 

// written for clarity, not conciseness
#define OSXSAVEFlag (1UL<<27)
#define AVXFlag     ((1UL<<28)|OSXSAVEFlag)
#define VAESFlag    ((1UL<<25)|AVXFlag|OSXSAVEFlag)
#define FMAFlag     ((1UL<<12)|AVXFlag|OSXSAVEFlag)
#define CLMULFlag   ((1UL<< 1)|AVXFlag|OSXSAVEFlag)
 
bool DetectFeature(unsigned int feature)
	{
	int CPUInfo[4], InfoType=1, ECX = 1;
	__cpuidex(CPUInfo, 1, 1);       // read the desired CPUID format
	unsigned int ECX = CPUInfo[2];  // the output of CPUID in the ECX register. 
	if ((ECX & feature) != feature) // Missing feature 
		return false; 
	__int64 val = _xgetbv(0);       // read XFEATURE_ENABLED_MASK register
	if ((val&6) != 6)               // check OS has enabled both XMM and YMM support.
		return false; 
	return true;
	}

Mandelbrot Example

To demonstrate using the new instructions, compute Mandelbrot set images using straight C/C++ code (checking to ensure that the compiler did not convert the code to Intel® AVX instructions!) and the new Intel AVX instructions as intrinsics, comparing their performance. A Mandelbrot set is a computationally intensive operation on complex numbers, defined in pseudocode as shown in Listing 3.

Listing 3. Mandelbrot Pseudocode

z,p are complex numbers
for each point p on the complex plane
	z = 0
	for count = 0 to max_iterations
		if abs(z) > 2.0
			break
		z = z*z+p
	set color at p based on count reached

The usual image is over the portion of the complex plane in the rectangle (-2,-1) to (1,1). Coloring can be done in many ways (not covered here). Raise the maximum iteration count to zoom into portions and determine whether a value "escapes" over time.

To really stress the CPU, zoom in and draw the box (0.29768, 0.48364) to (0.29778, 0.48354), computing the grid of counts at multiple sizes and using a max iteration of 4096. The resulting grid of counts, when colored appropriately, is shown in Figure 4.

Figure 4. Mandelbrot set (0.29768, 0.48364) to (0.29778, 0.48354), with max iterations of 4096

A basic C++ implementation to compute the iteration counts is provided in Listing 4. The absolute value of the complex number compared to 2 is replaced with the norm compared to 4.0, almost doubling the speed by removing a square root. For all versions, use single-precision floats to pack as many elements into the YMM registers as possible, which is faster but loses precision compared to doubles when zooming in further.

Listing 4. Simple Mandelbrot C++ Code

// simple code to compute Mandelbrot in C++
#include 
void MandelbrotCPU(float x1, float y1, float x2, float y2, 
                   int width, int height, int maxIters, unsigned short * image)
{
	float dx = (x2-x1)/width, dy = (y2-y1)/height;
	for (int j = 0; j < height; ++j)
		for (int i = 0; i < width; ++i)
		{
			complex c (x1+dx*i, y1+dy*j), z(0,0);
			int count = -1;
			while ((++count < maxIters) && (norm(z) < 4.0))
				z = z*z+c;
			*image++ = count;
		}
}

Test multiple versions for performance: the basic one in Listing 4, a similar CPU version made by expanding the complex types with floats, an intrinsic-based SSE version, and an intrinsic-based Intel® AVX version shown in Listing 5. Each version is tested on image sizes of 128×128, 256×256, 512×512, 1024×1024, 2048×2048, and 4096×4096. The performance of each implementation could likely be improved while retaining its underlying instruction set constraints with more work, but they should be representative of what you can obtain.

The Intel AVX version has been carefully crafted to fit as much as possible into the 16 YMM registers. To help track how you want them to be allocated, the variables are names ymm0 through ymm15. Of course, the compiler allocates registers as it sees fit, but by being careful, you can try to make all computations stay in registers this way. (Actually, from looking at the disassembly, the compiler does not allocate them nicely, and recasting this in assembly code would be a good exercise to anyone learning Intel AVX).

Listing 5. Intel® AVX-intrinsic Mandelbrot Implementation

float dx = (x2-x1)/width;
float dy = (y2-y1)/height;
// round up width to next multiple of 8
int roundedWidth = (width+7) & ~7UL; 
 
float constants[] = {dx, dy, x1, y1, 1.0f, 4.0f};
__m256 ymm0 = _mm256_broadcast_ss(constants);   // all dx
__m256 ymm1 = _mm256_broadcast_ss(constants+1); // all dy
__m256 ymm2 = _mm256_broadcast_ss(constants+2); // all x1
__m256 ymm3 = _mm256_broadcast_ss(constants+3); // all y1
__m256 ymm4 = _mm256_broadcast_ss(constants+4); // all 1's (iter increments)
__m256 ymm5 = _mm256_broadcast_ss(constants+5); // all 4's (comparisons)
 
float incr[8]={0.0f,1.0f,2.0f,3.0f,4.0f,5.0f,6.0f,7.0f}; // used to reset the i position when j increases
__m256 ymm6 = _mm256_xor_ps(ymm0,ymm0); // zero out j counter (ymm0 is just a dummy)
 
for (int j = 0; j < height; j+=1)
{
	__m256 ymm7  = _mm256_load_ps(incr);  // i counter set to 0,1,2,..,7
	for (int i = 0; i < roundedWidth; i+=8)
	{
		__m256 ymm8 = _mm256_mul_ps(ymm7, ymm0);  // x0 = (i+k)*dx 
		ymm8 = _mm256_add_ps(ymm8, ymm2);         // x0 = x1+(i+k)*dx
		__m256 ymm9 = _mm256_mul_ps(ymm6, ymm1);  // y0 = j*dy
		ymm9 = _mm256_add_ps(ymm9, ymm3);         // y0 = y1+j*dy
		__m256 ymm10 = _mm256_xor_ps(ymm0,ymm0);  // zero out iteration counter
		__m256 ymm11 = ymm10, ymm12 = ymm10;        // set initial xi=0, yi=0
 
		unsigned int test = 0;
		int iter = 0;
		do
		{
			__m256 ymm13 = _mm256_mul_ps(ymm11,ymm11); // xi*xi
			__m256 ymm14 = _mm256_mul_ps(ymm12,ymm12); // yi*yi
			__m256 ymm15 = _mm256_add_ps(ymm13,ymm14); // xi*xi+yi*yi
 			
			// xi*xi+yi*yi < 4 in each slot
			ymm15 = _mm256_cmp_ps(ymm15,ymm5, _CMP_LT_OQ);        
			// now ymm15 has all 1s in the non overflowed locations
			test = _mm256_movemask_ps(ymm15)&255;      // lower 8 bits are comparisons
			ymm15 = _mm256_and_ps(ymm15,ymm4);
			// get 1.0f or 0.0f in each field as counters
			// counters for each pixel iteration
			ymm10 = _mm256_add_ps(ymm10,ymm15);        
 
			ymm15 = _mm256_mul_ps(ymm11,ymm12);        // xi*yi 
			ymm11 = _mm256_sub_ps(ymm13,ymm14);        // xi*xi-yi*yi
			ymm11 = _mm256_add_ps(ymm11,ymm8);         // xi <- xi*xi-yi*yi+x0 done!
			ymm12 = _mm256_add_ps(ymm15,ymm15);        // 2*xi*yi
			ymm12 = _mm256_add_ps(ymm12,ymm9);         // yi <- 2*xi*yi+y0	
 
			++iter;
		} while ((test != 0) && (iter < maxIters));
 
		// convert iterations to output values
		__m256i ymm10i = _mm256_cvtps_epi32(ymm10);
 
		// write only where needed
		int top = (i+7) < width? 8: width&7;
		for (int k = 0; k < top; ++k)
			image[i+k+j*width] = ymm10i.m256i_i16[2*k];
 
		// next i position - increment each slot by 8
		ymm7 = _mm256_add_ps(ymm7, ymm5);
		ymm7 = _mm256_add_ps(ymm7, ymm5);
	}
	ymm6 = _mm256_add_ps(ymm6,ymm4); // increment j counter
}

The full code for all versions and a Microsoft Visual Studio* 2010 with SP1 project, including a testing harness, is available at from the links in the "For More Information" section.

The results are shown in Figures 5 and 6. To prevent tying numbers too much to a specific CPU speed, Figure 5 shows performance of each version relative the CPU version, which represents a straightforward non-SIMD C/C++ implementation of the algorithm. As expected, the Intel® SSE version performs almost 4 times as well, because it is doing 4 pixels per pass, and the Intel® AVX version performs almost 8 times as well as the CPU version. Because there is overhead from loops, memory access, less-than-perfect instruction ordering, and other factors, 4- and 8-fold improvements should be about the best possible, so this is pretty good for a first try.

Figure 5. Relative performance across sizes

The second graph in Figure 6 shows that the pixels computed per millisecond are fairly constant over each size; again, the algorithms show almost quadrupling of performance from the CPU to Intel® SSE version and another doubling from the Intel SSE to Intel® AVX version.

Figure 6. Absolute performance across sizes

Conclusion

This article provided a mid-level overview of the new Intel® Advanced Vector Extensions (Intel® AVX). These extensions are similar to previous Intel® SSE instructions but offer a much larger register space and add some new instructions. The Mandelbrot example shows performance gains over previous technology in the amount expected. For full details, be sure to check out the Intel Advanced Vector Extensions Programming Reference (see "For More Information" for a link).

Happy hacking!

For More Information

Intel® Advanced Vector Extensions Programming Reference at http://software.intel.com/en-us/avx/

Federal Information Processing Standards Publication 197, "Announcing the Advanced Encryption Standard," at http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf

The IEEE 754-2008 floating-point format standard at http://en.wikipedia.org/wiki/IEEE_754-2008

Floating-Point Support for 64-Bit Drivers at http://msdn.microsoft.com/en-us/library/ff545910.aspx

Wikipedia's entry on the Mandelbrot set at http://en.wikipedia.org/wiki/Mandelbrot_set

Intel® Software Development Emulator at http://software.intel.com/en-us/articles/intel-software-development-emulator

The complete Mandelbrot Intel® AVX implementation for download at http://www.lomont.org

About the Author

Chris Lomont works as a research engineer at Cybernet Systems, working on projects as diverse as quantum computing algorithms, image processing for NASA, developing security hardware for United States Homeland Security, and computer forensics. Before that he obtained a PhD. in math from Purdue, three Bachelors degrees in physics, math, and computer science, worked as a game programmer, did brief stints in financial modeling, robotics work, and various consulting roles. The rest of his time is spent hiking with his wife, watching movies, giving talks, recreational programming, doing math research, learning more physics, playing music, and performing various experiments. Visit his website www.lomont.org or his electronic gadget site www.hypnocube.com.

Appendix A: Instruction Set Reference

Many instructions come in packed or scalar form, meaning that they work on multiple parallel elements or on a single element in the register-a distinction marked as [P/S]. Entry lengths come in double or single precision for floating-point (doubles and singles, for brevity); marked [D/S]; and the integer forms byte, word, doubleword, and quadword, marked [B/W/D/Q]. Integer forms also sometimes come in signed or unsigned forms, marked [S/U]. Some instructions work on high or low portions of registers, marked as [H/L]; other optional components are in the tables. Instructions coming in Intel® SSE form and Intel® AVX form are prefixed with a (V) for the Intel® AVX form, allowing three operands and 256-bit register support. Entries in square brackets ([]) are required; entries in parentheses (()) are optional.

Examples:

(V)ADD[P/S][D/S] is the addition of packed or scalar, double or single, with eight possible forms-VADDPD, VADDPS, VADDSD, VADDSS, and versions without the leading V.
(V)[MIN/MAX][P/S][D/S] represents 16 different instructions for a min or max of packed or scalar of double or single precision.

The next table represents the multiple comparison types. VEX-prefixed instructions have 32 comparison types; non-VEX-prefixed comparisons only allow those eight types in parentheses. Each comparison type comes in multiple flavors, where O = ordered, U = unordered, S = signaling, and Q = non-signaling. Ordered/unordered tells whether the comparison is false or true if one operand is NaN (Not-a-Number in floating point, which happens when something failed during the computation, such as divide by 0 or the square root of a negative number). Signaling/non-signaling states whether an exception is fired when at least one operand is QNaN (Quiet Not-a-Number-useful for error trapping).

Type	Flavors	Meaning
`EQ`	`(OQ), UQ, OS, US`	Equal
`LT`	`(OS), OQ`	Less than
`LE`	`(OS), OQ`	Less than or equal to
`UNORD`	`(Q), S`	Tests for unordered (NaN)
`NEQ`	`(UQ), US, OQ, OS`	Not equal
`NLT`	`(US), UQ`	Not less than
`NLE`	`(US), UQ`	Not less than or equal to
`ORD`	`(Q), S`	Tests for ordered (not NaN)
`NGE`	`US, UQ`	Not greater than or equal to
`NGT`	`US, UQ`	Not greater than
`FALSE`	`OQ, OS`	Comparison is always false
`GE`	`OS, OQ`	Greater than or equal to
`GT`	`OS, OQ`	Greater than
`TRUE`	`UQ, US`	Comparison is always true

Finally, here are all the Intel® AVX instructions:

Arithmetic	Description
`(V)[ADD/SUB/MUL/DIV][P/S][D/S]`	Add/subtract/multiply/divide packed/scalar double/single
`(V)ADDSUBP[D/S]`	Packed double/single add and subtract alternating indices
`(V)DPP[D/S]`	Dot product, based on immediate mask
`(V)HADDP[D/S]`	Horizontally add
`(V)[MIN/MAX][P/S][D/S]`	Min/max packed/scalar double/single
`(V)MOVMSKP[D/S]`	Extract double/single sign mask
`(V)PMOVMSKB`	Make a mask consisting of the most significant bits
`(V)MPSADBW`	Multiple sum of absolute differences
`(V)PABS[B/W/D]`	Packed absolute value on bytes/words/doublewords
`(V)P[ADD/SUB][B/W/D/Q]`	Add/subtract packed bytes/words/doublewords/quadwords
`(V)PADD[S/U]S[B/W]`	Add packed signed/unsigned with saturation bytes/words
`(V)PAVG[B/W]`	Average packed bytes/words
`(V)PCLMULQDQ`	Carry-less multiplication quadword
`(V)PH[ADD/SUB][W/D]`	Packed horizontal add/subtract word/doubleword
`(V)PH[ADD/SUB]SW`	Packed horizontal add/subtract with saturation
`(V)PHMINPOSUW`	Min horizontal unsigned word and position
`(V)PMADDWD`	Multiply and add packed integers
`(V)PMADDUBSW`	Multiply unsigned bytes and signed bytes into signed words
`(V)P[MIN/MAX][S/U][B/W/D]`	Min/max of packed signed/unsigned integers
`(V)PMUL[H/L][S/U]W`	Multiply packed signed/unsigned integers and store high/low result
`(V)PMULHRSW`	Multiply packed unsigned with round and shift
`(V)PMULHW`	Multiply packed integers and store high result
`(V)PMULL[W/D]`	Multiply packed integers and store low result
`(V)PMUL(U)DQ`	Multiply packed (un)signed doubleword integers and store quadwords
`(V)PSADBW`	Compute sum of absolute differences of unsigned bytes
`(V)PSIGN[B/W/D]`	Change the sign on each element in one operand based on the sign in the other operand
`(V)PS[L/R]LDQ`	Byte shift left/right amount in operand
`(V)SL[L/AR/LR][W/D/Q]`	Bit shift left/arithmetic right/logical right
`(V)PSUB(U)S[B/W]`	Packed (un)signed subtract with (un)signed saturation
`(V)RCP[P/S]S`	Compute approximate reciprocal of packed/scalar single precision
`(V)RSQRT[P/S]S`	Compute approximate reciprocal of square root of packed/scalar single precision
`(V)ROUND[P/S][D/S]`	Round packed/scalar double/single
`(V)SQRT[P/S][D/S]`	Square root of packed/scalar double/single
`VZERO[ALL/UPPER]`	Zero all/upper half of YMM registers

Comparison	Description
`(V)CMP[P/S][D/S]`	Compare packed/scalar double/single
`(V)COMIS[S/D]`	Compare scalar double/single, set EFLAGS
`(V)PCMP[EQ/GT][B/W/D/Q]`	Compare packed integers for equality/greater than
`(V)PCMP[E/I]STR[I/M]`	Compare explicit/implicit length strings, return index/mask

Control	Description
`V[LD/ST]MXCSR`	Load/store MXCSR control/status register
`XSAVEOPT`	Save processor extended states optimized

Conversion	Description
`(V)CVTx2y`	Convert type x to type y, where x and y are chosen from `DQ and P[D/S],` `[P/S]S and [P/S]D, or` `S[D/S] and SI.`

Load/store	Description
`VBROADCAST[SS/SD/F128]`	Load with broadcast (loads single value into multiple locations)
`VEXTRACTF128`	Extract 128-bit floating-point values
`(V)EXTRACTPS`	Extract packed single precision
`VINSERTF128`	Insert packed floating-point values
`(V)INSERTPS`	Insert packed single-precision values
`(V)PINSR[B/W/D/Q]`	Insert integer
`(V)LDDQU`	Move quad unaligned integer
`(V)MASKMOVDQU`	Store selected bytes of double quadword with `NT Hint`
`VMASKMOVP[D/S]`	Conditional SIMD packed load/store
`(V)MOV[A/U]P[D/S]`	Move aligned/unaligned packed double/single
`(V)MOV[D/Q]`	Move doubleword/quadword
`(V)MOVDQ[A/U]`	Move double to quad aligned/unaligned
`(V)MOV[HL/LH]P[D/S]`	Move high-to-low/low-to-high packed double/single
`(V)MOV[H/L]P[D/S]`	Move high/low packed double/single
`(V)MOVNT[DQ/PD/PS]`	Move packed integers/doubles/singles using a non-temporal hint
`(V)MOVNTDQA`	Move packed integers using a non-temporal hint, aligned
`(V)MOVS[D/S]`	Move or merge scalar double/single
`(V)MOVS[H/L]DUP`	Move single odd/even indexed singles
`(V)PACK[U/S]SW[B/W]`	Pack with unsigned/signed saturation on bytes/words
`(V)PALIGNR`	Byte align
`(V)PEXTR[B/W/D/Q]`	Extract integer
`(V)PMOV[S/Z]X[B/W/D][W/D/Q]`	Packed move with sign/zero extend (only `up in length, DD, DW,` etc. disallowed)

Logical	Description
`(V)[AND/ANDN/OR]P[D/S]`	Bitwise logical `AND/AND NOT/OR` of packed double/single values
`(V)PAND(N)`	Logical `AND (NOT)`
`(V)P[OR/XOR]`	Bitwise `logical OR/exclusive OR`
`(V)PTEST`	Packed bit test, set zero flag if bitwise `AND` is all `0`
`(V)UCOMIS[D/S]`	Unordered compare scalar doubles/singles and set `EFLAGS`
`(V)XORP[D/S]`	Bitwise logical `XOR` of packed double/single

Shuffle	Description
`(V)BLENDP[D/S]`	Blend packed double/single; selects elements based on mask
`(V)BLENDVP[D/S]`	Blend values
`(V)MOVDDUP`	Copies even values to all values
`(V)PBLENDVB`	Variable blend packed bytes
`(V)PBLENDW`	Blend packed words
`VPERMILP[D/S]`	Permute double/single values
`VPERM2F128`	Permute floating-point values
`(V)PSHUF[B/D]`	Shuffle packed bytes/doublewords based on immediate value
`(V)PSHUF[H/L]W`	Shuffle packed high/low words
`(V)PUNPCK[H/L][BW/WD/DQ/QDQ]`	Unpack high/low data
`(V)SHUFP[D/S]`	Shuffle packed double/single
`(V)UNPCK[H/L]P[D/S]`	Unpack and interleave packed/scalar doubles/singles

AES	Description
`AESENC/AESENCLAST`	Perform one round of AES encryption
`AESDEC/AESDECLAST`	Perform one round of AES decryption
`AESIMC`	Perform the AES `InvMixColumn` transformation
`AESKEYGENASSIST`	AES Round Key Generation Assist

Future Instructions	Description
`[RD/WR][F/G]SBASE`	Read/write FS/GS register
`RDRAND`	Read random number (into r16, r32, r64)
`VCVTPH2PS`	Convert 16-bit floats to single precision floating-point values
`VCVTPS2PH`	Convert single-precision values to 16-bit floating-point values

FMA	Each [z] is the string 132 or 213 or 231, giving the order the operands A,B,C are used in: 132 is A=AC+B 213 is A=AB+C 231 is A=BC+A
`VFMADD[z][P/S][D/S]`	Fused multiply add A = r1 * r2 + r3 for packed/scalar of double/single
`VFMADDSUB[z]P[D/S]`	Fused multiply alternating add/subtract of packed double/single A = r1 * r2 + r3 for odd index, A = r1 * r2-r3 for even
`VFMSUBADD[z]P[D/S]`	Fused multiply alternating subtract/add of packed double/single A = r1 * r2-r3 for odd index, A = r1 * r2+r3 for even
`VFMSUB[z][P/S][D/S]`	Fused multiply subtract A = r1 * r2-r3 of packed/scalar double/single
`VFNMADD[z][P/S][D/S]`	Fused negative multiply add of packed/scalar double/single A = -r1 * r2+r3
`VFNMSUB[z][P/S][D/S]`	Fused negative multiply subtract of packed/scalar double/single A = -r1 * r2-r3

Chris Lomont

06-21-2011

Tech Articles

Intel® SSE

Intel® AVX

Intel® AVX improves upon Intel® SSE by adding 256-bit registers and additional instructions. This article shows how to detect Intel AVX, and deconstructs Intel AVX function names and explains Intel AVX intrinsics. It showcases a fractal calculation that achieves nearly 2x performance speedup relative to the Intel SSE version.

Practical Intel® AVX Optimization on 2nd generation Intel® Core™ Processors

Mon, 09 May 2011 00:00:00 -0700

Download Article

Download Practical Intel® AVX Optimization on 2nd generation Intel® Core™ Processors [PDF 672KB]

Introduction

The introduction of the 2nd generation Intel® Core™ processor family (Intel® microarchitecture code name Sandy Bridge) includes a tremendous opportunity to boost application performance. To many developers, those opportunities might be hidden under the hood, as they might not know how to utilize this huge performance treasure. The first effective way to boost performance is by utilizing multi-core CPUs. Another opportunity is Intel® Advanced Vector Extension (Intel® AVX), the new vectorization capabilities for floating point operations. Intel AVX is a new 256-bit instruction set extension to Intel® Streaming SIMD Extensions (Intel® SSE). It was released as part of the Intel microarchitecture code name Sandy Bridge and will be present in platforms ranging from notebooks to servers. The ability to work with 256-bit vectors enables operating on 8 float elements per iteration instead of a single element. This data parallelism can give great computation power to boost application programmers who know how to use it. The following article describes a simple methodology to utilize Intel AVX inside the 2nd generation Intel Core processor family (Intel architecture codename Sandy Bridge). The paper shares insights and methods gained during a shared project by Waves (http://www.waves.com/) and Intel Software and Services Group on optimizing a Linear Phase FIR Filter using Intel AVX. It focuses on how Intel AVX technology can be exploited to achieve a significant performance boost. The paper includes concrete code examples to demonstrate the optimization.

Identify your hotspots

A hotspot is where the CPU spends significant amount of time in an application. The first step of optimization work must be to find the hotspots. Don't skip this stage; otherwise you might optimize parts of the code which have almost no impact on the total application performance. Shortly before the introduction of the 2nd generation Intel® Core™ microprocessor, Intel released Intel® VTune™ Amplifier XE 2011 as part of the Intel® Parallel Studio XE 2011 tool suite. Intel Parallel Studio XE includes all the capabilities of Intel® Parallel Amplifier, plus a number of advanced capabilities for developers who need to dig deeper. Intel VTune Amplifier XE can pinpoint your hotspots, provide insights on CPU utilization, and show you where your parallelism is poor. There is no need for guessing about the location of performance bottlenecks in the application anymore.

One important way to optimize the hotspot is by multi-threading. If your desktop CPU has 4 cores (Quad Core) and supports Intel® Hyper-Threading Technology, it actually means you can run 8 simultaneous threads. If your application is still single-threaded, you have a theoretical opportunity to boost your performance by 8x. Learn more about parallel programming at http://software.intel.com/en-us/parallel/.

A new capability just introduced with the 2nd generation Intel® Core™2 Duo processor is Intel® Advanced Vector Extensions (Intel® AVX). This article describes how to use Intel AVX to improve performance.

SIMD (Single Instruction, Multiple Data) with Intel® AVX implementation

Before Intel microarchitecture code name Sandy Bridge, SIMD vectorization was provided by Intel® Streaming SIMD Extensions (Intel® SSE). Intel SSE is a technique for micro level data-level parallelism on x86 architecture. Intel SSE instructions use eight 128-bit registers, XMM0 through XMM7, where uniform type data can be packed. Intel SSE instructions operate on all data items in parallel. Intel SSE with 128-bit vectors enables operating on 4 float elements per iteration instead of a single element.

Figure 1: Going from Scalar to Intel® SSE can provide up to 4x faster performance

Intel® AVX brings new vectorization capabilities for floating point operations. The new ability to work with 256-bit vectors enables operating on 8 float elements per iteration instead of a single element.

Figure 2: Going from Scalar to Intel® AVX can provide up to 8x faster performance

It is easier to understand when looking at an example. Following is a representative inner loop from a Linear Phase FIR Filter that is often a hotspot. Let's implement it with Intel SSE first and then Intel AVX. In this case, it was possible to obtain up to a 2.1x increase in speed when using Intel AVX over Intel SSE.

Implementing A Linear Phase FIR Filter using Intel® AVX

The following code example demonstrates the use of a specific linear phase audio FIR filter. A FIR (Finite Impulse Response) filter is a vector of filter taps values multiplied and accumulated (convolution) with history input signal samples, which may be used, for example, to achieve EQ effect on an audio signal.

A linear phase filter is characterized by a symmetric shape of the filter taps values. Since the filter length is odd, the symmetry is around a single center tap. In this example, we use a double precision floating point implementation. Recognizing that floating point values are subject to large errors when a large value is summed to a much smaller value, it is more accurate to sum the values by their expected magnitude, from the smallest to the largest. In the code for filter implementation, this is approximately true when we sum the values from the edges towards the center. Given the symmetric nature of the filter, it is better to read only the first half of the filter taps - where the second half taps are mirrored.

Filtering an audio signal with a FIR filter requires the convolution of the audio signal with the FIR filter.
The convolution is implemented by iteration, one iteration per input sample, i.e., for each input sample, N filter taps are dot-multiplied by the last N input signal samples and summed to an accumulator. Infinite length input signal processors (such as in real-time processing) must use a memory FIFO buffer retaining old input samples.

A FIFO buffer (First-In-First-Out) data structure is implemented using an array and mask. The mask is of 2^M-1 value (e.g., 0x00FFF) and is used to compute the FIFO access index using bit-wise AND operation rather than conditions, as the bit-wise AND is a faster implementation.

The basic algorithm works like this:

For each input signal sample:

A. Insert sample to buffer, Advance the read/write offsets using a mask operation.
B. Loop by half of the filter length

1. Load 4 filter taps.
2. Load 4 input samples from buffer
3. Multiply and accumulate (1)+(2).
4. Mirror the 4 filter taps.
5. Load 4 new input samples from buffer.
6. Multiply and accumulate (4)+(5).
7. Advance memory buffer access indexes.

C. Multiply and accumulate center tap.

Note: Data Type is Double (64bit)

Original C implementation
The code example below shows only the implementation of the internal loop (B). It loads one filter tap per iteration.


 for (int j=0 ; j

Note: In this example, firHalfLength equals 1023 defining the iterations of the internal loop. In addition, the internal loop was nested with an external loop of 1,000,000 iterations in order to get meaningful numbers in the standalone measurements.

The original C code above runs in ~0.63 sec on Intel® Core™ i7-2820QM CPU @2.3 GHz, RAM 4GB, Windows* 7 Ultimate with Service Pack 1.

Intel® SSE 128-bit implementation
Converting the code into Intel SSE loads two filter taps per iteration. The new code looks like this:


__m128d DllVal, FIRCoef, mulVal;

for (int i = 0; i < firHalfLength; i += 2) //Operate on 2 elements at a time 
{
	FIRCoef = _mm_load_pd(pFIRBuf+i);
			
	//acc1
	DllVal = _mm_load_pd(pDllBuf+lFIRIndexRev);
	mulVal = _mm_mul_pd(FIRCoef, DllVal);
	acc1 = _mm_add_pd(acc1, mulVal);

	//acc2
	DllVal = _mm_load_pd(pDllBuf+lFIRIndex);
	DllVal = _mm_shuffle_pd(DllVal, DllVal, 0x1);
	mulVal = _mm_mul_pd(FIRCoef, DllVal);
	acc2 = _mm_add_pd(acc2, mulVal);
		
	lFIRIndex -= 2;
	lFIRIndex = (lFIRIndex & lMask);
	lFIRIndexRev += 2;
	lFIRIndexRev = (lFIRIndexRev & lMask);
}

The SSE 128-bit code runs in ~0.40 sec on the exact same hardware as used to run the original code. This is already a speedup of 1.58x over the original code. However, Intel AVX can provide an additional speedup!

Intel® AVX Implementation

Figure 3: Benefits of Intel® AVX.

This implementation uses basic Intel AVX intrinsics code. The use of Intel AVX 256-bit vectors enables twice the number of elements processed per iteration in the loop - thus operating on 4 elements at a time. The main difference when comparing the Intel SSE intrinsics to the new Intel AVX intrinsics is clear when looking at the instruction names, e.g., _mm_load_pd ? _mm256_load_pd. Unlike Intel SSE intrinsics, the Intel AVX intrinsics can support up to 4 operands. In addition, there are different registers type __m128 ? __m256 with different width (128bit ? 256bit). The example uses the basic Intel AVX instructions such as _mm256_mul_pd and _mm256_add_pd.

Intel® AVX - Arranging the Data

In this example we also use "cross-lane" shuffle "mm256_permute2f128_pd" to arrange the data and eliminate the number of loads. More details on the "lane" concept will be discussed below. Since the index is decremented by (-4) in each iteration, we need to arrange each of the 4 elements being loaded in an opposite consecutive order, as described below:

Intel® AVX - Implementation


__m256d DllVal, FIRCoef, mulVal;
__m128d tmph,tmpl,tmplsh,tmphsh;

for (int i = 0; i < firHalfLength; i += 4) //Operate on 4 elements at a time
{
	
	FIRCoef = _mm256_load_pd(pFIRBuf+i);

	//acc1
	DllVal = _mm256_load_pd(pDllBuf+lFIRIndexRev);
	mulVal = _mm256_mul_pd(FIRCoef, DllVal);
	acc1 = _mm256_add_pd(acc1, mulVal);

	//acc2
	DllVal = _mm256_load_pd(pDllBuf+lFIRIndex);
	DllVal = _mm256_permute2f128_pd (DllVal,DllVal ,0x1); // Cross lane shuffle
	DllVal = _mm256_permute_pd(DllVal, 0x5);
	mulVal = _mm256_mul_pd(FIRCoef, DllVal);
	acc2 = _mm256_add_pd(acc2, mulVal);

	lFIRIndex -= 4;
	lFIRIndex = (lFIRIndex & lMask);
	lFIRIndexRev += 4;
	lFIRIndexRev = (lFIRIndexRev & lMask);
}

The Intel® AVX 256-bit code runs in ~0.19 sec on the exact same hardware as used to run the original code.
Thus, the Intel AVX code shows a 2.1x improvement over the Intel SSE code.

Intel® AVX Lanes
Intel AVX implements 256-bit registers, divided into two lanes of 128 bits: a low lane and a high lane. Most of the 256-bit Intel AVX instructions are defined as in-lane: the destination elements in each lane are calculated using source elements only from the same lane. There are only a few cross-lane instructions such as the cross-lane" shuffle "mm256_permute2f128_pd" we used in the example.

Figure 4: Use of Intel® AVX lanes

Strided Load
The strided load technique is a programming method that uses Intel AVX instructions and is useful for algorithms that involve unsupported cross-lane shuffles. The method describes how to arrange data to avoid cross-lane shuffles. The main idea is to use 128-bit loads in a way that mimics the corresponding Intel SSE algorithm, and enables the 256-bit Intel AVX instructions to execute iterations i of the loop in the low lanes and the iteration and i+1 in the high lanes. This method eases the porting of Intel SSE algorithms, since 128 bits is still a natural data type for many algorithms, such as RGBA in single precision and Real and Imaginary in double precision complex. Intel AVX adds new in-lane data movement instructions such as _mm_shuffle_pd , _mm256_insertf128_pd and_mm256_castpd128_pd256 to convert 256-bit to 128-bit width. See examples of Strided load in the presentation: http://software.intel.com/file/29515 slide #59.

Cache alignment

Aligning data to vector length is always recommended. When using Intel SSE and Intel SSE2 instructions, loaded data should be aligned to 16 bytes. Similarly, to achieve best results use Intel AVX instructions on 32-byte vectors that are 32-byte aligned. The use of Intel AVX instructions on unaligned 32-byte vectors means that every second load will be across a cache-line split, since the cache line is 64 bytes. This doubles the cache line split rate compared to Intel SSE code that uses 16-byte vectors. A high cache-line split rate in memory-intensive code is extremely likely to cause performance degradation. For that reason, it is highly recommended to align the data to 32 bytes for use with Intel AVX.

Figure 5: Cache Line Split When Address is not 32-byte Aligned

It is important to make every effort to align the data to 32 bytes in advance to avoid potential performance degradation. However, there are instances where the programmer is not able to force the data structure to be 32-byte alignment. For example, this can happen when working with third-party code. In cases where 32-byte alignment is not possible, change the load instruction to unaligned load "_mm256_loadu_pd". This works, but it is clearly a compromise on performance. When we ran the sample Intel® AVX implementation of the linear phase audio FIR filter, we noticed a penalty degradation of more than 20% when working with misaligned data and using the loadu and storeu instructions.

Intel® Compilers

The Intel® C++ Compiler version 11.1 supports Intel® AVX both for Windows* and Linux*. In many cases (if there are not too many dependencies) it can generate Intel AVX instructions automatically. The Intel Composer XE (Intel® C++ Compiler version 12), part of Intel® Parallel Studio XE, has additional improvements for Intel AVX. On Windows*, use the command line switch /QxAVX. On Linux*, use -xavx. The switches /QaxAVX (Windows*) and -axavx (Linux*) may be used to build applications that will take advantage of Intel AVX instructions on Intel® processor-based systems that support these, but will use only Intel SSE instructions on other systems.

The Intel® C++ Compiler supports Intel AVX-based intrinsics via the header file immintrin.h. To see which loops have been vectorized, use the switch /Qvec-report1 (windows) or -vec-report1 (Linux).

Important notice for using Intel® AVX:

In order to take advantage of Intel AVX on Windows* 7, Service Pack 1 must have been installed. Microsoft* Windows* 7 SP1 Beta is available from Microsoft at: http://technet.microsoft.com/hi-in/evalcenter/ff183870.aspx Upon upgrade to Windows* 7 SP1, Intel AVX optimized software will automatically "opt-in" and take advantage of Intel AVX.
The Intel® Integrated Performance Primitives (Intel® IPP) library has been optimized for a variety of SIMD instruction sets. Automatic "dispatching" detects the SIMD instruction set that is available on the running processor and selects the optimal SIMD instructions for that processor. It is recommended to review Intel® IPP functions which are already optimized for Intel AVX. This can speed up the optimization and the development cycle. Below is a link for Intel IPP version 7 supporting Intel AVX that may be useful for optimization: http://software.intel.com/en-us/articles/intel-ipp-functions-optimized-for-intel-avx-intel-advanced-vector-extensions/
Download the Intrinsics Guide for Intel AVX from http://software.intel.com/en-us/avx/. It is an interactive reference tool that allows programmers to search, sort and reference all of Intel's SIMD instructions. The guide also provides descriptions for each instruction, data type support as well as the intrinsic mnemonics.
See additional Intel AVX papers at http://software.intel.com/en-us/avx/.

Summary and Results

Intel® AVX provides an opportunity to boost application performance on 2nd generation Intel® Core™ processor-based platforms. First, the main hotspots of the application must be identified. Next, Intel AVX vectorization provides an optimization opportunity which is done per core. This optimization has a great impact when using all cores/threads with Intel AVX. Before using direct Intel AVX coding, it is recommended to review the great variety of relevant Intel® IPP version 7 functions which are already optimized for many of existing processors and can speed up the development cycle.

Wave's Director of Research and Development Itai Neoran, said of AVX: "Similar double-precision FIR filters are widely used by Waves in real-time sound processing applications, in both music production studios and live-concert sound-processing. Since extensive sound processing is needed on each and every instrumental/vocal track, and since typical applications run hundreds such tracks in real-time, AVX parallelization allows sound engineers worldwide to use more simultaneous tracks, which until now could only be achieved with dedicated DSP hardware."

In this article we showed how to use direct Intel AVX coding on a simple loop which was identified as a hotspot. Moving from original C implementation through Intel® SSE 128-bit to the Intel® microarchitecture code name Sandy Bridge 256-bit implementation, we were able to get impressive improvement. We've seen a 2.1x improvement from Intel SSE 128-bit code to Intel AVX 256-bit code for the hotspot that enabled the speedup of the entire FIR Filter application from Waves.


Original C code	SSE 128 bit code	AVX 256 bit code
~0.63 sec	~0.40 sec	~0.19 sec

Table 1: Intel® AVX code vs. Intel® SSE: 2.1x Improvement

About the Authors

Guy Ben-Haim:
Guy Ben-Haim is a senior application engineer in Intel corporation at the EMEA Software and Services Group (SSG). Guy works on optimizing applications to take advantage of the latest Intel software and hardware innovations. Guy provides technical training, consultation, and hands-on assistance to SW developers in areas of software optimization and parallel programming. Guy has been working in Intel for the last 10 years as a senior CPU designer where he participated in the development of the first dual core processor and other Centrino mobile processors. Prior to Intel, Guy worked for several startup up companies in the fields of communication, computers HW and semiconductors. Guy holds a B.Sc. from Tel Aviv University and M.B.A from Technion - Israel Institute of Technology.

Itai Neoran:
Itai Neoran is the director of research and development in Waves Audio Ltd. (Waves), Israel, a Grammy-awarded maker of software and hardware tools for digital audio processing in the music industry. During the last 17 years, Itai is leading a group of researchers and engineers who develop audio processing algorithms for professional and home users, and implement them in real-time, particularly on Intel CPUs. Itai also develops methods for acceleration of audio algorithms using parallel computation such as SSE and AVX, while keeping high bit accuracy and low input to output latency. Prior to Waves, Itai worked as a researcher in several companies employing signal processing and image processing algorithms. Itai holds an M.Sc in Electrical Engineering from the Technion Institute, Israel, and a B.Sc. in Computer Science from The Hebrew University of Jerusalem.

Ishay Tubi:
Ishay Tubi is a senior DSP engineer at Waves Audio Ltd., Israel. For the past 5 years, Ishay has been developing professional high-end audio signal processing algorithms. His command of FIR filters and Intel's parallel programing facilitates acceleration on various Waves Audio Plugins. Ishay holds a B.Sc. degree in computer science from Tel-Hai Academic College and has 15 years of experience in computer programming. Ishay is also the CTO and co-founder of MixBox Audio - an iOS Interactive music applications software company who's goal is to enable consumers without a musical background to mix and play music.

Guy Ben Haim

05-09-2011

Tech Articles

Intel® VTune™ Amplifier XE

Performance Analysis

Intel® AVX

Intel® SSE

With the release of the 2nd generation Intel® Core™ processors (Intel® microarchitecture code name Sandy Bridge), there is a tremendous opportunity to boost application performance. This article discusses how one can identify hotspots in their applications and optimize them using SIMD with Intel® AVX implementation.

Developing Intel® AVX Optimized Microsoft* Real-Time Audio (MSRTA) Codec using Intel® IPP

Thu, 21 Apr 2011 00:00:00 -0700

Download Article

Download Developing Intel® AVX Optimized Microsoft* Real-Time Audio (MSRTA) Codec using Intel® IPP [PDF 284KB]

Introduction

This white paper describes how Intel® Integrated Performance Primitives (Intel® IPP) can provide the building blocks to develop the Microsoft* Real Time Audio (MSRTA) codec on the latest Intel® microarchitecture, code name Sandy Bridge. MSRTA is specifically designed for real-time two-way Voice over IP (VoIP) applications. We developed a speech codec that is fully bitstream compliant with the Microsoft RTAudio codec with comparable quality. It has been implemented using Intel® IPP.

We describe how to use Intel® IPP to build the Intel® Advanced Vector Extensions (Intel® AVX) optimized Microsoft* Real Time Audio Codec for VoIP applications. We provide performance results for Intel® microarchitecture code name Sandy Bridge.

Unified Speech Component

The Unified Speech Component (USC) interface is a C language framework designed for implementation of speech codecs, echo cancellers, and other voice processing modules using the Intel® IPP library. Most of the speech codec standards mentioned in the above section utilize this extensible USC interface.

The purpose of the USC interface is to provide unified access to an algorithm module, the access being independent of the algorithm internals. The USC interface also enables binaries to be easily integrated into existing software applications. Decoupling the interface and the algorithm details enables making the development of system components independent of the algorithm implementation.

For more information, refer to the Unified Speech Component Interface manual (uscmanual.pdf) in the speech coding sample (part of the Intel® IPP main sample).

Intel® IPP functions optimized for Intel® Advanced Vector Extensions

Intel® Advanced Vector Extensions (Intel® AVX) is a 256-bit instruction set extension to Intel® Streaming SIMD Extensions (Intel® SSE), designed to provide even higher performance for applications that are floating-point intensive. Intel AVX adds new functionality to the existing Intel SIMD instruction set (based on Intel® SSE), and includes a more compact SIMD encoding format.

The Intel® IPP library has been optimized for a variety of SIMD instruction sets. Automatic "dispatching" detects the SIMD instruction set that is available on the running processor and selects the optimal SIMD instructions for that processor. Refer to Understanding CPU Dispatching in the Intel® IPP Library for more information regarding dispatching.

Intel® AVX optimization in the Intel® IPP library consists of "hand-optimized" and "compiler-tuned" functions-code that has been directly optimized for the Intel® AVX instruction set.

For more information on Intel® IPP functions optimized for Intel® AVX, refer to the article Intel® IPP Functions Optimized for Intel® AVX.

Microsoft* Real-Time Audio Codec (MSRTA)

RTAudio* is the preferred Microsoft® Real-Time audio codec, and is used by Microsoft Lync Server* (formerly Microsoft Office Communications Server*) and other communications applications like Microsoft Lync* (formerly Microsoft Office Communicator*) and Microsoft LiveMeeting* Console.

To get more information about the Microsoft RTAudio Codec, refer to Overview of the Microsoft RTAudio* Speech Codec.

Intel® IPP Real Time Audio functions

By combing Intel® IPP RT audio functions (refer to Appendix B), it is possible to construct a speech codec compliant with the Microsoft RTAudio* codec. The primitives are primarily designed to implement the well-defined, computationally expensive core operations that comprise the codec portion of the RTAudio system.

USC MSRTA Codec

The USC MSRTA codec supports 16-bit wideband 16000 Hz and narrowband 8000 Hz PCM mono signal compression and decompression, with 20ms frame lengths at bitrates 8800 bps and 18000 bps respectively.

To understand the usage of Intel® IPP in developing and measuring the USC MSRTA codec, please download the following free code samples: Code Samples for the Intel® Integrated Performance Primitives (Intel® IPP) Library

Extract all files in w_ipp -sample_*.zip to the desired folder. Make sure to preserve the directory structure. The files for the USC Speech Codec, USC Echo Cancellation, UMC Speech RTP codec, and USC Filter and tones samples will be found in the

../ ipp_samples/speech-codecs folder.

How to build the source code

Set system environment variable IPPROOT
Open solution/project file in related Microsoft* Visual Studio
Select the configuration/platform you need
Build all projects in Microsoft Visual Studio*
Run the codec

To run the sample for encode or for decode, use the following command line:

usc_speech_rtp_codec.exe [options]

Depending on which of the two formats-WAVE or RTPDump-has the input file , either an encode or a decode operation will be performed respectively. For an encode operation, the output file is stored in RTPDump format; for decode, the ouput file is stored in WAVE format. Option list:

*-format*	- codec option
-r	- bitrate option (mandatory)
-v	-Voice Activity Detector(VAD) enabled. Default: VAD disabled

Codec name	Supported bitrate, in bps	Codec description
IPP_MSRTAnb_FP	8800	Narrowband 8000 KHz MSRTA codec, floating-point implementation
IPP_MSRTAwb_FP	18000	Wideband 16000 KHz MSRTA codec, floating-point implementation

To enable RT Audio codecs IPP_MSRTAnb_FP and IPP_MSRTAwb_FP, compile the sample source with _USC_MSRTA definition and link with the IPP RTA static library (ipp_rta.lib) provided in binary form.

For information on building the sample, see the readme file in the ipp_samples\ speech-codecs directory.

Refer to Appendix C for USC MSRTA Codec description

Performance of USC MSRTA Codec

System Configuration	Intel® microarchitecture code name Nehalem (NHM)	Intel® microarchitecture code name Sandy Bridge (SNB)
CPU	Intel® Xeon® processor X5570 @ 2.93GHz	Genuine Intel® CPU 0 @ 3.00GHz
Operating System	Microsoft Windows* 2003	Microsoft Windows* x64 with SNB patch

We used Intel® IPP 7.0.2 to measure the performance of USC MSRTA codec on Intel® microarchitecture code name Sandy Bridge with frequency of 3.0 GHz and Intel® microarchitecture code name Nehalem at 2.93 GHz. Both systems had Microsoft Windows* 64-bit installed. To measure the performance of the codec, different data sets were used to satisfy the requirement of the MSRTA codec. The two different types of data streams were 16-bit with 16000 Hz wideband data stream (s_16000_16.wav) and 16-bit with 8000 Hz narrowband (s_8000_16.wav).

IA32:

Intel® 64:

As mentioned in the performance table above, encode and decode performances are measured in MHz. By multiplying the performance number in seconds by CPU frequency, you will get performance numbers in MHz. The duration of narrowband data is 1070 sec, and wideband is 1090 sec. To measure the performance improvement of the USC MSRTA codec on Intel® microarchitecture code name Sandy Bridge compared to Intel® microarchitecture code name Nehalem, divide the performance number of Intel microarchitecture code name Nehalem by the performance number of Intel microarchitecture code name Sandy Bridge.

For example, on 32-bit Intel microarchitecture code name Nehalem, narrowband encode performance of USC MSRTA without VAD is 20.42 sec. Performance of the same data set on Intel microarchitecture code name Sandy Bridge is 16.65 sec. To measure the speedup, divide 20.42 by 16.65 to get 1.23x. USC MSRTA decode performance exhibits similar benefits.

Summary

Intel® IPP is a highly optimized library for the latest Intel architecture, including Intel® microarchitecture code name Sandy Bridge. By using Intel IPP functions, Intel® AVX optimized audio and speech codecs can be developed, including the Microsoft RTAudio* (real time audio) codec. Intel IPP offers sample code to demonstrate the development and usage of the MSRTA codec.

Appendix A: Introduction to Intel® IPP

Intel® Integrated Performance Primitives Intel® Integrated Performance Primitives (Intel® IPP) is an extensive library of multi-core ready, highly optimized software functions for digital media and data processing applications. Intel® IPP offers thousands of optimized functions covering frequently-used fundamental algorithms. Intel® IPP functions are designed to deliver performance beyond what optimized compilers alone can deliver. Intel® IPP is a software library that offers various highly optimized functions including multimedia and speech codecs. For advanced performance and greater value, Intel® IPP is also available with Intel® Parallel Studio XE 2011. For more information, go to http://software.intel.com/en-us/articles/intel-ipp/.

Intel® IPP Speech Codec

Intel® IPP includes functions that can be used for implementing speech codecs. These codecs follow the International Telecommunication Union (ITU)* recommendations for G711 (companding functions), G.722, G.722.1, G.723.1, G726, G.728, G.729.1 and G.729 codecs, G.167, G.168 for Echo Canceller, G.169 for Audio Level control, European Telecommunications Standards Institute (ETSI)* specifications for GSM-AMR and GSM-FR codecs, as well as 3GPP* specification for AMRWB and AMRWB+ codecs, and Microsoft* Real-Time Audio codec.

Note: Implementations of these standards or the standard-enabled platforms may require licenses from various entities, including Intel Corporation.

Appendix B: Intel® AVX-optimized Intel® IPP RT Audio Functions

Function Base Name	Operation
AdaptiveCodebookSearch_RTA	Searches for the adaptive codebook index and the lag, and computes the adaptive vector
FixedCodebookSearch_RTA, FixedCodebookSearchRandom_RTA	Searches for the fixed codebook vector
HighPassFilter_RTA	Performs high-pass filtering
LSPQuant_RTA	Performs quantization of LSP coefficients
LSPToLPC_RTA	Converts LSP coefficients to LP coefficients
QMFDecode_RTA	Performs QMF synthesis
PostFilter_RTA	Restores speech signal from the residual

Other required functions for RT Audio functions are as below

Function Base Name	Operation
LPCToLSP_RTA	Converts LP coefficients to LSP coefficients
LevinsonDurbin_RTA	Calculates LP coefficients from the autocorrelation coefficients
QMFGetStateSize_RTA	Calculates the size of the QMF filter state memory
QMFInit_RTA	Initializes the QMF filter state memory
QMFEncode_RTA	Performs QMF analysis
PostFilterGetStateSize_RTA	Calculates the size of the post filter state memory
PostFilterInit_RTA	Initializes the post filter state memory
BandPassFilter_RTA	Performs band pass filtering

*LSP - Line spectral pairs
*LP - Linear Prediction
*QMF- Quadrature mirror filter

Appendix C: USC MSRTA Codec Description

For a description of the USC Interface, refer to the USC manual in the sample directory. Note that in the description of the Codec API function Decode when out-> pBuffer is zero, fake decode is performed, with no PCM stream output setting. out-> nBytes is set to bitstream length bytes without FEC.

The USC MSRTA codec is supported with following parameters:


Codec names	: IPP_MSRTAnb_FP
Compression algorithms	: QMF, LPC, Adaptive and Fixed codebook for low band, Unvoiced Fixed codebook for high band, bandwidth control for variable bitrate mode
Linkage	: USC_MSRTAFP_Fxns
Signal	: 16bit linear PCM
Sampling	: 8000
Frame	: 20ms
Bitrates	: 8800 bps (176 bpf, 22 bytes)
Voice Activity Detection	: variable rate support
Packet Loss Concealment	: PLC supported
Frame type (value)	: default 0, not required
Standard	: Microsoft Real-Time Audio.
Codec names	: IPP_MSRTAwb_FP
Compression algorithms	: QMF, LPC, Adaptive and Fixed codebook for low band, Unvoiced Fixed codebook for high band, bandwidth control for variable bitrate mode.
Linkage:	: USC_MSRTAFP_Fxns
Signal:	: 16bit linear PCM.
Sampling:	: 16000 Hz.
Frame:	: 20ms.
Bitrates:	: 18000 bps (360 bpf, 45 bytes)
Voice Activity Detection	: variable rate support
Packet Loss Concealment	: PLC supported
Frame type (value):	: default 0, not required
Standard:	: Microsoft Real-Time Audio.

Additional Resources

About the Author

Naveen Gv is a Technical Consulting Engineer (TCE) in the performance library team. At Intel he has specialized on Multi-core programming, Intel Performance Primitives and Intel Math Kernel Library. His professional interests are teaching Multi core Programming Methodology to software community and implementing Digital Signal Processing algorithms on x86 platform. Naveen has worked with several universities across Asia Pacific to implement Multi-Core programming in academia. His e-mail address is naveen.gv at intel.com

Naveen Gv

04-21-2011

Tech Articles

Intel® AVX

This article describes how Intel® Integrated Performance Primitives can provide the building blocks to develop an Intel® AVX optimized Microsoft* Real Time Audio codec for VoIP applications.

Don't Spill That Register - Ensuring Optimal Performance From Intrinsics

Mon, 07 Mar 2011 00:00:00 -0800

Download Article

Download Don't Spill That Register - Ensuring Optimal Performance From Intrinsics [PDF 79KB]

Objective

The goal of this article is to help developers ensure their C/C++ code with intrinsics produces the optimal assembly, and to show how to spot unnecessary register spilling.

Abstract

Programming with intrinsics can be as optimal as implementing code directly in assembly. Compared to assembly code, C/C++ code using intrinsics is subject to more compilation steps to generate the final code. Compilation in Debug mode, and possibly in Release mode with improperly set compilation flags, may generate code with seemingly unnecessary instructions for copying registers to and from the stack. Register copying can also result from the source code using more __m256 or __m128 variables than the number of corresponding registers available in hardware. From a simple example using intrinsics, this short article shows good and bad assembly produced and then explains what happened and how to avoid it.

Introduction

The topics of x86 intrinsics, Intel® Streaming SIMD Extensions (Intel® SSE), and the Intel® Advanced Vector Extensions (Intel® AVX) are discussed in detail online on the Intel Software Network site (http://software.intel.com). An intrinsic looks just like a function call in C/C++ code, but the compiler sees it and turns that intrinsic into a single line of assembly. For example, consider the following code:

         __m128 a = _mm_rsqrt_ss(b);  // a = 1.0f/sqrt(b) approx

This line of code will cause the compiler to emit an RSQRTSS instruction at this spot.

Intrinsics let the programmer do instruction level optimization directly, but without the burden of dealing with register allocation, loop syntax, etc. Developers sometimes ask, "Are intrinsics as optimal as assembly?" The answer is usually yes, or at least close to optimal. Furthermore, code with intrinsics is more future-proof, since code initially written for Intel SSE can be recompiled using Intel AVX. Intel AVX versions normally run faster than their Intel SSE counterparts on the same hardware. Given the ease of use and forward compatibility, intrinsics are the logical choice for optimizing to the hardware.

To use intrinsics with the confidence that the program is optimal, it is worthwhile knowing how code gets compiled and be aware of what to look out for. We look at a short intrinsic example and see the corresponding assembly that should result, as well as compilation that generated suboptimal results.

Generated Assembly

For this example, we use an intrinsic implementation of the loop s[i]=a*x[i]+y[i], commonly known as "saxpy", and show the code generated.

  inline void saxpy_simd4(float* S,float _a,const float* X,const float *Y,int n)
  {
     __m128 a = _mm_set1_ps(_a); 
     for(int i=0; i!=n ;i+=4)  // process 4 elements at a time
     {
         __m128 x = _mm_load_ps(X+i);
         __m128 y = _mm_load_ps(Y+i);  
         __m128 s = _mm_add_ps(_mm_mul_ps(a,x),y);  // a*x + y 
         _mm_store_ps(S+i, s );
     }
  }

When this x86 code gets compiled, it should look something like:

    (AVX assembly instructions from loop listed only):
 001B4FF0  vmulps      xmm1,xmm0,xmmword ptr xpoints (1B97A0h)[eax]  
 001B4FF8  vaddps      xmm1,xmm1,xmmword ptr ypoints (1B9390h)[eax]  
 001B5000  vmovaps     xmmword ptr dest (1C0440h)[eax],xmm1  
 001B5008  add         eax,10h  
 001B500B  cmp         eax,400h  
 001B5010  jl          saxpy_128+20h (1B4FF0h)

This assembly sequence was generated by Microsoft Visual Studio* C++ Compiler 2010 with default release mode settings and /arch:AVX added to the command line. Only the loop instructions within the loop that are repeated many times are shown. Variable xmm0 is initially loaded with the constant a. Clearly the first 3 assembly instructions directly map to the intrinsics in the C++ code. The assembly is actually shorter than the corresponding C code, since the register loading intrinsics have been combined with the vmulps and vaddps instructions. The last 3 instructions correspond to the for i loop.

Compiling this small function in another project without optimization flag set resulted in the following assembly:

 00B5F7C2  mov         eax,dword ptr [i]  
 00B5F7C5  add         eax,4  
 00B5F7C8  mov         dword ptr [i],eax  
 00B5F7CB  cmp         dword ptr [i],100h  
 00B5F7D2  jge         saxpy_128+122h (0B5F872h)  
 00B5F7D8  mov         eax,dword ptr [i]  
 00B5F7DB  vmovaps     xmm0,xmmword ptr xpoints (0B715A0h)[eax*4]  
 00B5F7E4  vmovaps     xmmword ptr [ebp-1D0h],xmm0  
 00B5F7EC  vmovaps     xmm0,xmmword ptr [ebp-1D0h]  
 00B5F7F4  vmovaps     xmmword ptr [x],xmm0  
 00B5F7F9  mov         eax,dword ptr [i]  
 00B5F7FC  vmovaps     xmm0,xmmword ptr ypoints (0B71180h)[eax*4]  
 00B5F805  vmovaps     xmmword ptr [ebp-1B0h],xmm0  
 00B5F80D  vmovaps     xmm0,xmmword ptr [ebp-1B0h]  
 00B5F815  vmovaps     xmmword ptr [y],xmm0  
 00B5F81A  vmovaps     xmm0,xmmword ptr [x]  
 00B5F81F  vmovaps     xmm1,xmmword ptr [a]  
 00B5F824  vmulps      xmm0,xmm1,xmm0  
 00B5F828  vmovaps     xmmword ptr [ebp-190h],xmm0  
 00B5F830  vmovaps     xmm0,xmmword ptr [y]  
 00B5F835  vmovaps     xmm1,xmmword ptr [ebp-190h]  
 00B5F83D  vaddps      xmm0,xmm1,xmm0  
 00B5F841  vmovaps     xmmword ptr [ebp-170h],xmm0  
 00B5F849  vmovaps     xmm0,xmmword ptr [ebp-170h]  
 00B5F851  vmovaps     xmmword ptr [s],xmm0  
 00B5F859  vmovaps     xmm0,xmmword ptr [s]  
 00B5F861  mov         eax,dword ptr [i]  
 00B5F864  vmovaps     xmmword ptr dest (0B78240h)[eax*4],xmm0  
 00B5F86D  jmp         saxpy_128+72h (0B5F7C2h)

Here we see that the compiler issued additional instructions that do not correspond to loop management or intrinsics in the original source code. What is happening here is that after registers are loaded from memory, they are copied to and from the stack. The explanation is that, from the C language perspective, the __m128 variables reside on the stack, and the compiler is just putting the data to the place where it was declared. It is the O2 optimization step, not the fact that we used intrinsics, that is normally responsible for removing such unnecessary copying. The extra copying will likely happen in Debug, but may also happen in Release mode if the project's Optimization setting is not Maximum Speed or /O2. The example shown here compiles to Intel AVX, but the same thing happens with Intel SSE as well.

Register Shortage

When using temporary __m128 or __m256 variables for single instruction multiple data (SIMD) programming, the optimizing compiler usually does a good job of keeping these as registers. Even with optimizations, the compiler may still sometimes generate assembly code that copies temporary values to the stack. Consider for example a 3D spring (distance constraint) update written using hybrid structure of arrays (SOA) style programming. The following example is based on code from the AVX cloth sample available on Intel's website:

  void springupdate(__m256 A[][3], __m256 B[][3],__m256 &restlen)
  {
    __m256 half = _mm256_set1_ps(0.5f)
    for(int i=0;i != N ; i++)  // 8*N constraints in total
    {
      // each a and b contain the xyz endpoints for 8 pseudo-springs
      __m256 *a=A[i];
      __m256 *b=B[i];
      __m256 vx  = _mm256_sub_ps(b[0],a[0]); // v.x=b.x-a.x
      __m256 vy  = _mm256_sub_ps(b[1],a[1]); // v.x=b.x-a.x
      __m256 vz  = _mm256_sub_ps(b[2],a[2]); // v.x=b.x-a.x
      __m256 dp  = vx*vx+vy*vy+vz*vz;        // assume operator overloads for add and mul 
      __m256 imag= _mm256_rsqrt_ps(dp);      // inverse magnitude
      // normalize v
      vx = _mm256_mul_ps(vx,imag); // vx *= inverse magnitude 
      vy = _mm256_mul_ps(vy,imag); // vy *= imag
      vz = _mm256_mul_ps(vz,imag); // vz *= imag 
      __m256 half_stretch = ( dp*imag - restlen) * half;
      // move endpoints a and b together 
      a[0]=a[0]+ vx * half_stretch;    
      a[1]=a[1]+ vy * half_stretch;    
      a[2]=a[2]+ vz * half_stretch;    
      b[0]=b[0]- vx * half_stretch;    
      b[1]=b[1]- vy * half_stretch;    
      b[2]=b[2]- vz * half_stretch;   
    }   
  }

For brevity, the above code assumes the obvious operator overloads are implemented and inlined. Even if all intrinsic calls were written out by hand, there is a good chance that compiling a routine like this will produce assembly code that will copy data to and from the stack and/or load the same data from arrays A and B multiple times. While there is no limit to how many variables of this type a programmer uses, there are a limited number of hardware registers available. When Intel AVX code is compiled into a 32-bit executable, the compiler has only 8 YMM registers available. Even with this oversimplified distance constraint equation, the code uses 6 registers for the endpoints, another 3 for the vector v between them, as well as registers for the inverse magnitude, rest length, magnitude minus rest length, half constant, and half stretch amount. Clearly, the compiler must use the same register for more than one variable in this loop. Therefore, it will have to reload values from (and possibly copy values to) the stack. In this situation, the solution to avoid register spilling is to compile the code for 64-bit. Then, instead of just 8, the compiler has 16 YMM (256-bit) registers at its disposal, which is more than enough for this particular simulation.

Conclusion

Starting from a C/C++ intrinsics sample, we've shown the good and the bad of what sort of assembly code can be generated. The suboptimal extra register copying code generation can result from compiler settings such as not using fast-code optimization (O2), or from not using 64-bit when more than 8 registers are needed. There may be other reasons why a compiler might not generate the assembly the programmer expects. Therefore, while intrinsics are often the preferred choice for code optimization, it is still a good idea to inspect the generated assembly to ensure the compiled result is as expected.

Stan Melax

03-06-2011

Tech Articles

Intel® AVX

Intel® SSE

Compared to assembly code, C/C++ code using intrinsics is subject to more compilation steps to generate the final code. Improper compliation settings may result in assembly instructions that copy registers unnecessarily and reduce performance. From a simple example using intrinsics, this short article shows good and bad assembly produced and then explains what happened and how to avoid it.

3D Vector Normalization Using 256-Bit Intel® Advanced Vector Extensions (Intel® AVX)

Mon, 27 Sep 2010 00:00:00 -0700

Download Article

Download 3D Vector Normalization Using 256-Bit Intel® Advanced Vector Extensions (Intel® AVX) [PDF 202KB]

Objective

Show developers how to improve the performance of geometry computations by transposing packed 3D data on-the-fly to take advantage of 8-wide SIMD processing.

Abstract

This article shows how to exploit 256-bit Intel® Advanced Vector Extensions (Intel® AVX) to normalize an array of 3D vectors. We describe a shuffle approach to convert between AOS and SOA on-the-fly in order to make data ready for up to 8-wide single instruction multiple data (SIMD) processing. The 8x3 to 3x8 transpose can be done in 5 shuffles, and the 3x8 to 8x3 transpose back takes 6 shuffles. Results demonstrate the benefit of wider SIMD on the normalization computation with 2.3 and 2.9 times speedups on 128-bit and 256-bit respectively. The round trip cost, 11 extra instructions, of enabling SOA processing is low enough to justify its usage on this small computation.

Introduction

Many interactive applications do a lot of geometry processing of one form or another on the CPU. While structure of array (SOA) memory layout is most efficient for processing, in some applications it may not be practical to store data this way. Quite often there is performance-critical code that operates on a regular array of 3D vectors - a very common data structure found in 3D applications. A common example is making an array of 3D normal vectors to all be of unit length. In this article, we begin with this function and optimize the code to exploit the capabilities of the x86 processor, including new 256-bit instructions from the Intel® Advanced Vector Extensions (Intel® AVX), which are part of the new microarchitecture of the second-generation Intel® Core™ processor family, codenamed Sandy Bridge.

The C code for the loop being optimized is shown below:

  void Normalize(float V[][3],int N)
  {
    for(int i=0;i != N;i++)
    {
        float *v=V[i];
        float invmag = 1.0f/sqrtf(v[0]*v[0]+v[1]*v[1]+v[2]*v[2]); 
        v[0] *= invmag;
        v[1] *= invmag;
        v[2] *= invmag;
    }
  }

In practice, such a routine may be implemented using a C++ 3D vector class with overloaded operators, but the underlying operations and data layout would be the same.

Usage of single instruction multiple data (SIMD) processing with 4-wide 32-bit precision floating point has become widespread since the introduction of the Intel® Streaming SIMD Extensions 2 (SSE2) in 2001. Because the 128-bit register size is a natural fit for 4D data, many 3D interactive applications will use homogeneous vectors, or pad 3D vectors with an extra 32 bits. While an easy way to exploit SIMD processing the gains are rarely a 4x improvement due to other performance bottlenecks. Applying this style of SIMD to our normalization example would only use 3/4 of a 128-bit register. While the multiplications can be done in parallel, summing the squares and the inverse square root do not benefit from SIMD. Most importantly, this programming pattern does not scale to wider SIMD such as 256-bit Intel Advanced Vector eXtensions (AVX) that support parallel operations on 8 floating point numbers.

Another approach to exploit SIMD is to process 8 iterations of the loop at a time, i.e., normalize 8 vectors at once. This requires the data to be rearranged so all the X components of the 8 vectors fit into one register, the Y's in the next, and Z's in another. Loading the registers would be easy if our data happened to be stored this way in a structure of arrays (SOA). However, some applications do require the data storage to be a packed sequence of 3D vectors - an array of structures (AOS). Therefore, to utilize 8-wide SIMD processing, we need to transpose the data on-the-fly, do the computation, and transpose back. This article describes how to do the transpose using shuffles on the x86, followed by provides performance results, including both optimized serial and shuffle transpose implementations, that shows the speedup that can be obtained using Intel AVX for normalizing an (AOS) array of 3D vectors.

The 128-bit and 256-bit AOS to SOA shuffle

The most efficient way to move data from the first level cache into registers is to load 128 bits (or more) at a time. This is a different stride than our packed array of 3D values. However, the pattern of 128-bit alignment repeats for every fourth vector (or 12 floats). Therefore, three aligned 128-bit loads pull the next four 3D vectors into three 128-bit registers.

The order of the vector elements is not very useful right after the three loads. Next we need to get the data from four 3D vectors into a usable form by grouping the X's, Y's and Z's into separate registers. The following figure shows the 4x3 to 3x4 transpose using five Intel AVX 128-bit shuffles.

With the data now in SOA form, the computation is done with the same steps as the serial implementation but using the Intel AVX instructions to normalize 4 vectors at a time.

The result of the computation will be in SOA form and needs to be transposed back into AOS. Our conversion operations are not symmetrical. In other words, the transpose back from SOA to AOS cannot be done with the same code. In fact, it takes one more shuffle to convert back. The technique is illustrated in the following figure.

So far we have described 4 float or 128-bit SIMD usage. We can extend this to 8x3 by using the full 256-bit registers and instructions provided by Intel AVX. When shuffles are involved, the way to think about 256-bit Intel AVX is that there are two lanes that are 128-bit each. In other words, AVX provides two lanes of 4 wide SIMD units. As before, we load the first 12 floats into three 128-bit registers which actually are the lower halves of three 256-bit registers. All we have to do now is load the next 12 floats (or next four 3D vectors) into the upper halves of those same three 256-bit registers. Then the shuffle computation is implemented exactly the same way, but using the 256-bit versions of each of the instructions/intrinsics.

The 256 bit conversion back to AOS in 6 shuffles and subsequent stores are similarly an extension of the 128-bit version above. With the ability to transpose data into SOA and back, we are now ready to apply this technique to normalizing an array of 3D vectors.

Testing and Results

For our testing, we used a single core from an Intel® processor-based system with the microarchitecture codename Sandy Bridge that supports Intel AVX. To focus on instruction costs, L1 caches were primed with the data for 1024 3D vectors. The RDTSC instruction was called before and after each loop to measure the total number of cycles to process the entire dataset. In the tables below, we divide by the size of the array and show throughput results in CPU cycles per 3D vector processed. In addition to the various normalization implementations, we also measure loops with parts removed to gauge the cost of various steps in isolation.

The following table shows the cost per 3D vector of the round trip from AOS to SOA and back to AOS for 3D data on SIMD 128-bit (4 float) and 256-bit (8 float).

Transpose Only Round Trip	Cycles Per Vector Processed	Description
shuf_trans128:	3.0	128-bit AOS->SOA->AOS via shuffles
shuf_trans256:	1.5	256-bit transpose and back

The values in the table are cycles per 3D vector. Note that there is more than one vector being transposed per loop iteration. In both cases, the loop iteration takes 12 cycles which is reasonable since there are 11 shuffles per loop. The meaningful result is that the cost is three or fewer cycles per vector for the round trip conversion from our packed array of structures to a structure of arrays and back.

For the next test, we wanted to gauge what the cost would be if there was no transpose required, so we made another data structure where the data was already prearranged in SOA format, i.e., three separate arrays for all the X's, Y's, and Z's. The following table shows the cost of normalizing 3D vectors already in SOA format.

SIMD Math Only Tests	Cycles Per Vector Normalized	Description
norm_soa_data4:	1.8	128-bit intrinsics norm on SOA data
norm_soa_data8:	1.1	256-bit intrinsics norm on SOA data

Results are presented for both 128-bit and 256-bit SIMD normalization implementations on preformatted SOA data. Because the cost of shuffling data is not free, it is doubtful that normalizing the AOS data could be as fast as these times. This is meant for reference and provides an indication of the absolute performance limit for just the mathematical computation for our loop.

Note that normalization is using the low level of numerical accuracy. The result of the approximate inverse square root assembly routine is not refined with Newton-Raphson. For some applications, such as generating mesh normals for lighting calculations, low accuracy may be considered sufficient. The goal of these tests is to evaluate the potential benefit for the transpose on-the-fly technique for very small computations.

The cost of normalizing our regular array of 3D vectors (AOS dataset) is shown in the following table:

3D Vector Normalization	Cycles Per Vector Normalized	Description
x87:	45.0	serial function without any Intel® SSE or Intel® AVX
arch_optim:	24.3	serial normalization (no SIMD) with -arch optimization
rsqrt_ss:	8.0	serial vector normalization using RSQRT assembly
mask_trans:	9.4	AOS->SOA->AOS uses masking for 4x3 trans
shuf_trans4:	3.5	4 float (128 bit) SOA AOS conversion with shuffles
shuf_trans8:	2.7	8 float (256 bit) 8x3 transpose with shuffles

The results are shown for a variety of implementations. To get a fair basis for comparison, we wrote, using hand-picked assembly instructions, an optimal serial implementation, with less accuracy, that averages 8 cycles per loop or 8 cycles per vector normalization. Also for interest, we compared with another 128-bit AOS->SOA->AOS programming pattern that uses masks and bitwise logical operations to implement the transpose. On current hardware, this method is slower than the serial implementation.

Using Intel AVX's 256 bit SIMD, a regular 3D array of vectors are normalized with a 2.7 cycle throughput. In other words, the SIMD implementation using shuffles performs better than the best serial implementation. Note the cost is not exactly the sum of the computation and the shuffle from the previous two tables. Intel's x86 CPU has independent execution ports for processing shuffles, multiplies, additions, and moves. Overall, the shuffle-based transpose to enable an SOA implementation results in a respectable speedup.

SIMD Benefit	Best Serial	128 bit	256 bit
Speedup	1 (baseline)	2.3	2.9

Conclusion

From a vanilla C/C++ implementation, there are a number of incremental steps that a developer can take to improve code performance, including setting compiler flags to use the available architecture/instructions, picking the faster instructions within the implementation, and exploiting SIMD. Starting from our humble non-tuned Release mode compilation and progressing all the way to our 256-bit Intel® AVX version, the performance improves by over an order of magnitude.

Working with data structures that are already in SOA layout would certainly be the most efficient, but this is not always possible in many applications. Shuffling 3D data between SOA and AOS on the fly is a worthwhile way to get better utilization of the Intel® CPU processor and potentially make 3D applications run faster. The amount of computation per vector in this example is quite small. When there is more work to be done in the innermost loop, the speedup attained by this approach will likely be even greater.

About The Author

Stan Melax received his Bachelor's and Master's degrees in Computing Science from the University of Alberta in Canada in the early 90s. He has spent much of his career in the game industry including BioWare, EA, and Ageia. Stan is now a Graphics Software Engineer at Intel Corporation where he gets to help developers write faster code and make better applications.

Appendix: Transpose Source Code

AOS to SOA 128-bit

C source code with 128-bit Intel® Advanced Vector Extensions (Intel® AVX) intrinsics for converting AOS to SOA:

 float *p;  // address of first vector
 __m128 x0y0z0x1 = _mm_load_ps(p+0);
 __m128 y1z1x2y2 = _mm_load_ps(p+4);  
 __m128 z2x3y3z3 = _mm_load_ps(p+8);
 __m128 x2y2x3y3 = _mm_shuffle_ps(y1z1x2y2,z2x3y3z3,_MM_SHUFFLE( 2,1,3,2));  
 __m128 y0z0y1z1 = _mm_shuffle_ps(x0y0z0x1,y1z1x2y2,_MM_SHUFFLE( 1,0,2,1)); 
 __m128 x        = _mm_shuffle_ps(x0y0z0x1,x2y2x3y3,_MM_SHUFFLE( 2,0,3,0)); // x0x1x2x3
 __m128 y        = _mm_shuffle_ps(y0z0y1z1,x2y2x3y3,_MM_SHUFFLE( 3,1,2,0)); // y0y1y2y3
 __m128 z        = _mm_shuffle_ps(y0z0y1z1,z2x3y3z3,_MM_SHUFFLE( 3,0,3,1)); // z0z1z2z3

The output is found in __m128 registers x,y and z.

SOA to AOS 128-bit.

C source code with 128-bit Intel® Advanced Vector Extensions (Intel® AVX) intrinsics for converting SOA to AOS:

  __m128 x,y,z;  // Starting SOA data
  __m128 x0x2y0y2 = _mm_shuffle_ps(x,y, _MM_SHUFFLE(2,0,2,0)); 
  __m128 y1y3z1z3 = _mm_shuffle_ps(y,z, _MM_SHUFFLE(3,1,3,1)); 
  __m128 z0z2x1x3 = _mm_shuffle_ps(z,x, _MM_SHUFFLE(3,1,2,0)); 

  __m128 rx0y0z0x1= _mm_shuffle_ps(x0x2y0y2,z0z2x1x3, _MM_SHUFFLE(2,0,2,0));  
  __m128 ry1z1x2y2= _mm_shuffle_ps(y1y3z1z3,x0x2y0y2, _MM_SHUFFLE(3,1,2,0));  
  __m128 rz2x3y3z3= _mm_shuffle_ps(z0z2x1x3,y1y3z1z3, _MM_SHUFFLE(3,1,3,1));  

  _mm_store_ps(p+0, rx0y0z0x1 );
  _mm_store_ps(p+4, ry1z1x2y2 );
  _mm_store_ps(p+8, rz2x3y3z3 );

Registers x,y,z containing the data for 4 vectors is shuffled and stored into packed array starting at pointer p.

AOS to SOA 256-bit

C source code with 256-bit Intel® Advanced Vector Extensions (Intel® AVX) intrinsics for converting AOS to SOA:

  float *p;  // address of first vector
 __m128 *m = (__m128*) p;
 __m256 m03;
 __m256 m14; 
 __m256 m25; 
 m03  = _mm256_castps128_ps256(m[0]); // load lower halves
 m14  = _mm256_castps128_ps256(m[1]);
 m25  = _mm256_castps128_ps256(m[2]);
 m03  = _mm256_insertf128_ps(m03 ,m[3],1);  // load upper halves
 m14  = _mm256_insertf128_ps(m14 ,m[4],1);
 m25  = _mm256_insertf128_ps(m25 ,m[5],1);

 __m256 xy = _mm256_shuffle_ps(m14, m25, _MM_SHUFFLE( 2,1,3,2)); // upper x's and y's 
 __m256 yz = _mm256_shuffle_ps(m03, m14, _MM_SHUFFLE( 1,0,2,1)); // lower y's and z's
 __m256 x  = _mm256_shuffle_ps(m03, xy , _MM_SHUFFLE( 2,0,3,0)); 
 __m256 y  = _mm256_shuffle_ps(yz , xy , _MM_SHUFFLE( 3,1,2,0)); 
 __m256 z  = _mm256_shuffle_ps(yz , m25, _MM_SHUFFLE( 3,0,3,1));

Eight 3D vectors are loaded from address p and the output is found in __m256 registers x,y and z. Although this may appear intimidating, it is a natural extension of the 128 bit version.

SOA to AOS 256-bit

C source code with 256-bit Intel® Advanced Vector Extensions (Intel® AVX) intrinsics for converting SOA to AOS:

  __m256 x,y,z;  // Starting SOA data
  float *p; // output pointer
__m128 *m = (__m128*) p;

  __m256 rxy = _mm256_shuffle_ps(x,y, _MM_SHUFFLE(2,0,2,0)); 
  __m256 ryz = _mm256_shuffle_ps(y,z, _MM_SHUFFLE(3,1,3,1)); 
  __m256 rzx = _mm256_shuffle_ps(z,x, _MM_SHUFFLE(3,1,2,0)); 

  __m256 r03 = _mm256_shuffle_ps(rxy, rzx, _MM_SHUFFLE(2,0,2,0));  
  __m256 r14 = _mm256_shuffle_ps(ryz, rxy, _MM_SHUFFLE(3,1,2,0)); 
  __m256 r25 = _mm256_shuffle_ps(rzx, ryz, _MM_SHUFFLE(3,1,3,1));  

  m[0] = _mm256_castps256_ps128( r03 );
  m[1] = _mm256_castps256_ps128( r14 );
  m[2] = _mm256_castps256_ps128( r25 );
  m[3] = _mm256_extractf128_ps( r03 ,1);
  m[4] = _mm256_extractf128_ps( r14 ,1);
  m[5] = _mm256_extractf128_ps( r25 ,1);

Registers x,y,z containing the data for eight 3D vectors is shuffled and stored into packed array starting at pointer p.

Stan Melax

09-27-2010

Tech Articles

Intel® AVX

This article shows how to exploit 256-bit Intel® Advanced Vector Extensions (Intel® AVX) to normalize an array of 3D vectors. A shuffle approach is described to convert between an Array of Structures (AOS) and a Structure of Arrays (SOA) on-the-fly in order to make data ready for up to 8-wide single instruction multiple data (SIMD) processing. The round trip cost, 11 extra instructions, of enabling SOA processing is low enough to justify its usage on this small computation.

Intel® AVX C/C++ Intrinsics Emulation

Wed, 23 Jun 2010 00:00:00 -0700

Intel® AVX instruction set extension [1] will appear in the next generation Intel microarchitecture codename ‘Sandy Bridge'. We chose to announce AVX early to get as much support from software vendors as possible by the hardware launch time. Now, most software development platforms are supporting Intel AVX, examples are compilers and assemblers from Intel, Microsoft and GCC as well as UNIX binutils.

For early adopters we introduced support of AVX in Intel® Software Development Emulator [2], it allows you to run and check functional correctness of the code with the actual AVX instructions before hardware is available.

Today we are adding another useful piece to help those who may not be able to use new tools supporting AVX in their current development environment but plan to migrate in the future or are using a software platform which is not supported by Intel SDE. These software developers can still start programming with Intel AVX using intrinsics.

Here we are providing the C and C++ header file which emulates Intel AVX intrinsics. The AVX emulation header file uses intrinsics for the prior Intel instruction set extensions up to Intel SSE4.2. SSE4.2 support in your development environment as well as hardware is required in order to use the AVX emulation header file.

To use simply have this file included:

#include "avxintrin_emu.h"

Instead of usual:

#include

One can also create alternative immintrin.h file (which in turn includes avxintrin_emu.h) to avoid an intrusive change to the source base and then simply switch between real AVX code generation and emulation via alternating the path to include directories.

Emulation header is primarily targeting UNIX type of environments, and was tested on such with GCC and Intel C/C++ compilers. We have a strong support with other tools (compilers, assemblers and SDE) on Microsoft Windows platform, but this header file can still be used on Windows, if desired, with Intel Compiler.

Note that the AVX emulation header file is designed to allow functional correctness of an AVX implementation and not recommended for long-term usage or release in a final product. Once your development environment and hardware supports AVX, we recommend that you switch to the real AVX intrinsic header file.

Although we did our best to debug it, this file must not be considered a reference functional implementation of AVX instructions or even bug-free. Please see the current version's limitations and caveats in the beginning of the file. Please let us know about the issues you faced using it.

Example

#include "avxintrin_emu.h"  // #include 

void saxpy( float a, const float* x, const float* y, float* __restrict z, size_t len )
{
    size_t i = 0;
    __m256 a_ = _mm256_set1_ps( a );

    for ( size_t len16_ = len & -16; i + 16 <= len16_; i += 16 )
    {
        __m256 x1_ = _mm256_loadu_ps( x + i );
        __m256 x2_ = _mm256_loadu_ps( x + i + 8 );

        __m256 y1_ = _mm256_loadu_ps( y + i );
        __m256 y2_ = _mm256_loadu_ps( y + i + 8 );

        x1_ = _mm256_mul_ps( x1_, a_ );
        x2_ = _mm256_mul_ps( x2_, a_ );

        x1_ = _mm256_add_ps( x1_, y1_ );
        x2_ = _mm256_add_ps( x2_, y2_ );

        _mm256_storeu_ps( z + i     , x1_ );
        _mm256_storeu_ps( z + i + 8 , x2_ );
    }

    for ( ; i < len; ++i )
        z[i] = x[i] * a + y[i];
}

References

[1] Intel AVX - http://software.intel.com/en-us/avx/

[2] Intel Software Development Emulator - http://software.intel.com/en-us/articles/intel-software-development-emulator/

Using Intel® Advanced Vector Extensions to Implement an Inverse Discrete Cosine Transform

Mon, 07 Jun 2010 21:00:00 -0700

Download Article

Download Using Intel® Advanced Vector Extensions to Implement an Inverse Discrete Cosine Transform [PDF 335KB]
Download IDCT source code [Zip 22KB]

Introduction

Transform coding is an important step of image and video processing applications. Pixels in an image have a level of correlation with their neighboring pixels. Adjacent pixels in successive frames show a very high correlation. These correlations can be used to predict the value of a pixel from its neighbor. The highly correlated spatial data is transformed into uncorrelated coefficients in the frequency domain. The transformed data, represented as coefficients, are independent, allowing them to be manipulated separately. The human eye is more perceptive to low-frequency changes than high-frequency changes. Encoders can achieve compression after setting the high-frequency coefficients to zero. An inverse transform is used in the decoder pipeline to reconstruct the source data.

Figure 1 and Figure 2 depict the components of a typical image encoder/decoder system.

Figure 1 - Source Encoder Block Diagram

Figure 2 - Source Decoder Block Diagram

The Discrete Cosine Transform (DCT) and Inverse Discrete Cosine Transform (IDCT) are widely used in the encoder and decoder pipelines of image processing systems such as MPEG, JPEG, and other standards. A fast and accurate IDCT transformation is crucial for the best user experience.

Intel® Advanced Vector Extensions (Intel® AVX), introduced with the new Intel® processor microarchitecture codenamed Sandy Bridge, extends the capabilities of Intel® Streaming SIMD Extensions (Intel® SSE) for floating point data and operations such as Inverse Discrete Cosine Transforms. Intel® AVX essentially doubles the width of the current XMM registers and adds new extensions that can operate on the wider data width. Intel® AVX significantly increases the floating-point performance density with improved power efficiency over previous 128-bit SIMD instruction set extensions. This document specifically examines how Intel® AVX and Sandy Bridge microarchitecture features such as wider 256-bit SIMD registers, non-destructive source operands, new data manipulation and arithmetic primitives, dual 128-bit load ports, and doubling of computational execution width can benefit the Inverse Discrete Cosine Transform (IDCT) operation. Intel® AVX improves the computational throughput of high performance precision-sensitive floating point transform applications. Integer implementations may also benefit from features such as non-destructive source operands.

Intel also provides a set of Intel® AVX software development tools like the Intel® AVX-enabled Intel® C++ Compiler, the Intel® Software Development Emulator (SDE), and the Intel® Architecture Code Analyzer. All of these tools were effectively used during the development of this kernel, and can be downloaded from the Intel® AVX website: http://software.intel.com/en-us/avx/

Testing Environment

The performance speedups stated in this paper are based on runs on actual Sandy Bridge microarchitecture-based pre-production silicon. It can also be assumed that the test data is already in the first level processor cache prior to the computation of the IDCT algorithm. Performance comparisons are made based on the relative performance of Intel® AVX versus corresponding Intel® SSE implementations using C intrinsic instructions, both run on the Sandy Bridge microarchitecture-based silicon. The code was compiled using the 64-bit Intel® C++ Compiler, version 11.1.038. The applications were compiled using the following command line options:

Intel® SSE: /QxSSE4.1 /O3
Intel® AVX: /QxAVX /O3

A comparison of short integer performance was performed by compiling the Intel^® SSE implementation for different architectures. The compiler will generate VEX-encoded Intel^® SSE instructions when the /QxAVX switch is specified. The compiler will generate Intel^®SSE 4.1 instructions when the /QxSSE4.1 switch is specified. Non-destructive source operands are introduced with the VEX-encoded instructions.

The test application follows the 8x8 IDCT accuracy requirements as documented in section 3 of the IEEE standard 1180-1900 (Reference 5). Randomly generated input data is processed by a reference DCT. The output of the DCT is rounded to the nearest integer and clipped such that -2048 <= value < 2047. The clipped values are inputs to both the reference IDCT and the IDCT under test. Their outputs are rounded to the nearest integer and clipped such that -256 <= output < 255. The two results are then compared and accuracy measurements are made.

In this application, the references are implemented using double precision floating point scalar C code. Several proposed IDCT implementations are tested; a short integer Intel® SSE version, a single precision floating point Intel® SSE version, and a single precision floating point Intel® AVX version. All the vector versions are implemented with C intrinsics.

Figure 3 - Test Application

Discrete Cosine Transforms

The Discrete Cosine Transform (DCT) converts the spatial data of an image into the frequency domain. The mathematical operations are described in detail in Reference [1].

The two-dimensional (2D) DCT transforms 64 pixel values, an 8x8 block of pixels, producing 64 coefficients. The image's pixel values are transformed into coefficients derived from amplitudes of cosine basis functions.

Where:

Reference work [2] summarizes the number of mathematical operations; the numbers are worth repeating. Comparing the DCT and IDCT equations, it can be shown that the transforms require the same number of operations. An analysis of Equation 2 shows that the 2D DCT requires 64 multiplications and 63 additions for each coefficient. Therefore 4096 multiplications and 4023 additions are required to transform an 8x8 block. Replacing the 2D DCT with 8 1D DCTs for the 8 rows and 8 1D DCTs for the 8 columns can reduce the number of operations. Equation 1 shows that the 1D DCT requires 64 multiplications and 56 additions producing 8 coefficients. Transforming the 8x8 block with 1D DCTs requires 1024 multiplications and 896 additions. References [1] and [4] discuss other DCT and IDCT algorithms.

Previous Work and Specifications

The following previous works and specifications should be consulted for background information:

Reference [1] for the JPEG specification.
The IDCT algorithm used in the Intel® AVX implementation discussed in this whitepaper follows the optimized algorithm discussed in Reference [2].
Reference [3] provides additional background information.

Intel® AVX Implementation of IDCT

As in the previous works, the algorithm in this whitepaper performs 8 1D IDCT transforms on the rows of an 8x8 block. Then 8 1D IDCT is performed on the columns of those results. Two different 1D IDCT transforms are used, and neither transform requires a transpose.

The AVX implementation operates on two rows of data simultaneously. It begins by loading four floats from each row into separate 128-bit registers. The reason for performing 128-bit loads will become clearer in a moment. It is important to remember that the 128-bit XMM registers overlay the lower 128-bits of the corresponding 256-bit YMM register. The upper 128-bits of the YMM register is loaded with the next set of four floats from those rows via the _mm256_insertf128_ps instruction. The resultant YMM register contains the first set of four floats from one row in the lower 128-bits, and the second set of four floats from the second row in the upper 128-bits. This seems like a lot of work to load eight floats, so why take this approach? This is known as a strided load (because the next load will read from an area of memory that is more than a unit stride away from the current memory location).

The strided load is effective in certain applications.

Eight products need to be summed in the IDCT and it is more straightforward to sum the products if they are in the same 128-bit lane of two YMM registers. Two registers are built with the contents from one row in the bottom 128-bits of both registers, and the contents from the second row in the upper 128-bits of both register. This is depicted in Figure 4 through Figure 6.
Another benefit of the strided load approach is that data movement, or placement of the 128-bit memory operand into the upper 128 bits, can take place on a different execution port than the shuffle port. This reduces the utilization of the port that performs shuffles, and allocates the work more evenly across the execution ports.
One last benefit of placing floats 4-7 of two rows into one register allows for a reduction in shuffles that takes place near the end of row processing.

Figure 4 - Loading Four Floats from Row 0

The combination of the _mm_load_ps and _mm256_castps128_ps256 instructions loads 128-bits into the least significant 128-bits of a ymm register.

Figure 5 depicts the strided load of Row 4 data, and the insertion into the most significant 128-bits of ymm0.

Figure 5 - Strided Load of Second Row

A second YMM register is populated in a similar manner; it contains the second set of four floats (floats 4-7) from both rows. The results are shown in Figure 6.

Figure 6 - Second Set of Floats from Two Rows

The input data must be multiplied by different cosine terms. A single float from each row is broadcasted via the _mm256_shuffle_ps to prepare for the multiplication, as shown in Figure 7. In this case float 0 is duplicated.

Figure 7 - Preparing for Cosine Multiplication

Two _mm256_mul_ps instructions perform the necessary cosine multiplications. This completes one of the seven sums of product terms.

This sequence of instructions is repeated three times, operating on a different pair of input values and cosine terms each time. At this point there are four registers containing sums of products. One register contains sums of products for floats 0 and 2, another contains sums of products for floats 1 and 3, 4 and 6, and finally 5 and 7.

Three more sums are necessary to complete the sum of products. The sums of products of all even terms are added, and the sums of products of all odd terms are added. That is only two of the three sums.

      ymm_even = _mm256_add_ps(r_ymm02, r_ymm46);
ymm_odd = _mm256_add_ps(r_ymm13, r_ymm57);

The seventh and final sum produces the least significant four floats of the output. A subtraction is necessary to produce the most significant four floats; however, they are not in the correct order.

     ymm_diff = _mm256_sub_ps(ymm_even, ymm_odd);
ymm_sum = _mm256_add_ps(ymm_even, ymm_odd);

The order of the most significant floats of two rows can be properly adjusted using a single _mm256_shuffle_ps instruction, as shown in Figure 8. The benefit of placing the most significant four floats from each input row is more apparent.

Figure 8 - Reversing the Order of Differences

The final step for the 1D IDCT for these two rows is to recombine the results for each row into the same 256-bit register using two _mm256_ permute2f128 instructions.

Figure 9 - Recombining Row 0

Figure 10 - Recombining Row 4

That completes the 1D IDCT for two sets of rows. The sequence is repeated for rows 1 and 7, 2 and 6, and finally 3 and 5 to complete the 1D IDCT for eight rows.

Now that the 1D IDCT of the eight rows is completed, the 1D IDCT of the eight columns can begin. The algorithm is similar, yet simpler because the sums of products are already in the proper position for the addition. The 1D IDCT of the columns does not require any shuffles.

It was possible to eliminate four add instructions in the floating point IDCT of the columns.

      //row5*tangent + row5
r_ymm0 = _mm256_mul_ps(row5, tangent);
r_ymm0 = _mm256_add_ps(row5, r_ymm0);

A single multiply of the row times the tangent term plus 1 produces identical results in the floating point implementation.

Tangent_p1 = tan(x) + 1

//row5*(tangent+1)
r_ymm0 = _mm256_mul_ps(row5, tangent_p1);
//row3*(tangent+1)
r_ymm1 = _mm256_mul_ps(row3, tangent_p1);

This optimization could not be applied to the short integer implementation due to the scaling of numbers in the short algorithm.

Using Intel® Architecture Code Analyzer

The Intel® Architecture Code Analyzer is an excellent tool that identifies the critical path in a basic block, execution port utilization, and instruction usage alternatives. There is much more Intel® Architecture Code Analyzer information available at the Intel® AVX developer site.

The basic block analysis in Figure 11 shows that the vshufps (invoked via the _mm256_shuffle_ps intrinsic) executes on port 5. The analysis also shows that the vinsertf128 instruction (_mm256_insertf128_ps intrinsic) offers the programmer choices for port execution. The "X" in the Intel® Architecture Code Analyzer report indicates that an instruction can be executed on the port, but in this particular algorithm it was executed elsewhere. In this case, the vinsertf128 was executed on port 0 and port 3 rather than port 3 and port 5 because the second source operand was loaded from memory. The insert takes place on port 0, thus reducing the pressure on port 5.

Figure 11 - Basic Block Analysis

The Intel® Architecture Code Analyzer is an effective optimization tool for software developers who want to fine tune and improve their application's performance.

Results

The 128-bit code was compiled for the Intel® microarchitecture codenamed Nehalem to generate Intel® SSE code, and executed on the Sandy Bridge microarchitecture-based silicon. The corresponding 256-bit Intel® AVX-enabled code was compiled for the Sandy Bridge microarchitecture and executed on the Sandy Bridge microarchitecture-based silicon. Data was aligned on 16-Byte boundaries for the Intel® SSE code and 32-Byte boundaries for the Intel® AVX code. Both applications were compiled using the 64-bit version of the Intel® C++ Compiler Professional Edition, version 11.1.038. The speedups listed here are for 256-bit code relative to the128-bit code.

Table 1 - Performance Results

Algorithm	Speedup	Parameters
Intel® AVX floating point implementation	0.94x	Compared to Intel® SSE short integer implementation
Intel® SSE short integer implementation compiled with /QxAVX	1.07x	Compared to Intel® SSE short integer implementation compiled with /QxSSE4.1
Intel® AVX floating point implementation	1.78x	Compared to Intel® SSE floating point implementation

Table 2 summarizes the overall mean error results for each implementation.

Table 2 - Overall Mean Error Results

L, H (Reference 5)	Intel® SSE Short	Intel® SSE Single Precision Floating Point	Intel® AVX Single Precision Floating Point
256, 255	3.44e-5	6.25e-6	6.25e-6
5, 5	2.58e-4	1.56e-6	1.56e-6
300, 300	4.69e-6	6.25e-6	6.25e-6
-255, 256	7.53e-4	3.13e-6	3.13e-6
-5, 5	0	0	0
-300, 300	0	0	0
Input = 0	0	0	0

Both the short integer and single precision floating point implementations meet the error requirements of IEEE 1180-1900 (Reference 5). The floating point implementations produce lower overall mean error in many of the tests; 5x, 165x, and 240x lower than the short integer implementation.

Conclusion

The results for the IDCT of 10,000 8x8 blocks shows that the Intel® AVX version outperformed the Intel® SSE single precision floating point implementation by 1.78x. The accuracy is excellent when compared to the reference IDCT.

The results also show the Intel^® SSE short integer version compiled with the /QxAVX option outperformed the exact same code compiled with the /QxSSE4.1 option by 1.07x. The assembly language produced when compiling with /QxSSE4.1 has 22 register-to-register moves. The code produced with the /QxAVX switch did not have any register-to-register moves. The Intel^® AVX non-destructive source instructions reduce the need for register copies in this application. There can be benefits to using Intel AVX for integer-based algorithms today.

Although the Intel® AVX implementation is slightly slower than the Intel® SSE short version, the Intel® AVX single precision floating point version is more accurate. The Intel® SSE short version required several adjustments to improve the accuracy and minimize rounding errors. Those adjustments were not necessary in either floating point implementation, resulting in a cleaner and more accurate implementation.

Source Code for IDCT

The source code for the IDCT can be downloaded here

Partial source code listing is provided in the following section.

void idctAVX(void)	{
__m128 r_xmm0, r_xmm2, r_xmm1;
__m256 r_ymm02, r_ymm46, r_ymm13, r_ymm57;
__m256 r_ymm0, r_ymm1, r_ymm2, r_ymm3, r_ymm4, r_ymm5, r_ymm6, r_ymm7;
__m256 row0, row1, row2, row3, row4, row5, row6, row7;
__m256 ymm_even, ymm_odd, ymm_sum, ymm_diff;
__m256 temp3, temp7;
__m256 tangent_1, tangent_2, tangent_3, cos_4;

tangent_1 = AVX_tg_1_16;
tangent_2 = AVX_tg_2_16;
tangent_3 = AVX_tg_3p1_16;
cos_4 = AVX_cos_4p1_16;

const float * pInput;
float * pOutput;
float * pFTab_i_04 = float_tab_i_04;
float * pFTab_i_26 = float_tab_i_26;
float * pFTab_i_17 = float_tab_i_17;
float * pFTab_i_35 = float_tab_i_35;
const int blockSize = 8*8;

//Transform all the blocks N times
//The iteration count is a command line option
const int maxLoopCount = g_loopCount;
long startTime = getTimestamp();

for(int loopCount = 0; loopCount < maxLoopCount; loopCount++)	{
	
   //Operate on all the blocks
   for(int i = 0; i < g_blockCount; i++)	{

      //Get pointers for this input and output
      pInput = &dctData[i*blockSize];				
      pOutput = &kernelResults[i*blockSize];
      pFTab_i_04 = float_tab_i_04;
      pFTab_i_26 = float_tab_i_26;
      pFTab_i_17 = float_tab_i_17;
      pFTab_i_35 = float_tab_i_35;
//    IACA_START

      //Rows 0 and 4
      //Process the first four floats of these two rows
      //Read input data from row 0, read eight floats via two 128-bit loads
      r_ymm0 = _mm256_castps128_ps256(_mm_load_ps(pInput));
      r_ymm1 = _mm256_castps128_ps256(_mm_load_ps(&pInput[4]));
 
      //Insert data from row 4 into the upper lane
      r_ymm0 = _mm256_insertf128_ps(r_ymm0, _mm_load_ps(&pInput[4*8]), 1);
      r_ymm4 = _mm256_insertf128_ps(r_ymm1, _mm_load_ps(&pInput[4*8+4]), 1);

      //Broadcast float 0 
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x00);
   
      //Multiply by the coefficients
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) pFTab_i_04));

      //Broadcast float 2 and multiply by coefficients
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_04[8]));

      //Add
      r_ymm02 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Broadcast float 1 and multiply by coefficients
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_04[16]));

      //Broadcast float 3 and multiply by coefficients
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_04[24]));
   
      //Add
      r_ymm13 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process the second four floats of these two rows
      //Broadcast float 4 
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x00);

      //Multiply by the coefficients
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_04[32]));

      //Broadcast float 6 and multiply by coefficients
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_04[40]));

      //Add
      r_ymm46 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Broadcast float 5 and multiply by coefficients
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_04[48]));

      //Broadcast float 7 and multiply by coefficients
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_04[56]));

      //Add to create this partial sum, 
      //then create final sums and differences.
      r_ymm57 = _mm256_add_ps(r_ymm2, r_ymm3);
      ymm_even = _mm256_add_ps(r_ymm02, r_ymm46);
      ymm_odd = _mm256_add_ps(r_ymm13, r_ymm57);
      ymm_diff = _mm256_sub_ps(ymm_even, ymm_odd);
      ymm_sum = _mm256_add_ps(ymm_even, ymm_odd);

      //Reverse the order of the differences, then build outputs 0 and 4
      ymm_diff = _mm256_shuffle_ps(ymm_diff, ymm_diff, 0x1b);
      row0 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x20);
      row4 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x31);
      //End of Rows 0 and 4 processing

      //Rows 1 and 7
      //Process the first four floats of these two rows
      //Read input data from row 1, read eight floats via two 128-bit loads
      r_ymm0 = _mm256_castps128_ps256(_mm_load_ps(&pInput[8]));
      r_ymm1 = _mm256_castps128_ps256(_mm_load_ps(&pInput[8+4]));

      //Insert data from row 7 into the upper lane
      r_ymm0 = _mm256_insertf128_ps(r_ymm0, _mm_load_ps(&pInput[7*8]), 1);
      r_ymm4 = _mm256_insertf128_ps(r_ymm1, _mm_load_ps(&pInput[7*8+4]), 1);
      
      //Process floats 0 and 2 
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x00);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) pFTab_i_17));
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_17[8]));
      r_ymm02 = _mm256_add_ps(r_ymm2, r_ymm3);

       //Process floats 1 and 3
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_17[16]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_17[24]));
      r_ymm13 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process the second four floats of these two rows
      //Process floats 4 and 6
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x00);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_17[32]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_17[40]));
      r_ymm46 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process floats 5 and 7
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_17[48]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_17[56]));
      r_ymm57 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Create final sums and differences
      ymm_even = _mm256_add_ps(r_ymm02, r_ymm46);
      ymm_odd = _mm256_add_ps(r_ymm13, r_ymm57);
      ymm_diff = _mm256_sub_ps(ymm_even, ymm_odd);
      ymm_sum = _mm256_add_ps(ymm_even, ymm_odd);
    
      //Reverse the order of the differences, then build outputs 1 and 7
      ymm_diff = _mm256_shuffle_ps(ymm_diff, ymm_diff, 0x1b);
      row1 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x20);
      row7 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x31);
      //End of Rows 1 and 7 processing

      //Rows 2 and 6
      //Process the first four floats of these two rows
      //Read input data from row 2, read eight floats via two 128-bit loads
      r_ymm0 = _mm256_castps128_ps256(_mm_load_ps(&pInput[2*8]));
      r_ymm1 = _mm256_castps128_ps256(_mm_load_ps(&pInput[2*8+4]));

      //Insert data from row 6 into the upper lane
      r_ymm0 = _mm256_insertf128_ps(r_ymm0, _mm_load_ps(&pInput[6*8]), 1);
      r_ymm4 = _mm256_insertf128_ps(r_ymm1, _mm_load_ps(&pInput[6*8+4]), 1);   
   
      //Process floats 0 and 2 
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x00);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) pFTab_i_26));
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_26[8]));
      r_ymm02 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process floats 1 and 3
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_26[16]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_26[24]));
      r_ymm13 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process the second four floats of these two rows
      //Process floats 4 and 6
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x00);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_26[32]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_26[40]));
      r_ymm46 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process floats 5 and 7
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_26[48]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_26[56]));
      r_ymm57 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Create the final sum and difference
      ymm_even = _mm256_add_ps(r_ymm02, r_ymm46);
      ymm_odd = _mm256_add_ps(r_ymm13, r_ymm57);
      ymm_diff = _mm256_sub_ps(ymm_even, ymm_odd);
      ymm_sum = _mm256_add_ps(ymm_even, ymm_odd);
   
      //Reverse the order of the differences, then build outputs 2 and 6
      ymm_diff = _mm256_shuffle_ps(ymm_diff, ymm_diff, 0x1b);
      row2 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x20);
      row6 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x31);
      //End of Rows 2 and 6 processing

      //Rows 3 and 5
      //Process the first four floats of these two rows
      //Read input data from row 3, read eight floats via two 128-bit loads
      r_ymm0 = _mm256_castps128_ps256(_mm_load_ps(&pInput[3*8]));
      r_ymm1 = _mm256_castps128_ps256(_mm_load_ps(&pInput[3*8+4]));

      //Insert data from row 5 into the upper lane
      r_ymm0 = _mm256_insertf128_ps(r_ymm0, _mm_load_ps(&pInput[5*8]), 1);
      r_ymm4 = _mm256_insertf128_ps(r_ymm1, _mm_load_ps(&pInput[5*8+4]), 1);
    
      //Process floats 0 and 2 
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x00);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) pFTab_i_35));
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_35[8]));
      r_ymm02 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process floats 1 and 3
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_35[16]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm0, r_ymm0, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_35[24]));
      r_ymm13 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process the second four floats of these two rows
      //Process floats 4 and 6
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x00);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_35[32]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xaa);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_35[40]));
      r_ymm46 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Process floats 5 and 7
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0x55);
      r_ymm2 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_35[48]));
      r_ymm1 = _mm256_shuffle_ps(r_ymm4, r_ymm4, 0xff);
      r_ymm3 = _mm256_mul_ps(r_ymm1, *((__m256 *) &pFTab_i_35[56]));
      r_ymm57 = _mm256_add_ps(r_ymm2, r_ymm3);

      //Create the final sum and difference
      ymm_even = _mm256_add_ps(r_ymm02, r_ymm46);
      ymm_odd = _mm256_add_ps(r_ymm13, r_ymm57);
      ymm_diff = _mm256_sub_ps(ymm_even, ymm_odd);
      ymm_sum = _mm256_add_ps(ymm_even, ymm_odd);

      //Reverse the order of the differences, then build outputs 3 and 5
      ymm_diff = _mm256_shuffle_ps(ymm_diff, ymm_diff, 0x1b);
      row3 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x20);
      row5 = _mm256_permute2f128_ps(ymm_sum, ymm_diff, 0x31);
      //End of Rows 3 and 5 processing

      //******************************
      //perform 1D IDCT on the columns
   
      //Multiply several rows by the appropriate tangent value
      //row5*(tangent3+1)
      r_ymm0 = _mm256_mul_ps(row5, tangent_3);
      //row3*(tangent3+1)
      r_ymm1 = _mm256_mul_ps(row3, tangent_3);	
      //row7*tangent1
      r_ymm4 = _mm256_mul_ps(row7, tangent_1);	
      //row1*tangent1
      r_ymm5 = _mm256_mul_ps(row1, tangent_1);

      //Begin to create results	
      //[row5*tangent3 + row5] + row3
      r_ymm0 = _mm256_add_ps(r_ymm0, row3);
      //row5 - [row1*tangent3 + row1]		
      r_ymm2 = _mm256_sub_ps(row5, r_ymm1);
      //row6*tangent2
      r_ymm7 = _mm256_mul_ps(row6, tangent_2);	
      //row2*tangent2
      r_ymm3 = _mm256_mul_ps(row2, tangent_2);
      //row1*tangent1 - row7	
      r_ymm5 = _mm256_sub_ps(r_ymm5, row7);
      //row7*tangent1 + row1		
      r_ymm4 = _mm256_add_ps(r_ymm4, row1);		

      //Save intermediate row 7 results, used as an input later
      //[row7*tangent1 + row1] + [row5*tangent3 + row3]
      temp7 = _mm256_add_ps(r_ymm4, r_ymm0);		

      //Save intermediate row 3 results, used as an input later
      //[row1*tangent1 - row7] + [row5 - [row1*tangent3 + row1]]
      temp3 = _mm256_add_ps(r_ymm5, r_ymm2);		

      //[row7*tangent1 + row1] - [row5*tangent3 + row3]			
      r_ymm4 = _mm256_sub_ps(r_ymm4, r_ymm0);
      //[row1*tangent1 - row7] - [row5 - [row1*tangent3 + row1]]
      r_ymm5 = _mm256_sub_ps(r_ymm5, r_ymm2);	
      //{[row7*tangent1 + row1] - [row5*tangent3 + row3]} - 
      //{[row1*tangent1 - row7] - [row5 - [row1*tangent3 + row1]]}	
      r_ymm1 = _mm256_sub_ps(r_ymm4, r_ymm5);	
      //{[row7*tangent1 + row1] - [row5*tangent3 + row3]} + 
      //{[row1*tangent1 - row7] - [row5 - [row1*tangent3 + row1]]}	
      r_ymm4 = _mm256_add_ps(r_ymm4, r_ymm5);
      //multiply by cos_4+1		
      r_ymm4 = _mm256_mul_ps(r_ymm4, cos_4);
      //row6*tangent2 + row2
      r_ymm7 = _mm256_add_ps(r_ymm7, row2);
      //row2*tangent2 - row6		
      r_ymm3 = _mm256_sub_ps(r_ymm3, row6);
      //multiply by cos_4+1		
      r_ymm0 = _mm256_mul_ps(r_ymm1, cos_4);
      //row0 + row4
      r_ymm5 = _mm256_add_ps(row0, row4);	
      //row0 - row4
      r_ymm6 = _mm256_sub_ps(row0, row4);	
      //[row0 + row4] - [row6*tangent2 + row2]		
      r_ymm2 = _mm256_sub_ps(r_ymm5, r_ymm7);
      //[row0 + row4] + [row6*tangent2 + row2]
      r_ymm5 = _mm256_add_ps(r_ymm5, r_ymm7);		
      //[row0 - row4] - [row2*tangent2 - row6]
      r_ymm1 = _mm256_sub_ps(r_ymm6, r_ymm3);
      //[row0 - row4] + [row2*tangent2 - row6]	
      r_ymm6 = _mm256_add_ps(r_ymm6, r_ymm3);		
      //[[row7*tangent1 + row1] + [row5*tangent3 + row3]] + 
      //[[row0 + row4] + [row6*tangent2 + row2]]
      r_ymm7 = _mm256_add_ps(temp7, r_ymm5);		
			
      //Store row 0 results (store 1 of 8)
      _mm256_store_ps(pOutput, r_ymm7);
      //[[row0 - row4] + [row2*tangent2 - row6]] - 
      //cos4*{[row7*tangent1 + row1] - [row5*tangent3 + row3]} +
      //{[row1*tangent1 - row7] - [row5 - [row1*tangent3 + row1]]}
      r_ymm3 = _mm256_sub_ps(r_ymm6, r_ymm4);
      //[[row0 - row4] + [row2*tangent2 - row6]] + 
      //cos4*{[row7*tangent1 + row1] - [row5*tangent3 + row3]} + 
      //{[row1*tangent1 - row7] - [row5 - [row1*tangent3 + row1]]}	
      r_ymm6 = _mm256_add_ps(r_ymm6, r_ymm4);		

      //Store row 1 results (store 2 of 8)
      _mm256_store_ps(&pOutput[1*8], r_ymm6); 

      r_ymm7 = _mm256_sub_ps(r_ymm1, r_ymm0);
      r_ymm1 = _mm256_add_ps(r_ymm1, r_ymm0);

      r_ymm6 = _mm256_add_ps(r_ymm2, temp3);
      r_ymm2 = _mm256_sub_ps(r_ymm2, temp3);	 

      r_ymm5 = _mm256_sub_ps(r_ymm5, temp7); 

      //Store final results
      _mm256_store_ps(&pOutput[2*8], r_ymm1); 
      _mm256_store_ps(&pOutput[3*8], r_ymm6); 
      _mm256_store_ps(&pOutput[4*8], r_ymm2);
      _mm256_store_ps(&pOutput[5*8], r_ymm7);
      _mm256_store_ps(&pOutput[6*8], r_ymm3);
      _mm256_store_ps(&pOutput[7*8], r_ymm5); 

//      IACA_END
      }	
   }
   long duration = getTimestamp() - startTime;
   cout << "AVX Timestamp = " << duration << endl;
}

References

The following documents are referenced in this application note, and provide background or supporting information for understanding the topics presented in this document.

1. Pennebaker and Mitchell, JPEG: Still Image Data Compression Standard, Van Nostrand Reinhold, New York, 1993, pp. 29-64.
2. A Fast Precise Implementation of 8x8 Discrete Cosine Transform Using the Streaming SIMD Extensions and MMXTM Instructions, Intel Application Note, AP-922, Copyright 1999
3. Using Streaming SIMD Extensions 2 (SSE2) to Implement and Inverse Discrete Cosine Transform, Intel Application Note, AP-945, Copyright 2000
4. Rao and Yip, Discrete Cosine Transform Algorithms, Advantages, Applications, Academic Press, Inc., Boston, 1990, Appendix A.2
5. IEEE Standard Specifications for the Implementations of 8x8 Inverse Discrete Cosine Transform, IEEE Std 1180-1990.

About the Author

Richard Hubbard is a Senior Software Engineer and member of the SSG Apple enabling team, working on optimizing Mac OS X* applications for power and performance. Richard holds a Masters degree in Electrical Engineering from Stevens Institute of Technology and a Bachelors in Computer Engineering from New Jersey Institute of Technology.

Wiener Filtering Using Intel® Advanced Vector Extensions

Tue, 26 Jan 2010 00:00:00 -0800

1 Introduction
Intel® Advanced Vector Extensions (Intel® AVX) is a 256 bit instruction set extension to Intel® Streaming SIMD Extensions (Intel® SSE) and is designed for applications that are floating point intensive. These instructions provide a means to accelerate applications that rely heavily on floating-point operations, such as 3D geometry, video processing, image processing, and spatial (3D) audio. This application note discusses Wiener filtering, and includes an example of code that has been optimized using Intel® AVX. The source reference for the code and application note for this article has been AP-807 Wiener Filtering Using Streaming SIMD Extensions [4]. The original article included optimizations using streaming SIMD extensions and this whitepaper discusses migration of the code to 256 bit instruction set extension (i.e. Intel® AVX).

2 The Wiener Filter Algorithm
Wiener filtering (also known as Least Mean Square filtering) is a technique for removing unwanted noise from an image. The description of this algorithm is from The Pocket Handbook of Image Processing Algorithms in C, by Harley R Myler and Aruthur R. Weeks [1]. The algorithm has four (Fourier transformed) vector inputs, representing (one component of) the original image (Image), the degraded image (Guv), the noise image spectra (Noise) and the degradation function (Huv). Each input is a vector of row*col complex numbers. The complex numbers are represented as two contiguous floats for the real and imaginary parts of the number. An additional parameter, gamma, is included in the computation. When gamma is 1.0, the filter is known as non-parametric. The parameters to the filter can be adjusted until the filtered image is satisfactory.

2.1 Applications for Wiener Filters
Wiener filters are commonly used in image processing applications to remove noise from reconstructed images. Wiener filtering is often used to restore a blurry image. However, the Wiener filter has proved important in adaptive filtering, has been used for wavelet transforms, and has found application in communications and other DSP-related disciplines. The reader also should be aware that Fourier transformation is a key element in any signal processing discipline. Please refer to the Intel application note, Split-Radix FFT (AP-808) for further information on implementing a Fourier transform.

2.2 Implementing the Wiener Filter
As described in Section 2.1, the input to the function is four arrays of complex numbers. For each
Element of the image, the following operations are carried out using complex arithmetic. The complex variables D and Hs are intermediate variables used during the computation. The function
Complex_conj is used to take the complex conjugate of a complex number. When divides occur, a check (using an if statement) must be done to ensure that the denominator is non-zero. In the event that a denominator is zero, the result should be set to zero.
1. Complex Noise = gamma * (Noise * Complex_conj ( Noise ) )
2. Complex D = Image * Complex_conj ( Image )
3. Complex D = Noise / D
4. Complex Hs = Huv * Complex_conj ( Huv )
5. Complex Num = Complex_conj ( Huv ) * Guv
6. Complex Image = Num / (Hs + D)

3 Vectorizing the code with 128-bit SIMD
The reference source code for the article (both the C version and 128-bit SIMD version) is available in Intel application note AP-807. The following relates to the original port of the scalar code to 128-bit SIMD. First, the code can be optimized simply by observing that many of the operations involve multiplying a number by its complex conjugate. Since the result has no imaginary component, many of the operations specified in Section 2.2 can be simplified. The resultant C code is given in Section 5. Before the code can be optimized for Intel® SSE, it must be converted to a form suitable for SIMD execution. Four iterations of the original C code are gathered together and processed in a single iteration of the new loop. Each pass through the new loop does the work of four of the original iterations. The list of operations in Section 2.2 shows that three divides must be done per iteration: one divide in Step 3, (the imaginary component requires no divide since the imaginary part is zero), and two divides in Step 6. One of the latter two divides can be removed by noting that they both have the same denominator. The if statement required to check for a zero denominator can be removed by using a masking technique. Further improvements can be obtained by replacing all divides with reciprocal approximations. These techniques are described below. After the code has been converted to a SIMD format, the checks for zero denominators (if statements) can be removed by creating a mask for the non-zero denominator elements, and ANDing that mask to the result of the division to zero out the elements where a division by zero occurred. This technique assumes that QNANs are being generated rather than SNANs, by masking SIMD floating-point exceptions in the MXCSR register, so that no floating point divide by zero exceptions will occur. For example, assume that you want to compute the quantity ( N / D ), where N and D are floats. A typical code sequence is given below.

If ( D != 0 )
Result = N / D;
Else
Result = 0.0;

The result can be computed as (and ( div ( N, D ), cmp_neq ( D, 0 ) ) ), a
computation that does not require an if statement. Using intrinsics, this would be expressed as

mm_and_ps ( mm_div_ps ( N, D ), mm_cmpneq_ps ( D, zero ) ).

This technique is specific to vectorized code, and is implemented in the attached Intrinsics
and assembly language versions of the code.

The Newton-Raphson method is a classic technique for approximating functions. The initial “guess” at the reciprocal is computed using the rcpps instruction. Subsequently, the “guess” is improved using the Newton-Raphson method. The result is not as accurate as what is provided by the divide instruction; however, it can be obtained significantly faster. (Programmers must determine if their application allows a reduced precision answer.) Full details and discussion of this technique are available in the Newton-Raphson application note [2]. The specific code sequence employed in this Wiener filter is given below. (The denominator should be examined to be certain a division by zero is not occurring.)

RC = _mm_rcpps( D );
RECIP = _mm_sub( _mm_add( RC, RC ), _mm_mul( RC, _mm_mul( RC, D ) ) );

4 Vectorizing the code with 256-bit SIMD
The 128-bit vectorized code can be easily ported to 256-bit SIMD by simply migrating the changes. The 256-bit code will do 8 iterations in one loop (the 128-bit Intel® SSE will do 4 iterations in one loop). The load/store instructions as well rcp/mul/add/sub instructions will now operate on 8 data points. The branchless technique described above is retained in the Intel® AVX code as well. As can be seen from the code, using the corresponding Intel® AVX intrinsics (for example _mm256_mul_ps for 256-bit vs _mm_mul_ps for 128-bit) completes the port. Section 6.3 provides the complete 256-bit Intel® AVX code.

5 Grouping input/output arrays
Additionally, the 256-bit SIMD code performance can be improved by grouping input and output arrays in sequential order. Having sequential memory accesses through cache/memory decreases potential cache way conflicts. This provides a simpler accessing pattern for the CPU hardware prefetcher which leads to more accurate data.

6 Conclusion
The performance results (interms of CPU clock cycles) of the 128-bit SIMD code vs 256-bit SIMD code over large number of iterations are as below.

	Intel® AVX (256-bit)	Intel® SSE (128-bit)	Intel® AVX vs. Intel® SSE
Wiener filter	45871	66933	1.46x
Wiener filter with grouped arrays	42464	64473	1.51x

The overall improvement of Intel® AVX over Intel® SSE with Wiener filtering is 1.46x. Intel® AVX can provide a significant performance improvement for the Wiener filter algorithm, compared to coding with 128-bit SIMD. By grouping the input/output arrays, the overall improvement of Intel® AVX over Intel® SSE with Wiener filtering is 1.51x. The improvements cited in this document are the result of several techniques. The techniques include using Intel® AVX (vectorizing the code), and removing conditional branch instructions (if statements) by using masking operations provided by both Intel® SSE and Intel® AVX. If it is acceptable to reduce the numerical precision of the result, then a further gain can be realized by replacing divide operations with reciprocal approximations (employing a Newton-Raphson technique). Further, the article also highlights the ease with which an existing floating point code can be ported to Intel® AVX.

7 Coding Example

/*
* Wiener Filter (also known as the Least Mean Square filter)
*
* Reference: The Pocket Handbook of Image Processing Algorithms in C
* by Harley R Myler & Arthur R. Weeks
* 1993 Prentice-Hall, ISBN 0-13-642240-3 p260-3.
*
* The data is several arrays of complex floats in row major order.
* The description for the algorithm from p260 states:
*
* The algorithm computes a parametric Wiener filter on the
* Fourier transform of a degraded image, Guv, with noise
* spectra N, degradation function Huv, and original image Img.
* The computation is in place, so that the filtered version of
* the input is returned in the original image variable. The
* original and noise images are either estimations form some
* predictive function or ad hoc approximations. If the noise
* image is zero, the process reduces to the inverse filter.
*
* The Weiner parameter gamma is passed to the algorithm.
* If this parameter is 1.0, the filter is non-parametric.
* Methods exist in the literature to derive the parameter value;
* however, it is sometimes determined from trial and error.
*
*NOTE!!!! The code on page 263 has an error. In cxml, the complex
* multiply routine, the imaginary part of the computation should be
* a*d + b*c, not a*d - b*c.
*
*NOTE! (another error) The *complex* array length is rows*cols, so the
* *float* array length should be 2*rows*cols. Also, note that the
* algorithm operates on one component of the pixel.
*/
void wiener_filter ( float *Img,
float *Huv,
float *No,
float *Guv,
float gamma,
int rows,
int cols)
{
int i, sz;
float numr, numi, dr, hsr;
sz = 2 * rows * cols;
for (i = 0; i < sz; i += 2)
{
/* Compute (in place) the noise spectral density with Wiener gamma*/
No[i] = (float) ( gamma * ( No[i]*No[i] + No[i+1]*No[i+1] ) );
No[i+1] = (float) 0.0;
/* Compute image spectral density */
dr = (float) ( Img[i]*Img[i] + Img[i+1]*Img[i+1] );
/* Compute denominator spectral density term */
if (dr != 0.0)
dr = (float) (No[i] / dr) ;
/* Compute degradation power spectrum */
hsr = (float) ( Huv[i]*Huv[i] + Huv[i+1]*Huv[i+1] );
/* Compute numerator term */
numr = (float) ( Huv[i]*Guv[i] + Huv[i+1]*Guv[i+1] );
numi = (float) ( Huv[i]*Guv[i+1] - Huv[i+1]*Guv[i ] );
/* Final computation */
if ( (hsr + dr) != 0.0 )
{
Img[i] = (float) (numr / (hsr + dr));
Img[i+1] = (float) (numi / (hsr + dr));
}
else
{
Img[i] = (float) 0.0;
Img[i+1] = (float) 0.0;
}
}
} /* wiener_filter */

7.2 128-bit Intrinsics Code

/*
#include 
//#define MM_FUNCTIONALITY
#include 
#include 
void intrin_wiener_rcp_sse( float *Img,
float *Huv,
float *No,
float *Guv,
float gamma,
int rows,
int cols )
{
int i, sz;
__m128 first2, next2, nor4, noi4, nr4, inr4, ini4, dr4;
__m128 hr4, hi4, hsr4, gr4, gi4, numr4, numi4;
__m128 rc, denom;
__m128 zero = _mm_set_ps1 (0.0);
sz = 2 * rows * cols;
assert( (sz > 3) & !(sz & 3) );
assert( !( ((int)Img) & 15 ) ); /* Assume alignment */
assert( !( ((int)Huv) & 15 ) );
assert( !( ((int)No) & 15 ) );
assert( !( ((int)Guv) & 15 ) );
for (i = 0; i < sz; i += 8)
{
* Compute (in place) the noise spectral density with Wiener gamma
*
* complex Noise = gamma * (Noise * complex conj Noise)
*
* No[i] = (float) ( gamma * ( No[i]*No[i] + No[i+1]*No[i+1] ) );
* No[i+1] = (float) 0.0;
*/
first2 = _mm_load_ps ( &No[i] );
next2 = _mm_load_ps ( &No[i+4] );
nor4 = _mm_shuffle_ps( first2, next2, 0x88 );
noi4 = _mm_shuffle_ps( first2, next2, 0xdd );
nr4 = _mm_mul_ps ( _mm_set_ps1( gamma ) ,
_mm_add_ps ( _mm_mul_ps( nor4 , nor4 ),
_mm_mul_ps( noi4 , noi4 ) ) );
_mm_store_ps( &No[i ], _mm_unpacklo_ps ( nr4, zero ) );
_mm_store_ps( &No[i+4], _mm_unpackhi_ps ( nr4, zero ) );
/*
* Compute image spectral density
*
* Complex D = Image * complex conj Image
*
* dr = (float) ( Img[i]*Img[i] + Img[i+1]*Img[i+1] );
*/
first2 = _mm_load_ps ( &Img[i] );
next2 = _mm_load_ps ( &Img[i+4] );
inr4 = _mm_shuffle_ps( first2, next2, 0x88 );
ini4 = _mm_shuffle_ps( first2, next2, 0xdd );
dr4 = _mm_add_ps ( _mm_mul_ps( inr4 , inr4),
_mm_mul_ps( ini4 , ini4) );
/*
* Compute denominator spectral density term
*
* Complex D = noise / D
*
* if (dr != 0.0)
* dr = (float) (No[i] / dr) ;
*
* Do that reciprical division thing!
*/
rc = _mm_rcp_ps(dr4);
rc = _mm_sub_ps( _mm_add_ps( rc, rc),
_mm_mul_ps( rc, _mm_mul_ps( rc, dr4) ) );
dr4 = _mm_and_ps ( _mm_mul_ps ( nr4 , rc ),
_mm_cmpneq_ps( dr4, zero ) );
/*
* Compute degradation power spectrum
*
* Complex Hs = Huv * complex conj Huv
*
* hsr = (float) ( Huv[i]*Huv[i] + Huv[i+1]*Huv[i+1] );
*/
first2 = _mm_load_ps ( &Huv[i] );
next2 = _mm_load_ps ( &Huv[i+4] );
hr4 = _mm_shuffle_ps( first2, next2, 0x88 );
hi4 = _mm_shuffle_ps( first2, next2, 0xdd );
hsr4 = _mm_add_ps ( _mm_mul_ps (hr4 , hr4 ),
_mm_mul_ps (hi4 , hi4 ) );
/*
* Compute numerator term
*
* Complex Num = complex conj Huv * Guv
*
* numr = (float) ( Huv[i]*Guv[i] + Huv[i+1]*Guv[i+1] );
* numi = (float) ( Huv[i]*Guv[i+1] - Huv[i+1]*Guv[i ] );
*/
first2 = _mm_load_ps ( &Guv[i] );
next2 = _mm_load_ps ( &Guv[i+4] );
gr4 = _mm_shuffle_ps( first2, next2, 0x88 );
gi4 = _mm_shuffle_ps( first2, next2, 0xdd );
numr4 = _mm_add_ps ( _mm_mul_ps (hr4 , gr4),
_mm_mul_ps (hi4 , gi4) );
numi4 = _mm_sub_ps ( _mm_mul_ps (hr4 , gi4),
_mm_mul_ps (hi4 , gr4) );
/*
* Final computation
*
* Complex Image = Num / (Hs + D)
*
* if ( (hsr + dr) != 0.0 )
* {
* Img[i] = (float) (numr / (hsr + dr));
* Img[i+1] = (float) (numi / (hsr + dr));
* }
* else
* {
* Img[i] = (float) 0.0;
* Img[i+1] = (float) 0.0;
* }
*
* Do the reciprical division thing
*/
denom = _mm_add_ps( hsr4, dr4 );
rc = _mm_rcp_ps(denom);
rc = _mm_sub_ps( _mm_add_ps( rc, rc),
_mm_mul_ps( rc, _mm_mul_ps( rc, denom) ) );
inr4 = _mm_and_ps( _mm_mul_ps ( numr4 , rc ) ,
_mm_cmpneq_ps( denom, zero ) );
ini4 = _mm_and_ps( _mm_mul_ps ( numi4 , rc ) ,
_mm_cmpneq_ps( denom, zero ) );
_mm_store_ps( &Img[i ], _mm_unpacklo_ps ( inr4, ini4 ) );
_mm_store_ps( &Img[i+4], _mm_unpackhi_ps ( inr4, ini4 ) );
}
} /* intrin_wiener_rcp */

7.3 256-bit Intrinsics Code

void intrin_wiener_rcp_avx( float *Img,
					   float *Huv,
					   float *No,
					   float *Guv,
					   float gamma,
					   int rows,
					   int cols )
{
	int i, sz;
	__m256 first2, next2, nor4, noi4, nr4, inr4, ini4, dr4;
	__m256 hr4, hi4, hsr4, gr4, gi4, numr4, numi4;
	__m256 rc, denom;
	__m256 zero = _mm256_setzero_ps();
	sz = 2 * rows * cols;
	assert( (sz > 3) & !(sz & 3) );
	assert( !( ((int)Img) & 15 ) ); /* Assume alignment */
	assert( !( ((int)Huv) & 15 ) );
	assert( !( ((int)No) & 15 ) );
	assert( !( ((int)Guv) & 15 ) );
	for (i = 0; i < sz; i += 16)
	{
		* Compute (in place) the noise spectral density with Wiener gamma
		*
		* complex Noise = gamma * (Noise * complex conj Noise)
		*
		* No[i] = (float) ( gamma * ( No[i]*No[i] + No[i+1]*No[i+1] ) );
		* No[i+1] = (float) 0.0;
		*/
		first2 = _mm256_load_ps ( &No[i] );
		next2 = _mm256_load_ps ( &No[i+4*2] );
		nor4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		noi4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		nr4 = _mm256_mul_ps ( _mm256_set1_ps( gamma ) ,
		_mm256_add_ps ( _mm256_mul_ps( nor4 , nor4 ),
		_mm256_mul_ps( noi4 , noi4 ) ) );
		_mm256_store_ps( &No[i ], _mm256_unpacklo_ps ( nr4, zero ) );
		_mm256_store_ps( &No[i+4*2], _mm256_unpackhi_ps ( nr4, zero ) );
		
		/*
		* Compute image spectral density
		*
		* Complex D = Image * complex conj Image
		*
		* dr = (float) ( Img[i]*Img[i] + Img[i+1]*Img[i+1] );
		*/
		first2 = _mm256_load_ps ( &Img[i] );
		next2 = _mm256_load_ps ( &Img[i+4*2] );
		inr4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		ini4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		dr4 = _mm256_add_ps ( _mm256_mul_ps( inr4 , inr4),
		_mm256_mul_ps( ini4 , ini4) );
		/*
		* Compute denominator spectral density term
		*
		* Complex D = noise / D
		*
		* if (dr != 0.0)
		* dr = (float) (No[i] / dr) ;
		*
		* Do that reciprical division thing!
		*/
		rc = _mm256_rcp_ps(dr4);
		rc = _mm256_sub_ps( _mm256_add_ps( rc, rc),
		_mm256_mul_ps( rc, _mm256_mul_ps( rc, dr4) ) );
		dr4 = _mm256_and_ps ( _mm256_mul_ps ( nr4 , rc ),
		_mm256_cmpneq_ps( dr4, zero ) );
		/*
		* Compute degradation power spectrum
		*
		* Complex Hs = Huv * complex conj Huv
		*
		* hsr = (float) ( Huv[i]*Huv[i] + Huv[i+1]*Huv[i+1] );
		*/
		first2 = _mm256_load_ps ( &Huv[i] );
		next2 = _mm256_load_ps ( &Huv[i+4*2] );
		hr4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		hi4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		hsr4 = _mm256_add_ps ( _mm256_mul_ps (hr4 , hr4 ),
		_mm256_mul_ps (hi4 , hi4 ) );
		/*
		* Compute numerator term
		*
		* Complex Num = complex conj Huv * Guv
		*
		* numr = (float) ( Huv[i]*Guv[i] + Huv[i+1]*Guv[i+1] );
		* numi = (float) ( Huv[i]*Guv[i+1] - Huv[i+1]*Guv[i ] );
		*/
		first2 = _mm256_load_ps ( &Guv[i] );
		next2 = _mm256_load_ps ( &Guv[i+4*2] );
		gr4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		gi4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		numr4 = _mm256_add_ps ( _mm256_mul_ps (hr4 , gr4),
		_mm256_mul_ps (hi4 , gi4) );
		numi4 = _mm256_sub_ps ( _mm256_mul_ps (hr4 , gi4),
		_mm256_mul_ps (hi4 , gr4) );
		/*
		* Final computation
		*
		* Complex Image = Num / (Hs + D)
		*
		* if ( (hsr + dr) != 0.0 )
		AP-807 Wiener Filtering Using Streaming SIMD Extensions
		01/28/99 15
		* {
		* Img[i] = (float) (numr / (hsr + dr));
		* Img[i+1] = (float) (numi / (hsr + dr));
		* }
		* else
		* {
		* Img[i] = (float) 0.0;
		* Img[i+1] = (float) 0.0;
		* }
		*
		* Do the reciprical division thing
		*/
		denom = _mm256_add_ps( hsr4, dr4 );
		rc = _mm256_rcp_ps(denom);
		rc = _mm256_sub_ps( _mm256_add_ps( rc, rc),
		_mm256_mul_ps( rc, _mm256_mul_ps( rc, denom) ) );
		inr4 = _mm256_and_ps( _mm256_mul_ps ( numr4 , rc ) ,
		_mm256_cmpneq_ps( denom, zero ) );
		ini4 = _mm256_and_ps( _mm256_mul_ps ( numi4 , rc ) ,
		_mm256_cmpneq_ps( denom, zero ) );
		_mm256_store_ps( &Img[i ], _mm256_unpacklo_ps ( inr4, ini4 ) );
		_mm256_store_ps( &Img[i+4*2], _mm256_unpackhi_ps ( inr4, ini4 ) );

	}
} /* intrin_wiener_rcp */

7.4 256-bit Intrinsics Code with grouped arrays

blockHNG Structure

Huv[0]

…

Huv[15]

No[0]

…

No[15]

Guv[0]

…

Guv[15]

Huv[16]

…

Huv[31]

No[16]

…

void intrin_wiener_rcp_avx ( float *Img,
					float *_blockHNG,
					float gamma,
					int rows,
					int cols)
{
	int sz;
	__m256 first2, next2, nor4, noi4, nr4, inr4, ini4, dr4;
	__m256 hr4, hi4, hsr4, gr4, gi4, numr4, numi4;
	__m256 rc, denom;
	__m256 zero = _mm256_setzero_ps();
	sz = 2 * rows * cols;

	assert( (sz > 3) & !(sz & 3) );
	assert( !( ((int)Img) & 15 ) ); // Assume alignment 
	assert( !( ((int)_blockHNG) & 15 ) ); // Assume alignment 

	float *Huv;
	float *No;
	float *Guv;
	
	int j = 0;	// img index
	for (int _blockHNG_tracker = 0; _blockHNG_tracker < 2 * rows * cols * 3; _blockHNG_tracker += 48)
	{
		Huv = &(_blockHNG[_blockHNG_tracker]);
		No = &(_blockHNG[_blockHNG_tracker + 16]);
		Guv = &(_blockHNG[_blockHNG_tracker + 32]);

		/*
		* Compute (in place) the noise spectral density with Wiener gamma
		*
		* complex Noise = gamma * (Noise * complex conj Noise)
		*
		AP-807 Wiener Filtering Using Streaming SIMD Extensions
		01/28/99 13
		* No[i] = (float) ( gamma * ( No[i]*No[i] + No[i+1]*No[i+1] ) );
		* No[i+1] = (float) 0.0;
		*/
		first2 = _mm256_load_ps ( &No[0] );
		next2 = _mm256_load_ps ( &No[8] );
		nor4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		noi4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		nr4 = _mm256_mul_ps ( _mm256_set1_ps( gamma ) ,
		_mm256_add_ps ( _mm256_mul_ps( nor4 , nor4 ),
		_mm256_mul_ps( noi4 , noi4 ) ) );

		_mm256_store_ps( &No[0], _mm256_unpacklo_ps ( nr4, zero ) );
		_mm256_store_ps( &No[8], _mm256_unpackhi_ps ( nr4, zero ) );

		/*
		* Compute image spectral density
		*
		* Complex D = Image * complex conj Image
		*
		* dr = (float) ( Img[i]*Img[i] + Img[i+1]*Img[i+1] );
		*/
		first2 = _mm256_load_ps ( &Img[j] );
		next2 = _mm256_load_ps ( &Img[j+8] );
		inr4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		ini4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		dr4 = _mm256_add_ps ( _mm256_mul_ps( inr4 , inr4),
		_mm256_mul_ps( ini4 , ini4) );
		/*
		* Compute denominator spectral density term
		*
		* Complex D = noise / D
		*
		* if (dr != 0.0)
		* dr = (float) (No[i] / dr) ;
		*
		* Do that reciprical division thing!
		*/
		rc = _mm256_rcp_ps(dr4);
		rc = _mm256_sub_ps( _mm256_add_ps( rc, rc),
		_mm256_mul_ps( rc, _mm256_mul_ps( rc, dr4) ) );
		dr4 = _mm256_and_ps ( _mm256_mul_ps ( nr4 , rc ),
		_mm256_cmpneq_ps( dr4, zero ) );
		/*
		* Compute degradation power spectrum
		*
		* Complex Hs = Huv * complex conj Huv
		*
		* hsr = (float) ( Huv[i]*Huv[i] + Huv[i+1]*Huv[i+1] );
		*/
		first2 = _mm256_load_ps ( &Huv[0] );
		next2 = _mm256_load_ps ( &Huv[8] );
		hr4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		hi4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		hsr4 = _mm256_add_ps ( _mm256_mul_ps (hr4 , hr4 ),
		_mm256_mul_ps (hi4 , hi4 ) );
		/*
		* Compute numerator term
		*
		* Complex Num = complex conj Huv * Guv
		*
		* numr = (float) ( Huv[i]*Guv[i] + Huv[i+1]*Guv[i+1] );
		* numi = (float) ( Huv[i]*Guv[i+1] - Huv[i+1]*Guv[i ] );
		*/
		first2 = _mm256_load_ps ( &Guv[0] );
		next2 = _mm256_load_ps ( &Guv[8] );
		gr4 = _mm256_shuffle_ps( first2, next2, 0x88 );
		gi4 = _mm256_shuffle_ps( first2, next2, 0xdd );
		numr4 = _mm256_add_ps ( _mm256_mul_ps (hr4 , gr4),
		_mm256_mul_ps (hi4 , gi4) );
		numi4 = _mm256_sub_ps ( _mm256_mul_ps (hr4 , gi4),
		_mm256_mul_ps (hi4 , gr4) );
		/*
		* Final computation
		*
		* Complex Image = Num / (Hs + D)
		*
		* if ( (hsr + dr) != 0.0 )
		AP-807 Wiener Filtering Using Streaming SIMD Extensions
		01/28/99 15
		* {
		* Img[i] = (float) (numr / (hsr + dr));
		* Img[i+1] = (float) (numi / (hsr + dr));
		* }
		* else
		* {
		* Img[i] = (float) 0.0;
		* Img[i+1] = (float) 0.0;
		* }
		*
		* Do the reciprical division thing
		*/
		denom = _mm256_add_ps( hsr4, dr4 );
		rc = _mm256_rcp_ps(denom);
		rc = _mm256_sub_ps( _mm256_add_ps( rc, rc),
		_mm256_mul_ps( rc, _mm256_mul_ps( rc, denom) ) );
		inr4 = _mm256_and_ps( _mm256_mul_ps ( numr4 , rc ) ,
		_mm256_cmpneq_ps( denom, zero ) );
		ini4 = _mm256_and_ps( _mm256_mul_ps ( numi4 , rc ) ,
		_mm256_cmpneq_ps( denom, zero ) );

		_mm256_store_ps( &Img[j ], _mm256_unpacklo_ps ( inr4, ini4 ) );
		_mm256_store_ps( &Img[j+8], _mm256_unpackhi_ps ( inr4, ini4 ) );
		j+=16;
	}
} /* Intrin_wiener_rcp_avx */

Acknowledgements
The authors would like to thank Phil Kerly, Raghu Muthyalampalli and Justin Landon who assisted with the performance assessment of the code, providing performance recommendations and review of the whitepaper.

References
The following documents are referenced in this application note, and provide background or supporting information for understanding the topics presented in this document.

1. The Pocket Handbook of Image Processing Algorithms in C, by Harley R Myler and Aruthur R.
Weeks. ISBN 0-13-642240-3.
2. Increasing the Accuracy of the Results from the Reciprocal and Reciprocal Square Root
Instructions using the Newton-Raphson Method, Intel application note (AP-803, Order Number: 243637-001).
3. Split-Radix FFT, Intel application note (AP-808, Order Number: 243642-001).
4. Wiener Filtering Using Streaming SIMD Extensions, Intel application note (AP-807).

Kit Chung

Karthikeyan Krishnan

01-25-2010

Tech Articles

Intel® SSE

Intel® AVX

Wiener filtering is a technique for removing unwanted noise from an image. Wiener filtering is often used to restore a blurry image. Intel® AVX is a SIMD 256-bit instruction set that improves upon the existing Intel® SSE 128-bit instruction set. This article shows an Intel AVX Wiener filter that has 1.46x performance relative to the Intel SSE Wiener filter.

Intel Software Network articles Feed

Improving the Compute Performance of Video Processing Software Using AVX (Advanced Vector Extensions) Instructions (by Eli Hernandez and Larry Moore)

Download Article

Abstract

1. Introduction

2. Intel SIMD Overview

3. Video Processing Code

3.1. Desaturation - Sample Code

3.2. Desaturation - Optimization with AVX

3.3 Desaturation - Performance Test Results3

4. Packed Integer Conversion Instructions

5. Conclusion

About the Authors

Appendix A: Inner loop quivalent assembly of the serial code

Appendix B: Equivalent assembly of inner loop optimized with AVX

Introduction to Intel® Advanced Vector Extensions

Download Article

Instruction Set Overview

Intel® AVX Instruction Classes

Availability and Support

Usage

Mandelbrot Example

Conclusion

For More Information

About the Author

Appendix A: Instruction Set Reference

Practical Intel® AVX Optimization on 2nd generation Intel® Core™ Processors

Download Article

Introduction

Identify your hotspots

SIMD (Single Instruction, Multiple Data) with Intel® AVX implementation

Implementing A Linear Phase FIR Filter using Intel® AVX

Cache alignment

Intel® Compilers

Important notice for using Intel® AVX:

Summary and Results

About the Authors

Developing Intel® AVX Optimized Microsoft* Real-Time Audio (MSRTA) Codec using Intel® IPP

Download Article

Introduction

Unified Speech Component

Intel® IPP functions optimized for Intel® Advanced Vector Extensions

Microsoft* Real-Time Audio Codec (MSRTA)

Intel® IPP Real Time Audio functions

USC MSRTA Codec

Summary

Appendix A: Introduction to Intel® IPP

Intel® IPP Speech Codec

Appendix B: Intel® AVX-optimized Intel® IPP RT Audio Functions

Appendix C: USC MSRTA Codec Description

Additional Resources

Don't Spill That Register - Ensuring Optimal Performance From Intrinsics

Download Article

Objective

Abstract

Introduction

Generated Assembly

Register Shortage

Conclusion

3D Vector Normalization Using 256-Bit Intel® Advanced Vector Extensions (Intel® AVX)

Download Article

Objective

Abstract

Introduction

The 128-bit and 256-bit AOS to SOA shuffle

Testing and Results

Conclusion

About The Author

Appendix: Transpose Source Code

Intel® AVX C/C++ Intrinsics Emulation

Using Intel® Advanced Vector Extensions to Implement an Inverse Discrete Cosine Transform

Download Article

Introduction

Testing Environment

Discrete Cosine Transforms

Previous Work and Specifications

Intel® AVX Implementation of IDCT

Using Intel® Architecture Code Analyzer

Results

Conclusion

3.3 Desaturation - Performance Test Results³