Third Generation Intel® Xeon® Processor Scalable Family Technical Overview

Introduction

This paper discusses the new features and enhancements available in the third generation Intel® Xeon® processor Scalable family and how developers can take advantage of them. The latest processor and platform provide additional new technologies, including BFLOAT16 to help with deep learning training, the second generation of Intel® Optane™ DC persistent memory, Intel® Platform Firmware Resilience (Intel® PFR) and Intel® Speed Select Technology - Turbo Frequency (Intel® SST-TF).

The table below provides a comparison between the second and third generation of the Intel® Xeon® processor Scalable family. The third generation contains many of the same features as found in the second generation. New capabilities/changes relative to the previous generation are in Bolded Italic.

Table 1. The Next-Generation Intel® Xeon® Processor Scalable Family Microarchitecture Overview

  Second Generation Intel® Xeon® processor Scalable family with Intel® C620 series chipset Third Generation Intel® Xeon® processor Scalable family with Intel® C620A series chipset
Socket Count 1, 2, 4, 8 4 and 8 glue less
Die Size 14nm
Processor Core Count Up to 28 cores (56 cores with Intel® Hyper-Threading Technology (Intel® HT Technology)) per socket Up to 28 cores (56 cores with Intel® Hyper-Threading Technology (Intel® HT Technology)) per socket
TDP Up to 205W Up to 250W
New Features N/A BFLOAT16, Intel® PFR, Intel® SST-TF on select skus, next generation of Intel® Virtual RAID on CPU (Intel® VROC), Converged Intel® Boot Guard and Intel® Trusted Execution Technology (Intel® TXT)
Socket Type Socket P Socket P+
Memory DDR4 Up to 6 channels DDR4 per CPU, up to 12 DIMMs per socket, up to 2666 MT/s 2DPC, up to 2933 MT/s 1DPC Up to 6 channels DDR4 per CPU, up to 12 DIMMs per socket, up to 2933 MT/s 2DPC, up to 3200 MT/s 1DPC
Number of Intel® Ultra Path Interconnect (Intel® UPI) Interconnects Up to 3 links per CPU Up to 6 links per CPU
Intel® UPI Interconnect Speed Up to 10.4 GT/s
PCIe PCIe Gen 3: up to 48 lanes per CPU (bifurcation support: x16, x8, x4) PCIe Gen 3: up to 48 lanes per CPU (bifurcation support: x16, x8, x4)
Chipset Features Intel® QuickAssist Technology (Intel® QAT)
Enhanced Serial Peripheral Interface (eSPI)
Up to 20 ports PCIe 3.0 (8 GT/s)
Up to 14 SATA 3, up to 14 USB 2.0, up to 10 USB 3.0

 

an increase in the number of Intel U P I pathways as compared to the previous generation
Figure 1. Third Generation Intel® Xeon® processor Scalable family with Intel® C620A Series Chipset on a four-socket platform. There is an increase in the number of Intel® UPI pathways as compared to the previous generation, which can help with workloads that rely on remote memory accesses.

Intel® Optane™ persistent memory 200 series

The next generation of Intel® Optane™ persistent memory modules are a form of large capacity memory that can function in either a volatile or non-volatile state and provides a new tier positioned between DRAM and storage. Traditional memory in computer architecture is volatile. The content of volatile memory is only able to exist so long as the system supplies power. If there is a power loss, the data is immediately lost. Persistent memory maintains the integrity of the information stored in it, even when power has been lost or cycled on the system. The persistent memory is byte addressable, cache coherent, and provides software with direct access to persistence without paging.

The table below provides a memory comparison between the first and next generation of persistent memory. New capabilities/changes relative to the previous generation in Bolded Italic.

Table 2. Intel® Optane™ persistent memory overview

Intel® Optane™ DC Persistent Memory Family Intel® Optane™ persistent memory 100 series Intel® Optane™ persistent memory 200 series
Supported Number of Sockets 2, 4, and 8 sockets 4 sockets
Number of Channels per Socket Up to 6 Up to 8
DIMM Capacity 128GB, 256GB, 512GB, 3 TB per socket
Platform Capacity Up to 3TB per socket Up to 4TB per socket
DDR-T Speeds Up to 2666 MT/sec Up to 3200 MT/sec
TDP Power Sustained 18W 12-15W
Data Persistence in Power Failure Event ADR ADR, eADR (Optional)
Intel® Memory Bandwidth Boost N/A Smart boost to maximum TDP (varies by capacity)
Operating Modes App Direct (AD), Memory Mode (MM),
Mixed Mode (AD + MM)

 

Intel® Memory Bandwidth Boost

Intel® Memory Bandwidth Boost is a new feature of the next generation Intel® Optane™ persistent memory that provides a dynamic range of power and bandwidth when thermal headroom is available. The system can monitor the temperature of each memory module to profile its state of utilization. When the memory module is underutilized, the temperature will be lower and will have availability for additional memory bandwidth. If the workload on the system requires additional memory bandwidth beyond its normal sustained bandwidth rate it will be provided temporarily. This allows the memory bandwidth to burst up to higher rates temporarily until the memory hits a temperature threshold around 83 degrees Celsius. When that temperature limit is approached Intel® Memory Bandwidth Boost will throttle back the power of the memory lowering the memory bandwidth to its normal sustainable rate of operation. Intel® Memory Bandwidth Boost will not be available again for the memory module until the temperature cools down.

Since the Intel® Memory Bandwidth Boost feature cannot be sustained indefinitely a determination may need to be made for systems that are dedicated to a specific workload or usage model. There is an adjustable average time constant that can be set within the BIOS to determine how long the Intel® Memory Bandwidth Boost feature should remain active. Longer active periods will require longer cool down periods. Therefore, simply setting the average time constant so that Intel® Memory Bandwidth Boost runs for as long as it can may not produce the ideal result. Sometimes shorter active periods followed by shorter rest periods may end up being more optimal for a given workload.

The feature was originally designed to help with adoption and use of persistent memory across a wider range of platform thermal design power. From a software developer perspective, the memory bandwidth boost is a background feature. It can be enabled, disabled and adjusted via the average time constant within the BIOS. Intel® Memory Bandwidth Boost works across all available memory modes and interleaved sets.

Table 3. Memory Module TDP Comparisons

  128G Memory Module 256G and 512G Memory Modules
TDP Power - Sustained 12-15W 15W
TDP Power – Burst Burst up to 15W if sustained TDP is set lower than 15W. 18W

 

Enhanced - Asynchronous DRAM Refresh (eADR)

In the first generation of persistent memory a feature known as ADR (Asynchronous DRAM Refresh) causes a hardware interrupt on the memory controller that flushes the write protected data buffers. This feature helps to ensure that any data that can make it to the write pending queue (WPQ) on the memory controller will make it into persistent memory. This is done to protect data in the event of a power failure. ADR protects data only at the memory subsystem level it does not flush the processor caches. Cache flushes need to be done by applications using either the CLWB, CLFLUSH, CLFLUSHOPT, Non-Temporal Stores, or WBINVD machine instructions. This functionality still exists with the second generation of persistent memory, but an additional new feature has been added called eADR.

eADR extends the protection from the memory subsystem to the processor caches in the event of a power failure. An NMI routine needs to be initiated to flush the processor caches which can then be followed by an ADR event. Applications using the Persistent Memory Development Kit (PMDK) will detect if eADR is present and do not need to perform flush operations. An SFENCE operation is still required which maintains persistence for globally visible stores. eADR does require that the OEM provide additional stored energy such as a backup battery to specifically allow for this functionality.

Intel® Optane™ persistent memory management tools and resources

Linux* Management Tools for Intel® Optane™ DC Persistent Memory

ndctl - Manage the Linux LIBNVDIMM kernel subsystem.

pmemcheck - Perform a dynamic runtime analysis with an enhanced version of Valgrind.

pmempool - Manage and analyze persistent memory pools with this stand-alone utility.

FIO - Run benchmarks with FIO.

pmembench - Build and run PMDK benchmarks.

Windows* Server manageability is provided by Microsoft* through PowerShell*.

Persistent Memory Development Zone

Intel® Speed Select Technology (Intel® SST)

Often in the cloud service provider and enterprise environments multiple servers are purchased to handle diverse workloads and usages. This can increase total cost of ownership due to power consumption, complexity in management, ensuring all the systems are fully utilized, etc. Intel® Speed select technology is meant to provide a collection of features that help improve this situation by providing a way to prioritize processor attributes.

Two available features on the third generation Intel® Xeon® processor Scalable family include Intel® Speed Select Technology-Turbo Frequency (Intel® SST-TF) and Intel® Speed Select Technology-Core Power (Intel® SST-CP). Their availability is limited to the network focused processor SKUs. The features are discoverable using software tools enabled for Linux.

Intel® Speed Select Technology-Turbo Frequency (Intel® SST-TF)

Intel® SST-TF is a new feature that allows some of the cores to be tagged as high priority providing them with a turbo frequency that exceeds the nominal turbo frequency limits when all cores are active. The overall frequency envelope for the processor socket stays the same but how much frequency a specific core gets can deviate from the specified all core turbo frequency values. All the cores on the processor socket will remain active but the frequency envelope will shift providing higher frequencies to high priority cores, while low priority cores will drop proportionally in frequency to compensate. This feature can be turned on, off or adjusted on a per core basis at runtime.

A potential use case for this feature would be to combine it with Global Extensible Open Power Management (GEOPM). GEOPM is an open source power management runtime and framework that is focused on improving performance and efficiency of HPC workloads. Typically, when an HPC workload schedules a job it will do so in a serialized fashion. Intel® SST can adjust the turbo frequency of that single thread beyond its normal limit to maximum performance even when C-states are disabled on the platform.

single high priority core receiving additional turbo frequency
Figure 2. Illustration of single high priority core receiving additional turbo frequency when a surplus of turbo frequency is available.

When the workload shifts into parallel hardware threads during the execution phase Intel® SST and GEOPM can also provide a benefit. Invariably some of the parallel threads will be running slower than others, these threads can be given additional turbo frequency as needed.

Similar to GEOPM, Intel® SST-TF technology can be applied and exercised in any asymmetric or heterogenous workload scenario. Intel® SST-TF can also help in traditional cloud environments where certain virtual machines requiring higher service-level agreements can be scheduled to run on Intel® SST-TF enabled high priority cores while the rest of the cores can be scheduled with general purpose virtual machines.

Intel® Speed Select Technology - Core Power (Intel® SST-CP)

overview of Intel S S T - C P in operation
Figure 3. Illustration showing an overview of Intel® SST-CP in operation

Intel® SST-CP is the foundation feature for Intel® SST-TF where the concept of prioritization of cores is supported. Intel® SST-CP allows configuring cores into priority groups with each group identified with a specific class of service level characterized by a priority weight, minimum and maximum frequencies. When there is a surplus of processor power/frequency available, the power control unit (PCU) will distribute the surplus amongst the available cores. The surplus of processor power/frequency available is distributed based on the defined weights that have been assigned to the cores. In figure x you can see that the higher priority core receives the surplus frequency first. When Intel® SST-CP is disabled the PCU will simply distribute the surplus of frequency to the various cores without any concern for prioritizing the work being done by a given core.

In order to optimize the result with Intel® SST-CP considerations will need to be made for prioritized cores. For example, spreading out the physical location of high priority cores across NUMA nodes on a processor socket will help to minimize thermal issues. This feature will require workload-based tuning.

Communications based workloads are moving towards more virtual network functions. For use cases like this some of the higher performing virtual functions can end up forming bottlenecks on just a few cores. Intel® SST-CP can help alleviate pressure on the stressed cores and enhance the usage of those virtual network functions.

Enabling Intel® Speed Select Technology

Configuration can be done through third-party orchestration software utilizing the Intel® Speed Select Technology tool that’s part of the latest Linux kernel tools repository. Configuration can also be done by directly interfacing with the hardware exposed register interfaces.

Download scripts for exposing and provisioning Intel® Speed Select Technology features from GitHub.

Below is a list of articles on Intel® Speed Select Technology - Base Frequency (Intel® SST-BF). They cover enabling and configuration examples for different use cases. Even though Intel® SST-BF is not covered here, the family of Intel® Speed Select Technology features follows a similar enabling/configuration flow.

Intel® Speed Select Technology – Base Frequency Priority CPU Management for Open vSwitch* (OVS*)

Intel® Speed Select Technology – Base Frequency - Enhancing Performance

Intel® Speed Select - Base Frequency for Openstack* - software code for setup, automation, provisioning

Intel® Speed Select Technology – Base Frequency Configuration Automation on OpenStack* Compute Host – deployment guide

Intel® Speed Select Technology - Base Frequency (Intel® SST-BF) with Kubernetes* - deployment guide

Intel® Speed Select Technology - Base Frequency (Intel® SST-BF) – specification for support on Openstack* to split CPU cores into shared or dedicated sets based on administrative criteria

Intel® Speed Select Technology - Base Frequency (Intel® SST-BF) – source code / API for splitting CPU cores into shared or dedicated sets on Openstack*

Python* script that allows configuration of Intel® Speed Select Technology - Base Frequency (Intel® SST-BF)

Intel® Speed Select Technology - Base Frequency (Intel® SST-BF) – enabling guide (Bios and OS)

Intel® Virtual RAID on CPU (Intel® VROC)

Intel V R O C replaces RAID add-on cards
Figure 4. Intel® VROC replaces RAID add-on cards

Intel® VROC is a software solution that integrates with a hardware technology called Intel® Volume Management Device (Intel® VMD) to provide a compelling hybrid RAID solution for NVMe* (Non-Volatile Memory Express*) solid-state drives (SSDs). The CPU has onboard capabilities that work more closely with the chipset to provide quick access to the directly attached NVMe SSDs on the PCIe lanes of the platform. Since Intel® VROC is an integrated RAID solution leveraging technologies within the HW of the platform, features like hot insert and Bootable RAID are available even if the OS doesn’t provide it. This robust NVMe ecosystem with RAID and SSD management capabilities is a compelling alternative to RAID HBAs, therefore helping improve platform BOM costs and better preparing users to move to NVMe SSDs.

Intel® VMD is a technology designed primarily to improve the management of high-speed SSDs. Previously SSDs were attached to a SATA or other interface types and managing them through software or a discrete HBA was acceptable. When the industry moves toward faster NVMe SSDs over a PCIe interface in order to improve bandwidth, the discrete HBA adds delays and bottlenecks while a pure software solution is incomplete for most enterprise users. Intel® VMD with Intel® VROC uses hardware and software together to mitigate these issues.

Intel® VMD and Intel® VROC have been around since the first generation of Intel® Xeon® Scalable processors.  Additional information can be found on the Intel® VROC website or the Intel® VROC support site.

New Deep Learning Instruction - Brain Floating Point Format (BFLOAT16)

intel deep learning boost with new b float 16 instructions

Intel® Deep Learning Boost (Intel® DL Boost) was introduced on the previous generation of silicon to help with improving the performance of the inference aspect of AI workloads. Bfloat16 is a new instruction that is targeted at improving the performance of the training aspect for AI workloads. The instruction is part of Intel® Advanced Vector Extensions 512 (Intel® AVX-512) including the AVX-512F, AVX-512VL and AVX-512_BF16 instruction sets. Software developers whose applications are already working with the foundation set will find very little enabling is needed. The Intel® Deep Neural Network Library (Intel® DNNL) allows for integration and optimization into many popular AI frameworks such as Tensorflow, apache MXnet, etc. The Intel® oneAPI Deep Neural Network Library (oneDNN) also supports BF16.

Comparison between floating point 32, floating point 16 and Bfloat16 instructions
Figure 5. Comparison between floating point 32, floating point 16 and Bfloat16 instructions.

In figure 5 you can see a comparison between the different floating point instructions. You will notice that you can execute 2x floating point 16 (FP16) instructions in the same amount of time as a single floating point 32 (FP32) instruction. Bfloat16 tries to give you the best of both FP16 and FP32. It provides the throughput of FP16 with a similar dynamic range and performance of FP32. Range tends to be more important than precision for AI applications. To achieve good numerical behavior at an application level it is essential to accumulate to FP32. An example of this is when you multiply two BFP16 using the vdpbf16ps instruction. More information about BFP16 instructions can be found in the Intel® Architecture Instruction Set Extensions and Future Features Programming Reference.

Additional Bfloat16 Resources

Code Sample: Intel® Deep Learning Boost New Deep Learning Instruction bfloat16 - Intrinsic Functions

Lower Numerical Precision Deep Learning Inference and Training

Whitepaper: BFLOAT16 – Hardware Numerics Definition

Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations

Intel® Platform Firmware Resilience (Intel® PFR)

Intel® Platform Resilience is designed to protect, detect and correct against security threats such as permanent denial of service attacks. In a PDOS, the hardware is attacked with the intent to render the system permanently inoperable, such as by corrupting the system firmware in a manner that is not recoverable. This is a growing threat against critical infrastructure systems such as those associated with the power grid, banks, and other utilities.

Intel® PFR uses a built in Intel® MAX® 10 FPGA to improve protection against security threats. The FPGA along with soft-IP is used as the primary root of trust. The soft-IP enables visibility and flexibility in the design allowing for optional customizations to deal with changes in hardware, firmware or customer needs. This flexibility is of value for example when switching to a different bios chip manufacturer. The FPGA helps protect the firmware by attesting that it is safe prior to executing the code. It also engages in boot and runtime monitoring to assure the server is running known good firmware for various aspects of the system such as the BIOS, BMC, Intel ME, SPI Descriptor and the firmware on the power supply. One of the more interesting aspects is that the FPGA can provide for an automated recovery if corrupted firmware is detected, which previously required manual intervention.

Intel® PFR meets the NIST 800-193 specification for firmware resiliency. Support for the feature is included in the Intel® Security Libraries for Data Center (Intel® SecL - DC).

Intel® Security Libraries for Data Center (Intel® SecL - DC)

Intel® SecL - DC consists of software components providing end-to-end cloud security solutions with integrated libraries. Users have the flexibility to either develop their customized security solutions with the provided libraries or deploy the software components in their existing infrastructure.

There are new security features that are supported in Intel® SecL - DC. Through the Platform Control Registers (PCR) within the Trusted Platform Module (TPM), the remote attestation functionality can be extended to include files and folders on a Linux host system and are included in determining the host’s overall trust status. Lastly, virtual machines and Docker container images can be encrypted at rest, with key access tied to platform integrity attestation. Because security attributes contained in the platform integrity attestation report are used to control access to the decryption keys, this feature provides protection for at-rest data, IP, code, in Docker container or virtual machine images.

The Author: David Mulnix is a software engineer and has been with Intel Corporation for over 25 years. His areas of focus include software automation, server power, performance analysis, and he was one of the contributors of the Server Efficiency Rating Tool™.

Contributors: Andy Rudoff, Kartik Ananth and Vasudevan Srinivasan. 
 

Informações de produto e desempenho

1

Os compiladores da Intel podem ou não otimizar para o mesmo nível de microprocessadores não Intel no caso de otimizações que não são exclusivas para microprocessadores Intel. Essas otimizações incluem os conjuntos de instruções SSE2, SSE3 e SSSE3, e outras otimizações. A Intel não garante a disponibilidade, a funcionalidade ou eficácia de qualquer otimização sobre microprocessadores não fabricados pela Intel. As otimizações que dependem de microprocessadores neste produto são destinadas ao uso com microprocessadores Intel. Algumas otimizações não específicas da microarquitetura Intel são reservadas para os microprocessadores Intel. Consulte os Guias de Usuário e Referência do produto aplicáveis para obter mais informações sobre os conjuntos de instruções específicos cobertos por este aviso.

Revisão do aviso #20110804