DAOS | High-Performance Software Defined Storage over Persistent Memory

Overview

The Distributed Asynchronous Object Storage (DAOS) is an open-source software-defined object store designed from the ground up for massively distributed Non Volatile Memory (NVM). DAOS takes advantage of  next-generation  NVM technology like Storage Class Memory (SCM) and NVM express (NVMe) while presenting a key-value storage interface and providing features such as transactional non-blocking I/O, advanced data protection with self-healing on top of commodity hardware, end-to-end data integrity,  fine-grained  data control and elastic storage to optimize performance and cost.

Source code on GitHub

Community Mailing on Groups.io

DAOS Wiki

Persistent Memory Developer Kit (PMDK)

PMEM YouTube Playlist

Subscribe to the Intel Software YouTube Channel

Transcript

Hi I'm Micah Bhakti from Intel and in this video, I'm going to share a bit about DAOS, our software defined storage engine, designed specifically around persistent memory. I'll tell you a bit about how DAOS will benefit you as a developer, and give you some resources so that you can learn more. DAOS, or our Distributed Asynchronous Object Storage is a storage engine designed from the ground up to leverage a combination of persistent storage class memory, and traditional NVME SSDs. 

Persistent memory, also known as storage class memory, is a new hardware technology that provides byte addressable storage like DRAM, but which retains data when powered off. This is important because it allows us to build new storage systems with much higher performance than is possible with traditional disks. One of the problems with adding persistent memory into current storage systems is that conventional software defined storage was designed around block addressable media like SSDs and hard drives. These types of devices have relatively high latency. So a few extra microseconds for the software stack to process the I/O is not noticeable. 

With storage class memory however, this extra software overhead prevents applications from taking full advantage of the performance benefits. DAOS was created from the ground up to address this problem, and runs completely in userspace to minimize software overhead while using standardized PMDK and SPDK libraries to write to the underlying devices. This allows us to maximize the IOPS and bandwidth that applications can get from the storage class memory hardware. This means that if you want to implement an S3 interface, NFS, or provide virtual block storage on top of the DAOS engine you can do that, and as a developer, you don't have to worry about how your data is being written or persisted underneath. 

Within the storage engine DAOS is automatically taking care of the features, such as transactional non-blocking I/O, advanced data protection with self-healing on commodity hardware, end-to-end data integrity checks, fine grained data control, and elastic storage provisioning to optimize performance and cost. DAOS is an open source project under the Apache 2.0 license and it's freely and openly available from our GitHub project. I encourage you to contribute. Intel is looking to create a vibrant community around DAOS, so please download and try out the product as a user as well. Follow the links provided to learn more, and don't forget to like this video, and subscribe to the Intel Software YouTube channel. Thanks for watching.

1

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserverd for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804