Introducing Python Support for Persistent Memory

Overview

The video introduces PyNVM, an open source Python library designed to allow Python developers to work with persistent memory easily.

Transcript

Hi, I'm Catalin Manciu from Intel. In this video, I'll talk about PyNVM, an open source Python library that allows you to use the exciting new features introduced by persistent memory products, data retention, large capacities, and byte-level addressability. PyNVM is based on PMDK and provides persistent versions of commonly used Python containers, which retain their content over application restarts and even power cycles. 

The library's written entirely in Python using the CFFI or C Foreign Functions Interface framework. This approach has notable advantages. Its internal code is clear, concise, and easy to understand, since it is Python code. 

There is no reliance on arcane CPython APIs. This is because CFFI takes care of the runtime-specific details. It also ensures compatibility with other implementations of the Python language, such as PyPI. As Python developers, you'll feel right at home exploring PyNVM. 

The library is broken into a higher and a lower level API. The higher level API, called pmemobj, offers support for persistent containers, dictionaries, lists, sets, and tuples and arbitrary objects for Python. The lower level API consists of wrappers over some of PMDK's features, such as efficient logging to persistent memory or atomic updates to persistent memory resident arrays The persistent containers should feel familiar to Python users as their interface and behavior are mostly similar to their non-persistent Python counterparts. 

However, there are some differences. One is in the way they are instantiated. You will need to use an object factory provided by the library to create them. The other difference is, unlike regular Python objects, their lifetime extends beyond that of the runtime process they have been created by. 

Once they are no longer referenced by any other persistent object, they are removed by pmemobj, the garbage collector, and their associated persistent memory space gets released. Every container update operation is transactional, and data will be guaranteed to be stored in a consistent state, even in the face of a software or power failure. You'll find documentation and samples on PyNVM's GitHub page in the links provided. The library is also conveniently available as a PyPI package-- just do a PIP install NVM and you are good to go. 

Thanks for watching. Don't forget to like this video and subscribe to the Intel Software YouTube channel.

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804