Using Multiple Backends in Openstack Swift

OpenStack Swift is a highly available, distributed, eventually consistent object/blob store. Object Storage is ideal for cost effective, scale-out storage. It provides a fully distributed, API-accessible storage platform that can be integrated directly into applications or used for backup, archiving, and data retention. For more information please refer http://docs.openstack.org/developer/swift/.

Since V2.0 Swift supports multiple storage policies, which allows for some level of segmenting the cluster for various purposes through the creation of multiple object rings. There is a separate ring for account databases, container databases, and there is also one object ring per storage policy. By supporting multiple object rings, Swift allows the application and/or deployer to essentially segregate the object storage within a single cluster. However Swift has another great feature that supports pluggable backends since Juno release. Thanks to the highly abstracted DiskFile API in object server, storage venders are able to use different backend storage to store objects easily. There are several common things among these projects:

  • These projects are implemented as some new WSGI application of object server. Swift DiskFile abstraction is the engine of these multiple backend solutions.

  • These projects are trying to leverage Swift/S3 APIs to be able to join the object storage market or the OpenStack ecosystem.

  • Currently these projects are mostly in POC state and not very active in development.

Local disk backend

By default Swift will use local disks as the storage devices in object servers. In this implementation, a user uploaded file will be stored individually in the local filesystem on top of the disks. The metadata will be stored as a filesystem attribute along with the file. This requires a filesystem that supports Extended File Attributes, like xfs or ext4.

 

The DiskFile API in object server was a set of RESTFul interfaces, like READ, WRITE and DELETE. In this local disk backend, these interfaces were mostly implemented with the POSIX APIs. E.g., a WRITE request will call os.write() in python.

To use this backend, you only need to copy the sample object-server.conf. Note that the default WSGI Application should be:

[app:object-server]

use = egg: swift#object'

The other backend solutions need to implement these interfaces with their own interfaces.

In-memory backend

This is a sample implementation exists in Swift distribution. In this implementation, a user uploaded file will be stored in an in-memory hash table (python dict), along with its metadata. Each key is the combination of account, container and the object name. The corresponding value is the contents of the object and its metadata.

filesystem[name] = {data, metadata}

A PUT request in DiskFile would be a simple python dict update. This solution is more a prototype currently, which is not suitable for the production environment. As we can see all of the data will be lost if the object servers are shutdown.

To use this backend, you need to change the default WSGI Application in object-server.conf to:

[app:object-server]

use = egg: swift#mem_object'

And then restart the object servers.

Swift-Ceph backend

Currently this is a stackforge project initiated by eNovance. This implementation uses Ceph as the storage devices for Swift. Swift objects rings are configured as 1x copy only while Ceph can be configured as 3x copies. This means from the view of Swift, only 1 copy of object is stored in cluster. However in Ceph cluster, there will be 3 copies of the object and Ceph will do the consistency/replication work. The general design is a new derived class from DiskFile which translates Swift read/write into rados objects read/write using librados. An object in Swift will be stored as a file in Ceph, with its name as the combination of account, container and object name. The account/container DBs are stored in Swift as origin for now. The project also has a plan to store these SQLite DBs to Ceph also later.

 

 

This solution is implemented as a WSGI application. To use this backend, you need to install the swift-ceph-backend project and change the default WSGI Application in object-server.conf to

[app:object-server]

use = egg: swift_ceph_backend#rados_object'

And then restart the object servers.

 

Swift-On-File backend

Swift-on-File project is also a stackforge project started by Redhat. Currently it is a Swift Object Server implementation that enables users to access the same data, both as an object and as a file. Data can be stored and retrieved through Swift's REST interface or as files from NAS interfaces including native GlusterFS, NFS and CIFS.

 

To use this backend, you need to install swiftonfile project and then change the default WSGI Application in object-server.conf to

[app:object-server]

use = egg:swiftonfile#object

You also need to mount one NFS partition/GlusterFS volume mounted as /mnt/swiftonfile

 

Then object-ring is recommended to be configured as 1 copy only. All the consistency/replication work are handled in GlusterFS/NFS layer.

 

Seagate kinetics backend

Swift over Seagate is a project started by SwiftStack and Seagate. Currently it’s still under experimenting with the beta Kinetic library. Swift clusters using Kinetic drives allow access to any drives and, thus, any object. For the current Kinetic integration, a fraction of Object Server commands (Object Daemon) are embedded within the Proxy Server acting as a logical construct, shown below.

There’re some other deployment with kinetic devices. As this project is still under development, there aren’t much documents ready. You need to check the latest code for the details.

 

 

References:

http://docs.openstack.org/developer/swift/

https://swiftstack.com/blog/2014/02/04/swift-extensibility/

https://github.com/stackforge/swift-ceph-backend

https://github.com/stackforge/swiftonfile

https://github.com/swiftstack/kinetic-swift

https://developers.seagate.com/display/KV/OpenStack+Swift

 

 

For more complete information about compiler optimizations, see our Optimization Notice.