Accessing Image and Audio Data

Top  Previous  Next

For shared access or interoperability with other libraries, the SDK uses the PXC[M]Image interface to abstract the image storage, and the PXC[M]Audio interface to abstract the audio storage.

Access Image Data

To access the image buffers, use the image AcquireAccess function to lock access to the image storage and retrieve the storage details in the ImageData structure. The data planes are pointers by data.planes[0-3] and the pitches are stored in data.pitches[0-3]. When the access is completed, use the ReleaseAccess function to release the lock. Example 29 shows how to read image buffers.

The SDK supports different pixel formats. See the PixelFormat enumerator for details about the pixel formats and their memory layouts.

The SDK provides pixel format conversion. The application may request a different pixel format (from the internal pixel format) in the image AcquireAccess function.

For C#, the data planes planes[0-3] in the ImageData structure are pointers in the unmanaged memory. See Member Functions for a set of helper functions to access the image data.

C++ Example 29: Read Image Buffers

// image is a PXCImage instance

PXCImage::ImageData data;

image->AcquireAccess(PXCImage::ACCESS_READ,&data);

... // image planes are in data.planes[0-3] with pitch data.pitches[0-3]

image->ReleaseAccess(&data);

C# Example 29: Read Image Buffers

// image is a PXCMImage instance

PXCMImage.ImageData data;

image.AcquireAccess(PXCMImage.Access.ACCESS_READ,out data);

... // image planes are in data.buffer.planes[0-3] with pitch data.buffer.pitches[0-3]

image.ReleaseAccess(data);

Java Example 29: Read Image Buffers

// image is a PXCMImage instance

PXCMImage.ImageData data;

image.AcquireAccess(PXCMImage.Access.ACCESS_READ,data);

... // image planes are in data.buffer.planes[0-3] with pitch data.buffer.pitches[0-3]

image.ReleaseAccess(data);

Create an Image Instance

To create a PXC[M]Image instance from an image buffer, use the CreateImage function from the PXC[M]Session interface, as illustrated in Example 30 (C++) and Example 31 (C++). In Example 30, the application creates a PXCImage instance that internally points to the imported image buffer. The application must maintain the life cycle of the imported image buffer to be longer than that of the PXCImage instance. In Example 31, the application makes a copy of the imported image.

C++ Example 30: Create an Image Instance Without Data Copy

// Image info

PXCImage::ImageInfo info={};

info.format=PXCImage::PIXEL_FORMAT_RGB32;

info.width=image_width;

info.height=image_height;

 

// Image data

PXCImage::ImageData data={};

data.format=PXCImage::PIXEL_FORMAT_RGB32;

data.planes[0]=image_buffer;

data.pitches[0]=ALIGN64(info.width*4);

 

// Create the image instance

PXCImage *image=session->CreateImage(&info,0,&data);

 

...

 

image->Release();

C++ Example 31: Create an Image Instance With Data Copy

// Image info

PXCImage::ImageInfo info={};

info.format=PXCImage::PIXEL_FORMAT_RGB32;

info.width=image_width;

info.height=image_height;

 

// Create the image instance

PXCImage image=session->CreateImage(&info);

 

// Write data

PXCImage::ImageData data;

image->AcquireAccess(PXCImage::ACCESS_WRITE,&data);

... // copy the imported image to data.planes[0]

image->ReleaseAccess(&data);

For C# and Java, there is always a copy of the data as the imported image is in the managed memory while the image storage is in the unmanaged memory. Example 32 shows how to create a PXCMImage instance from a bitmap.

C# Example 32: Create an Image Instance from Bitmap

// Read bitmap into the memory

Bitmap bitmap = (Bitmap)Image.FromFile(file);

 

// Image info

PXCMImage.ImageInfo iinfo = new PXCMImage.ImageInfo();

iinfo.width  = bitmap.Width;

iinfo.height = bitmap.Height;

iinfo.format = PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32;

 

/* Create the image */

PXCMImage image=session.CreateImage(iinfo);

 

/* Copy the data */

PXCMImage.ImageData idata;

image.AcquireAccess(PXCMImage.Access.ACCESS_WRITE, out idata);

 

BitmapData bdata = new BitmapData();

bdata.Scan0 = idata.buffer.planes[0];

bdata.Stride = idata.buffer.pitches[0];

bdata.PixelFormat = PixelFormat.Format32bppRgb;

bdata.Width = bitmap.Width;

bdata.Height = bitmap.Height;

BitmapData bdata2 = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height),

          ImageLockMode.ReadOnly | ImageLockMode.UserInputBuffer,

          PixelFormat.Format32bppRgb, bdata);

image.ReleaseAccess(idata);

bitmap.UnlockBits(bdata2);

 

... // do something with the image

image.Dispose();

Access Audio Data

To access the audio buffer, use the audio AcquireAccess function to lock access to the audio storage and retrieve the storage details in the AudioData structure. The audio data is pointed by data.dataPtr with the number of samples in data.dataSize. After the access is completed, use the ReleaseAccess function to release the lock. Example 33 shows how to read the audio buffer.

The SDK supports audio formats as listed in the AudioFormat enumerator. See the AudioFormat enumerator description for details.

The SDK provides audio sample format conversion. Thus the application may request a different audio sample format (from the internal audio format) in the AcquireAccess function.

For C# and Java, the data buffer data.dataPtr is in the unmanaged memory. See Member Functions for a set of helper functions to access the audio data.

C++ Example 33: Read the Audio Buffer

// audio is a PXCAudio instance

PXCAudio::AudioData data;

audio->AcquireAccess(PXCAudio::ACCESS_READ,&data);

... // audio buffer is in data.dataPtr with number of samples in data.dataSize

audio->ReleaseAccess(&data);

C# Example 33: Read the Audio Buffer

// audio is a PXCMAudio instance

PXCMAudio.AudioData data;

audio.AcquireAccess(PXCMAudio.Access.ACCESS_READ,out data);

... // audio buffer is in data.dataPtr with number of samples in data.dataSize

audio.ReleaseAccess(data);

Java Example 33: Read the Audio Buffer

// audio is a PXCMAudio instance

PXCMAudio.AudioData data;

audio.AcquireAccess(PXCMAudio.Access.ACCESS_READ,data);

... // audio buffer is in data.dataPtr with number of samples in data.dataSize

audio.ReleaseAccess(data);

Audio Instance Creation

To create a PXC[M]Audio instance from an audio buffer, use the CreateAudio function from the PXC[M]Session interface, as illustrated in Example 34.

For C# and Java, as the data buffer data.dataPtr is in the unmanaged memory, use the Member Functions for a set of helper functions to copy the audio data.

C++ Example 34: Create an Audio Instance

// Audio info

PXCAudio::AudioInfo info={};

info.bufferSize=8200;

info.format=PXCAudio::AUDIO_FORMAT_PCM;

info.sampleRate=44100;

info.nchannels=2;

 

// Create the audio instance

PXCAudio *audio=session->CreateAudio(&info);

 

// Fill the data

PXCAudio::AudioData data;

audio->AcquireAccess(PXCAudio::ACCESS_WRITE,&data);

... // Copy the imported audio buffer to data.dataPtr

... // Update data.dataSize

audio->ReleaseAccess(&data);

 

...

 

// Clean up

audio->Release();

C# Example 34: Create an Audio Instance

// Audio info

PXCMAudio.AudioInfo info=new PXCMAudio.AudioInfo();

info.bufferSize=8200;

info.format=PXCMAudio.AudioFormat.AUDIO_FORMAT_PCM;

info.sampleRate=44100;

info.nchannels=2;

 

// Create the audio instance

PXCMAudio audio=session.CreateAudio(info);

 

// Fill the data

PXCMAudio.AudioData data;

audio.AcquireAccess(PXCMAudio.Access.ACCESS_WRITE,out data);

... // Marshal.Copy the imported audio buffer to data.dataPtr

... // Update data.dataSize

audio.ReleaseAccess(data);

 

...

 

// Clean up

audio.Dispose();

Java Example 34: Create an Audio Instance

// Audio info

PXCMAudio.AudioInfo info=new PXCMAudio.AudioInfo();

info.bufferSize=8200;

info.format=PXCMAudio.AudioFormat.AUDIO_FORMAT_PCM;

info.sampleRate=44100;

info.nchannels=2;

 

// Create an audio instance

PXCMAudio audio=session.CreateAudio(info);

 

// Fill the data

PXCMAudio.AudioData data=new PXCMAudio.AudioData();

audio.AcquireAccess(PXCMAudio.Access.ACCESS_WRITE,data);

... // copy the imported audio buffer to data.dataPtr

... // Update data.dataSize

audio.ReleaseAccess(data);

 

...

 

// Clean up

audio.close();