I'm trying to match depth image onto the color image using PXCImage::ImageData.planes component of the depth data that contains coordinates of a depth point in a color image. While it seems to do the job, I notice a significant misalignment. I attached a rather crude picture where I blacked out all pixels which have a depth bigger than a threshold.
The depth map does not exactly match the color image, actually the shift to the same side where the depth sensor is located on the camera with relation to the color sensor. Is this a deficiency of SDK?