OpticalFlowPyrLK

Intel® Integrated Performance Primitives (Intel® IPP) Developer Guide and Reference

Download PDF

ID 790148

Date 3/22/2024

Version

Public

Visible to Intel only — GUID: GUID-5FFE93E3-CACD-43AD-B9B1-9AE90FF53183

View Details

OpticalFlowPyrLK

Calculates optical flow for the set of feature points using the pyramidal Lucas-Kanade algorithm.

Syntax

IppStatus ippiOpticalFlowPyrLK_<mod>(IppiPyramid* pPyr1, IppiPyramid* pPyr2, const IppiPoint_32f* pPrev, IppiPoint_32f* pNext, Ipp8s* pStatus, Ipp32f* pError, int numFeat, int winSize, int maxLev, int maxIter, Ipp32f threshold, IppiOptFlowPyrLK_<mod>* pState);

Supported values for mod:

8u_C1R

16u_C1R

32f_C1R

Include Files

ippcv.h

Domain Dependencies

Headers: ippcore.h, ippvm.h, ipps.h, ippi.h

Libraries: ippcore.lib, ippvm.lib, ipps.lib, ippi.lib

Parameters

pPyr1: Pointer to the ROI in the first image pyramid structure.
pPyr2: Pointer to the ROI in the second image pyramid structure.
pPrev: Pointer to the array of initial coordinates of the feature points.
pNext: Pointer to the array of new coordinates of feature point; as input it contains hints for new coordinates.
pStatus: Pointer to the array of result indicators.
pError: Pointer to the array of differences between neighborhoods of old and new point positions.
numFeat: Number of feature points.
winSize: Size of the search window of each pyramid level.
maxLev: Pyramid level to start the operation.
maxIter: Maximum number of algorithm iterations for each pyramid level.
threshold: Threshold value.
pState: Pointer to the pyramidal optical flow structure.

Description

This function operates with ROI (see Regions of Interest in Intel IPP).

This function implements the iterative version of the Lucas-Kanade algorithms [Bou99]. It computes with sub-pixel accuracy new coordinates of the numFeat feature points of two images at time t and t+dt. Their initial coordinates are places in the pPrev array. Computed values of new coordinates of the feature points are stored in the array pNext, that initially contains estimations of them (or hints), for example, based on the flow values for the previous image in sequence. If there are not such hints, the pNext array contains the same initial coordinates as the pPrev array.

pStatus and pError are arrays of size numFeat with the respective data type.

The images are presented by the pyramid structures pPyr1 and pPyr2 respectively (see description of the PyramidGetSize and ippiPyramidInit functions for more details). These structures should be initialized by calling the function PyramidGetSize and ippiPyramidInit functions beforehand. The function uses the pyramidal optical flow structure pState that also should be previously initialized using OpticalFlowPyrLKGetSize and OpticalFlowPyrLKInit.

The function starts operation on the highest pyramid level (smallest image) that is specified by the maxLev parameter in the centered search window which size winSize could not exceed the corresponding value winSize that is specified in OpticalFlowPyrLKGetSize and OpticalFlowPyrLKInit. The operation for i-th feature point on the given pyramid level finishes if:

New position of the point is found:
Specified number of iteration maxIter is performed
Intersection between the pyramid layer and the search window became empty

In first two cases for non-zero levels the new position coordinates are scaled to the next pyramid level and the operation continues on the next level. For zero level or for third case the operation stops, the number of the corresponding level is written to the pStatus[i] element, the new coordinates are scaled to zero level and are written to pNext[i]. The square root of the average squared difference between neighborhoods of old and new positions is written to pError[i].

Return Values

ippStsNoErr	Indicates no error. Any other value indicates an error or a warning.
ippStsNullPtrErr	Indicates an error if one of the specified pointer is NULL.
ippStsSizeErr	Indicates an error condition if numFeat or winSize has zero or negative value.
ippStsBadArgErr	Indicates an error condition if maxLev or threshold has negative value, or maxIter has zero or negative value.

Example

/*******************************************************************************
* Copyright 2015 Intel Corporation.
*
*
* This software and the related documents are Intel copyrighted materials, and your use of them is governed by
* the express license under which they were provided to you ('License'). Unless the License provides otherwise,
* you may not use, modify, copy, publish, distribute, disclose or transmit this software or the related
* documents without Intel's prior written permission.
* This software and the related documents are provided as is, with no express or implied warranties, other than
* those that are expressly stated in the License.
*******************************************************************************/

// A simple example of performing Optical Flow algorithm 
// using Intel(R) Integrated Performance Primitives (Intel(R) IPP) functions:
//     ippiPyramidGetSize
//     ippiPyramidInit
//     ippiPyramidLayerDownGetSize_8u_C1R
//     ippiPyramidLayerDownInit_8u_C1R
//     ippiPyramidLayerDown_8u_C1R
//     ippiOpticalFlowPyrLKGetSize
//     ippiOpticalFlowPyrLKInit_8u_C1R


#include <stdio.h>
#include "ipp.h"

#define WIDTH  640  /* frames width  */
#define HEIGHT 480  /* frames height */

/* Next two defines are created to simplify code reading and understanding */
#define EXIT_MAIN exitLine:                                  /* Label for Exit */
#define check_sts(st) if((st) != ippStsNoErr) goto exitLine; /* Go to Exit if Intel(R) IPP function returned status different from ippStsNoErr */

/* Results of ippMalloc() are not validated because Intel(R) IPP functions perform bad arguments check and will return an appropriate status  */
void drawRect_8u(Ipp8u* pSrc, int srcStep, int x, int y, int w, int h)
{
    int i, j;
    for (j = y; j < y+h; j++) {
        for (i = x; i < x+w; i++) {
            pSrc[srcStep*j+i] = 255;
        }
    }
}

int main(void)
{
    IppStatus status = ippStsNoErr;
    Ipp8u* pPrevFrame = NULL, *pNextFrame = NULL;  /* Pointers to previous/next frames */
    int prevStep = 0, nextStep = 0;         /* Steps, in bytes, through the previous/next frames */
    IppiSize roiSize = { WIDTH, HEIGHT };   /* Size of previous/next ROI in pixels */
    int            numLevel = 5;            /* Pyramid level number */
    float          rate = 2.0f;             /* Pyramid rate  */
    Ipp16s pKernel[5] = {1,1,1,1,1};        /* Pyramid kernels */
    int            kerSize = 5;             /* Pyramid kernel size  */
    IppiPoint_32f *prevPt = NULL;           /* Coordinates on previous frame */
    IppiPoint_32f *nextPt = NULL;           /* Hint to coordinates on next frame */
    Ipp8s         *pStatus= NULL;           /* Array of differences between pPrev and pNext points */
    Ipp32f        *pError = NULL;           /* Array of result indicator */
    int            numFeat = 5;           /* Point number */
    int            winSize = 41;            /* Search window size */
    int            numIter = 5;             /* Iteration number */
    float          threshold = 0.0001f;

    Ipp8u* pBuf1 = NULL, *pPBuf2 = NULL, *pPBuf1 = NULL;   /* Pointer to the work buffers */

    Ipp8u* pBuf2    = NULL;
    Ipp8u* pStBuf1  = NULL, *pStBuf2 = NULL; /* Pointers to the buffer to initialize state structure for pyramids */
    Ipp8u* pOFStBuf = NULL;

    pPrevFrame = ippiMalloc_8u_C1(roiSize.width, roiSize.height, &prevStep);
    pNextFrame = ippiMalloc_8u_C1(roiSize.width, roiSize.height, &nextStep);
    prevPt = (IppiPoint_32f*)ippsMalloc_8u(numFeat * 2 * sizeof(IppiPoint_32f));
    nextPt = (IppiPoint_32f*)ippsMalloc_8u(numFeat * 2 * sizeof(IppiPoint_32f));
    pStatus = (Ipp8s*) ippsMalloc_8u(numFeat);
    pError = (Ipp32f*)ippsMalloc_8u(numFeat * sizeof(Ipp32f));

    check_sts(status = ippiSet_8u_C1R(0, pPrevFrame, prevStep, roiSize));
    check_sts(status = ippiSet_8u_C1R(0, pNextFrame, nextStep, roiSize));

    int x = 3;
    int y = 3;
    int w = 5;
    int h = 5;
    int shiftx = 3;
    int shifty = 4;

    /*imitate shift of rectangle*/
    drawRect_8u(pPrevFrame, prevStep, x, y, w, h);
    drawRect_8u(pNextFrame, nextStep, x+shiftx, y+shifty, w, h);

    check_sts( status = ippsSet_8u(0, (Ipp8u*)prevPt, numFeat * 2 * sizeof(IppiPoint_32f)) );
    check_sts( status = ippsSet_8u(0, (Ipp8u*)nextPt, numFeat * 2 * sizeof(IppiPoint_32f)) );
    
    prevPt[0].x = 0  ; prevPt[0].y = 0;
    prevPt[1].x = x  ; prevPt[1].y = y;
    prevPt[2].x = x+w; prevPt[2].y = y;
    prevPt[3].x = x  ; prevPt[3].y = y+h;
    prevPt[4].x = x+w; prevPt[4].y = y+h;

    {
        IppiPyramid *pPyr1 = NULL, *pPyr2 = NULL;   /* Pointer to the pointer to the pyramid structures */
        IppiOptFlowPyrLK *pOF = NULL;               /* Optical flow structure */
        int   PyrSize, BufSize, StateSize;          /* Size of pyramid structure, pyramid external work buffer and state structure */

        /* Compute the size of the structure for pyramids and the size of the required work buffer (in bytes) */
        check_sts( status = ippiPyramidGetSize(&PyrSize, &BufSize, numLevel, roiSize, rate) )

        pBuf1  = ippsMalloc_8u(BufSize);
        pPBuf1 = ippsMalloc_8u(PyrSize);

        /* Initialize structure for pyramids, calculates ROI for layers */
        check_sts( status = ippiPyramidInit(&pPyr1, numLevel, roiSize, rate, pPBuf1, pBuf1) )
        ippsFree(pBuf1); pBuf1 = NULL;

        /* Compute the size of the structure for pyramids and the size of the required work buffer (in bytes) */
        check_sts( status = ippiPyramidGetSize(&PyrSize, &BufSize, numLevel, roiSize, rate) )

        pBuf1  = ippsMalloc_8u(BufSize);
        pPBuf2 = ippsMalloc_8u(PyrSize);
        /* Initialize structure for pyramids, calculates ROI for layers */
        check_sts( status = ippiPyramidInit(&pPyr2, numLevel, roiSize, rate, pPBuf2, pBuf1) )
        ippsFree(pBuf1); pBuf1 = NULL;

        {
            IppiPyramidDownState_8u_C1R **pState1 = (IppiPyramidDownState_8u_C1R**)&(pPyr1->pState);/* Pyramid state structure */
            IppiPyramidDownState_8u_C1R **pState2 = (IppiPyramidDownState_8u_C1R**)&(pPyr2->pState);/* Pyramid state structure */
            Ipp8u   **pImg1 = pPyr1->pImage; /* Pointer to the source image */
            Ipp8u   **pImg2 = pPyr2->pImage; /* Pointer to the source image */
            int     *pStep1 = pPyr1->pStep, *pStep2 = pPyr2->pStep;
            IppiSize *pRoi1 = pPyr1->pRoi, *pRoi2 = pPyr2->pRoi;
            IppHintAlgorithm hint = ippAlgHintFast;
            int i, level = pPyr1->level;
            int pStateSize1, pBufSize1, pStateSize2, pBufSize2;

            /* Calculate the size of structure for creating a lower(an upper) pyramid layer and the size of the temporary buffer */
            check_sts( status = ippiPyramidLayerDownGetSize_8u_C1R(roiSize, rate, kerSize, &pStateSize1, &pBufSize1) )

            pBuf1 = ippsMalloc_8u(pBufSize1);
            pStBuf1 = ippsMalloc_8u(pStateSize1);

            /* Initialize the structure for creating a lower(an upper) pyramid layer */
            check_sts( status = ippiPyramidLayerDownInit_8u_C1R((IppiPyramidDownState_8u_C1R**)pState1, roiSize, rate, pKernel, kerSize, IPPI_INTER_LINEAR, pStBuf1, pBuf1) )
            ippsFree(pBuf1); pBuf1 = NULL;

            /* Calculate the size of structure for creating a lower(an upper) pyramid layer and the size of the temporary  buffer */
            check_sts( status = ippiPyramidLayerDownGetSize_8u_C1R(roiSize, rate, kerSize, &pStateSize2, &pBufSize2) )

            pBuf2 = ippsMalloc_8u(pBufSize2);
            pStBuf2 = ippsMalloc_8u(pStateSize2);

            /* Initialize the structure for creating a lower(an upper) pyramid layer */
            check_sts( status = ippiPyramidLayerDownInit_8u_C1R((IppiPyramidDownState_8u_C1R**)pState2, roiSize, rate, pKernel, kerSize, IPPI_INTER_LINEAR, pStBuf2, pBuf2) )
            ippsFree(pBuf2); pBuf2 = NULL;

            pImg1[0] = (Ipp8u*)pPrevFrame;
            pImg2[0] = (Ipp8u*)pNextFrame;
            pStep1[0] = prevStep;
            pStep2[0] = nextStep;
            pRoi1[0] = pRoi2[0] = roiSize;
            for (i = 1; i <= level; i++)
            {
                pPyr1->pImage[i] = ippiMalloc_8u_C1(pRoi1[i].width, pRoi1[i].height,
                    pStep1 + i);
                pPyr2->pImage[i] = ippiMalloc_8u_C1(pRoi2[i].width, pRoi2[i].height,
                    pStep2 + i);
                /* Perform downsampling / upsampling of the image with 5x5 Gaussian kernel*/
                check_sts( status = ippiPyramidLayerDown_8u_C1R(pImg1[i - 1], pStep1[i - 1], pRoi1[i - 1],
                    pImg1[i], pStep1[i], pRoi1[i], *pState1) )
                check_sts( status = ippiPyramidLayerDown_8u_C1R(pImg2[i - 1], pStep2[i - 1], pRoi2[i - 1],
                    pImg2[i], pStep2[i], pRoi2[i], *pState2) )
            }

            check_sts( status = ippiOpticalFlowPyrLKGetSize(winSize, roiSize, ipp8u, (IppHintAlgorithm)((int)hint), &StateSize) )

            pOFStBuf = (Ipp8u*)ippsMalloc_8u(StateSize);

            /* Initialize a structure for pyramidal L-K algorithm */
            check_sts( status = ippiOpticalFlowPyrLKInit_8u_C1R((IppiOptFlowPyrLK_8u_C1R**)&pOF, roiSize, winSize, (IppHintAlgorithm)((int)hint), pOFStBuf) )

            /* Indicate an error condition if maxLev or threshold has negative value, or maxIter has zero or negative value */
            check_sts( status = ippiOpticalFlowPyrLK_8u_C1R(pPyr1, pPyr2, prevPt, nextPt, pStatus, pError,
                numFeat, winSize, numLevel, numIter, threshold, pOF) )

            for (i = level; i > 0; i--)
            {
                if (pImg2[i]) ippiFree(pImg2[i]);
                if (pImg1[i]) ippiFree(pImg1[i]);
            }
        }
    }
    printf("shiftx=%d, ", shiftx);
    printf("shifty=%d\n", shifty);
    int n;
    for (n = 0; n < numFeat; n++) {
        printf("%f, %f -> %f, %f\n", prevPt[n].x, prevPt[n].y, nextPt[n].x, nextPt[n].y);
    }
EXIT_MAIN
    ippsFree(pOFStBuf);
    ippsFree(pBuf1);
    ippsFree(pBuf2);
    ippsFree(pPBuf1);
    ippsFree(pPBuf2);
    ippsFree(pStBuf1);
    ippsFree(pStBuf2);
    ippiFree(pPrevFrame);
    ippiFree(pNextFrame);
    ippsFree(prevPt);
    ippsFree(nextPt);
    ippsFree(pStatus);
    ippsFree(pError);
    printf("Exit status %d (%s)\n", (int)status, ippGetStatusString(status));
    return (int)status;
}

Parent topic: Optical Flow

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Intel® Integrated Performance Primitives (Intel® IPP) Developer Guide and Reference

OpticalFlowPyrLK

Syntax

Include Files

Domain Dependencies

Parameters

Description

Return Values

Example