A weird problem of the Compare function in Face Recognition

A weird problem of the Compare function in Face Recognition

imagem de JIANCHAO L. (Intel)

I have performed an experiment on face recognition: using static images to aquire face models and serialize them in a file, during recognition, I compare the current model extracted from one image to compare with the deserialized models array, and their will be a scores array returned, the problem is, when I add or remove some image to the repository, the scores will be different for each image, namely, there's inconsistency even bewteen two same image's scores with the same target image.

I don't think it's resonable to evaluate each image according to other images. When other images change, the score of similarity with the target image should not change.  The following is how I use "Compare" function:

bool FaceRecognition(vector<PXCFaceAnalysis::Recognition::Model *> refModelsArray, PXCImage* image, pxcF32 * scores, pxcU32 *index){
 PXCFaceAnalysis::Recognition::Model *cur = createModelFromImage(image);
 pxcU32 sizeRefArray = refModelsArray.size();
 PXCFaceAnalysis::Recognition::Model ** refModels = new PXCFaceAnalysis::Recognition::Model*[sizeRefArray];
 for(int i = 0; i < sizeRefArray; ++i)
 refModels[i] = refModelsArray[i];
 if(cur->Compare(refModels, sizeRefArray, scores, index) == PXC_STATUS_NO_ERROR)return true;
 else return false;
}

11 posts / 0 new
Último post
Para obter mais informações sobre otimizações de compiladores, consulte Aviso sobre otimizações.
imagem de N. Sadaka (Intel)

whenever you change any of the two images (even belonging to the same person) the output of the compare scores changes.

I do not clearly understand your problem. This is what I have verified:

1- compared modelA with a set of 6 models and the return scores = arrayA.. I  repeated this proeccess many times and the scores are always the same.

2- Added more models to the set and compared modelA with the new set of 7 models and the return scores = arrayB.. arrayA always match arrayB for the common models.

please can you restate your issue if the previous explanation did not help.

Thanks.

imagem de JIANCHAO L. (Intel)

Quote:

N. Sadaka (Intel) wrote:

whenever you change any of the two images (even belonging to the same person) the output of the compare scores changes.

I do not clearly understand your problem. This is what I have verified:

1- compared modelA with a set of 6 models and the return scores = arrayA.. I repeated this proeccess many times and the scores are always the same.

2- Added more models to the set and compared modelA with the new set of 7 models and the return scores = arrayB.. arrayA always match arrayB for the common models.

please can you restate your issue if the previous explanation did not help.

Thanks.

How do you get the models set, do you use serialize/deserialize? Here is the whole workflow:

1. I have a set of face images, I use the following function to load each image and convert it to PXCImage:

void LoadImageFromLocal(PXCSession* session, PXCImage **dst_img, const char * path)
{
        IplImage *image = cvLoadImage(path); 
        unsigned char *rgb_data; 
         int rgb_pitch = image->widthStep; // bytes between image lines 
         rgb_data = (unsigned char*)image->imageData; 
         PXCAccelerator * accelerator; 
         session->CreateAccelerator(PXCAccelerator::ACCEL_TYPE_CPU, &accelerator); 
         PXCImage::ImageInfo info; 
         memset(&info, 0, sizeof(info)); 
         info.height = image->height; 
         info.width = image->width; 
         info.format = PXCImage::COLOR_FORMAT_RGB24; 
         PXCImage::ImageData data; 
         memset(&data, 0, sizeof(data)); 
         data.format = PXCImage::COLOR_FORMAT_RGB24; 
         data.planes[0] = rgb_data; 
         data.pitches[0] = rgb_pitch; 
         pxcStatus sts = accelerator->CreateImage(&info, 0, &data, dst_img); 
} 

2. I serialize the PXCImage array to a single file using the function below:

 
void SerializeModelsFromImages(const char *refPathName, PXCImage *images[], int num){
         ofstream outputfile(refPathName, ofstream::binary);
        for(int i = 0; i < num; ++i){
                 pxcBYTE *buffer;
                 PXCFaceAnalysis::Recognition::Model * model = createModelFromImage(images[i]);
                 buffer = new pxcBYTE[buffSize];
                if (model != NULL)
                 { 
                         model->Serialize(buffer); 
                 }
                outputfile.write((const char*)buffer, buffSize); 
                 outputfile.flush();
                 delete buffer;
         } 
         outputfile.close();
         putfile.close(); 
}

3. Desialize the file to get the models set before recognition using the function below:

 
vector<PXCFaceAnalysis::Recognition::Model *> DeserializeReferenceModels(const char* refPathName, int &sizeRefArray){
         vector<PXCFaceAnalysis::Recognition::Model*> refModels; 
         ifstream inputfile(refPathName, ifstream::binary);
         long size;
         char *buffer = new char[buffSize];
         inputfile.seekg(0, ifstream::end);
         size = inputfile.tellg();
         sizeRefArray = size / buffSize;
         inputfile.seekg(0, ifstream::beg);
         for(int i = 0; i < sizeRefArray; ++i){
                 inputfile.read(buffer, buffSize);
                 PXCFaceAnalysis::Recognition::Model *ref = 0;
                 faceAnalyzer->DynamicCast<PXCFaceAnalysis::Recognition>()->DeserializeModel((pxcBYTE*)buffer, &ref);
                 refModels.push_back(ref);
         } 
         return refModels;
}

I load all the face images in dictionary order of the image names, when I change one of the image's name to change the loading order, add or remove one of the image, the scores will change. Is there any bug in the serialize/deserialize or JPG to PXCImage convert function ? I will upload all the source code here, you can verify this, thanks very much! This bizarre problem has drive me crazy.

ps: you should install OpenCV if you want to run the code. make sure you refer to the right path for each .h file

Anexos: 

AnexoTamanho
Download source-code-and-images.zip109.55 KB
imagem de N. Sadaka (Intel)

one mistake you need to enable face detection mode and not tracking mode. Since the inputs are images and not video sequences. Also make sure to check for images whose recognition model could not be generated.

see below for setting the detection mode:
PXCFaceAnalysis::ProfileInfo faInfo;
faceAnalyzer->QueryProfile(0, &faInfo);
faInfo.iftracking = 0;
faceAnalyzer->SetProfile(&faInfo);

I have fixed the attached code you sent and re-attached a fixed project that works. did some experiments and all the scores for each pair matches with renaming and shuffling around. I also added some comments in the doc file attached (scroll to the end of the document to see my comments)

I have emailed you more details.

Nabil.

Anexos: 

AnexoTamanho
Download recognition-test-sample.zip441.76 KB
imagem de PONRAM

good solution Sadaka...!

imagem de Ural (Intel)

did anybody tried the same thing in C#? I am trying to read image data using Emgu.cv, since I didnt find any other way to find get pitch data from bitmap in c#. but i keep getting PXCM_STATUS_PARAM_UNSUPPORTED status while trying to create accelator.CreateImage.

        private PXCMImage loadImage(PXCMSession session, string fileName)
        {
            PXCMImage image;
            PXCMAccelerator accelerator;
            session.CreateAccelerator(PXCMAccelerator.AccelType.ACCEL_TYPE_CPU, out accelerator);
            //load the image as Bgr color
            IntPtr ptr = CvInvoke.cvLoadImage(fileName, LOAD_IMAGE_TYPE.CV_LOAD_IMAGE_COLOR);
            if (ptr == IntPtr.Zero)
                throw new NullReferenceException(String.Format("Unable to load image from file \"{0}\".", fileName));
            MIplImage mptr = (MIplImage)Marshal.PtrToStructure(ptr, typeof(MIplImage));
            
            PXCMImage.ImageInfo imageInfo = new PXCMImage.ImageInfo();
            imageInfo.height = (uint)mptr.height;
            imageInfo.width = (uint)mptr.width;
            imageInfo.format = PXCMImage.ColorFormat.COLOR_FORMAT_RGB24;
            PXCMImage.ImageData imgData = new PXCMImage.ImageData();
            imgData.format = PXCMImage.ColorFormat.COLOR_FORMAT_RGB24;
            unsafe
            {
                imgData.buffer.planes[0] = (void*)mptr.imageData;
            };
            imgData.buffer.pitches[0] = mptr.widthStep;            
 
            pxcmStatus sts = accelerator.CreateImage(ref imageInfo,0,ref imgData, out image);
            return image;
        }

Ural
imagem de Shyam

Thanks Ural for information, I was trying to face recognization using C#. I wrote couple of functions for it.

I have couple of quries, how to convert image or bitmap image to PXCMImage. May be above can be used to convert it but I never have used or heard about it  Emgu.cv.

I am trying to get modelSize by Querying Profile I am getting size is 0.

Can you help with above.

Thanks

imagem de Shyam

Hey Ural, Your function above to convert image to PxcmImage object is throwing error from this line

//load the image as Bgr color
IntPtr ptr = CvInvoke.cvLoadImage(fileName, LOAD_IMAGE_TYPE.CV_LOAD_IMAGE_COLOR);

Error : TypeInitializationException The type initializer for 'Emgu.CV.CvInvoke' threw an exception

Have anyone tried converting image to PxcmImage object. There is no documentation at all on this.

-Shyam

imagem de Xintian Wu (Intel)

See example how to convert image to/from PXCMImage.

 

Anexos: 

AnexoTamanho
Download import.image.zip16.31 KB
imagem de Shyam

Thanks I example is working

imagem de Pruthviraj N.

Hi Xintian Wu,
i am trying to find the function to obtain image rgb and pitch but, after i downloaded the example you have attached i couldn't able to find the function! Can you please let me know the file. Thanks 

Faça login para deixar um comentário.