Test for possible IppiConvValid bug

Test for possible IppiConvValid bug

 Hi, I'm getting a -9 mem error with ippiConvValid_32f_C1R even though  my system has plenty of memory. I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows.  No problems there. Attached to this email is a compiled Windows x64 console app.  Everyone please download it and reply with the MAX size you can enter before you get a -9 err along with your system specs.  If you want to build it yourself, please go to my other post to download the VS project.  http://redfort-software.intel.com/en-us/forums/showthread.php?t=106382&o=a&s=lr Hopefully there is a solution to this because it's really hampering me! Best, Ryan

AttachmentSize
Downloadapplication/rar mem_test_exe.rar601.53 KB
19 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Quoting Ryan...I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows. No problems there...

Ryan,

Did you try to allocate more memory with a 'malloc' CRT function, for example 64GB and higher? Please take a look at
a test-application I created some time ago and attached to that post:

     http://redfort-software.intel.com/en-us/forums/showpost.php?p=189221

I'll take a look at your test-application.

Best regards,
Sergey

Quoting Sergey KostrovI'll take a look at your test-application...

What about source codes of your test-application? At least simplified version that reproduces the problem
with ippiConvValid_xxx IPP function.

Hi Sergey, Attached is the source code for my test app.  It's very easy to follow. Thanks for looking into this for me! Best, Ryan

Attachments: 

AttachmentSize
Downloadapplication/rar IppiConvValid_mem_test.rar2.04 MB

Hi Sergey, I ran your mem test app with the CM3 line uncommented:  __int64 _MEM_SIZE = ( 1073741824 * 64.0 ); // 64GB Recommended VM =  N/A  / 96GB 64-bit Windows platformAllocating memory with 'malloc' CRT-function - 1DSuccesfully Allocated 64.00GB64-bit Windows platformAllocating memory with 'malloc' CRT-function - 1DSuccesfully Allocated 64.00GB 

Update, I also tried my mem test app with IppiConvValid_16s_C1R, IppiConvValid_16u_C1R, and IppiConvValid_8u_C1R and the resutls are the same. On my machine: (Dual socket Intel Xeon E5645, 48GB DDR3) - 1.06 before I get -9 mem error.  On three other computers I've tried: (Dual socket Xeon E5420, 16GB DDR2) -1.89.(Dual socket Xeon E5420, 16GB DDR2) - 1.89.(Dual socket i7, 12GB DDR2) - 1.89.  Ryan

Quoting RyanAttached is the source code for my test app.  It's very easy to follow...

Hi Ryan,

Could you upload as a zip archive, please? I couldn't extract your sources from the rar archive:

   1   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\mem_test.cpp" header broken
   2   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\mem_test.cpp
   3   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\mem_test.vcproj" header broken
   4   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\mem_test.vcproj
   5   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\stdafx.cpp" header broken
   6   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\stdafx.cpp
   7   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\stdafx.h" header broken
   8   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\stdafx.h
   9   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\targetver.h" header broken
  10   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\targetver.h
  11   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\BuildLog.htm" header broken
  12   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\BuildLog.htm
  13   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.exe.intermediate.manifest" header broken
  14   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.exe.intermediate.manifest
  15   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.obj" header broken
  16   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.obj
  17   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.pch" header broken
  18   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.pch
  19   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mt.dep" header broken
  20   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mt.dep
  21   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\stdafx.obj" header broken
  22   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\stdafx.obj
  23   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\vc90.idb" header broken
  24   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\vc90.idb
  25   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\vc90.pdb" header broken
  26   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\vc90.pdb
  27   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.ncb" header broken
  28   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.ncb
  29   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.sln" header broken
  30   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.sln
  31   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.suo" header broken
  32   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.suo
  33   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\libiomp5md.dll
  34   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release\mem_test.exe" header broken
  35   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\mem_test.exe
  36   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release\mem_test.pdb" header broken
  37   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\mem_test.pdb
  38   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release" header broken
  39   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64" header broken
  40   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release" header broken
  41   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test" header broken
  42   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64" header broken
  43   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test" header broken

Best regards,
Sergey

PS: That is really strange but it looks like older versions of WinRAR are not compatible with the newer versions.

Hi Ryan, This ia a short follow up regarding the error -9...

Quoting Ryan...I'm getting a -9 mem error with ippiConvValid_32f_C1R even though  my system has plenty of memory. I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows.  No problems there. Attached to this email is a compiled Windows x64 console app. Everyone please download it and reply with the MAX size you can enter before you get a -9 err...

This is how it is declared:

     typedef enum
     {
         ...
         ippStsMemAllocErr           = -9,
         ...
     } IppStatus;

and I think the problem could be possibly related to limitation(s) of some IPP functions, including ippiConvValid_xxx. Your 64-bit system
allows to allocate up to 64GB of memory (!) and you've done two tests. These "magic" limitations were
disccused a couple of times on the IPP forum in the past. Unfortunately, there is no article that describes these limitations
in some functions of IPP library.

Best regards,
Sergey

Hi Sergey, That's weird.  Sorry about that. Please see attached for zip.  Best, Ryan

Attachments: 

Hi Sergey, Yes I looked up that enum.  That's exactly my thinking.  I want to know why IppConvValid is having that problem.  For it to run out of memory on my system it would have to be doing a HUGE FFT.  What I find even stranger is that an i7 with 12gigs of DD2 memory can handle more than my Xeon with 48gigs of DD3.  Ryan

OK! I tried using IppConvFull instead of Valid.  I was able to enter 3.2 into my mem test program before I got the -9 error code.  So WHY, does ConvValid crap out at 1.06?

//Allocate repImg
	Image repImg(srcImg.getSize().width + width_pad * 2, srcImg.getSize().height + height_pad * 2);

	//Repeat borders of imgSrc
	ippiCopyReplicateBorder_32f_C1R(srcImg.getPtr(), srcImg.getStepBytes(), srcImg.getSize(), repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(), height_pad, height_pad);

	//Allocate convImg
	Image convImg(repImg.getSize().width + kerImg.getSize().width - 1, repImg.getSize().height + kerImg.getSize().height - 1);

	IppStatus status = ippiConvFull_32f_C1R(repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(), kerImg.getPtr(), kerImg.getStepBytes(), kerImg.getSize(), convImg.getPtr(), convImg.getStepBytes());

Quoting RyanPlease see attached for zip... 
Received and no errors detected. Thank you.

    Testing ...
    testing: IppiConvValid_mem_test/   OK
    testing: IppiConvValid_mem_test/mem_test/   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.cpp   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.vcproj   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.vcproj.AFX-RYAN.afx-admin.user   OK
    testing: IppiConvValid_mem_test/mem_test/stdafx.cpp   OK
    testing: IppiConvValid_mem_test/mem_test/stdafx.h   OK
    testing: IppiConvValid_mem_test/mem_test/targetver.h   OK
    testing: IppiConvValid_mem_test/mem_test/x64/   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/BuildLog.htm   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.exe.intermediate.manifest   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.obj   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.pch   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mt.dep   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/stdafx.obj   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/vc90.idb   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/vc90.pdb   OK
    testing: IppiConvValid_mem_test/mem_test.ncb   OK
    testing: IppiConvValid_mem_test/mem_test.sln   OK
    testing: IppiConvValid_mem_test/mem_test.suo   OK
    testing: IppiConvValid_mem_test/x64/   OK
    testing: IppiConvValid_mem_test/x64/Release/   OK
    testing: IppiConvValid_mem_test/x64/Release/libiomp5md.dll   OK
    testing: IppiConvValid_mem_test/x64/Release/mem_test.exe   OK
    testing: IppiConvValid_mem_test/x64/Release/mem_test.pdb   OK
    No errors detected in compressed data of C:/Documents and ../Desktop/IppiConvValid_mem_test.zip.

Best regards,
Sergey

What are the results of your testing?

Quoting RyanWhat are the results of your testing?

Hi Ryan,

I'm sorry for some delay with my tests. I confirm that some internal problem with 'ippiConvValid_32f_C1R' exists. Please take a look at my results:

>> Test-case with Kernel size = 3 <<
     ...
     Enter '0' to quit
     Kernel size relative to input size 1080p: 3
     Input width: 7680, Input height: 4320
     Kernel width: 5761, Kernel height: 3241
     IppStatus code for ippiConvValid_8u_C1R: 0
     ...

>> Here is a summary for different Kernel sizes <<

     Kernel size relative to input size 1080p: 1
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 2
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 3
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 4
     IppStatus code for ippiConvValid_8u_C1R: -9 ( Error: ippStsMemAllocErr )

     Kernel size relative to input size 1080p: 5
     IppStatus code for ippiConvValid_8u_C1R: N/A because the test application crashed

Best regards,
Sergey

Quoting RyanWhat are the results of your testing?

Please take a look at a screenshot:

'ippiConvValid_32f_C1R' uses lots of memory, especially when kernel size gets bigger, and it crashes
when it tries to allocate more than 1.65GB of memory.

Best regards,
Sergey

Here is your test-case with my small modifications:

#include "stdafx.h"

#include 
#include 

#include "..IPPIncludeippi.h"

void doConvolution( float size );

struct Image
{
private:
	Ipp32f *ptr;
	IppiSize size;
	int stepBytes;

public:
	Image( int width, int height )
	{
		ptr = NULL;
		size.width = width;
		size.height = height;
		stepBytes = -1;

		ptr = ::ippiMalloc_32f_C1( width, height, &stepBytes );
	}

	Ipp32f * getPtr()
	{
		return ptr;
	}

	Ipp32f * getPtr( int x, int y )
	{
		Ipp32f *pixelPtr = ( Ipp32f * )( ( char * )ptr + x + y * size.width );
		return pixelPtr;
	}

	IppiSize getSize()
	{
		return size;
	}

	int getStepBytes()
	{
		return stepBytes;
	}

	~Image()
	{
		if( ptr != NULL )
		{
			::ippiFree( ptr );
			ptr = NULL;
		}
	}
};

int _tmain( int argc, _TCHAR *argv[] )
{
	float size = 0.0f;

	std::cout << "Enter '0' to quit" << std::endl;

	while( true )
	{
		std::cout << "Kernel size relative to input size 1080p: ";
		std::cin >> size;

		if( size == 0.0f )
			break;

		doConvolution( size );
	}

	return 0;
}

void doConvolution( float size )
{
	// Allocate srcImg 1920x1080
	Image srcImg( 1920, 1080 );
	_tprintf( _T("Allocate srcImg 1920x1080 - donen") );

	// Initialize srcImg
	{
		Ipp32f *pixelPtr = NULL;
		for( int y = 0; y < srcImg.getSize().height; y++ )
		{
			pixelPtr = srcImg.getPtr( 0, y );
			for( int x = 0; x < srcImg.getSize().width; x++ )
			{
				*pixelPtr = 100;
				pixelPtr++;
			}
		}
	}
	_tprintf( _T("Initialize srcImg - donen") );

	// Allocate kerImg. Size multiplied by srcImg rounded to next odd integer
	Image kerImg( ( int )ceil( srcImg.getSize().width * size ) | 1, ( int )ceil( srcImg.getSize().height * size ) | 1 );
	_tprintf( _T("Allocate kerImg - donen") );

	// Initialize kerImg
	{
		Ipp32f *pixelPtr = NULL;
		for( int y = 0; y < kerImg.getSize().height; y++ )
		{
			pixelPtr = kerImg.getPtr( 0, y );
			for( int x = 0; x < kerImg.getSize().width; x++ )
			{
				*pixelPtr = 100;
				pixelPtr++;
			}
		}
	}
	_tprintf( _T("Initialize kerImg - donen") );

	// Padding is half of kerSize rounded down to nearest even number
	int width_pad = kerImg.getSize().width >> 1;
	int height_pad = kerImg.getSize().height >> 1;

	// Allocate repImg
	Image repImg( srcImg.getSize().width + width_pad * 2, srcImg.getSize().height + height_pad * 2 );
	_tprintf( _T("Allocate repImg - donen") );

	// Repeat borders of imgSrc
	::ippiCopyReplicateBorder_32f_C1R( srcImg.getPtr(), srcImg.getStepBytes(), srcImg.getSize(),
									   repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(),
									   height_pad, height_pad );

	// Allocate convImg
	Image convImg( srcImg.getSize().width, srcImg.getSize().height );
	_tprintf( _T("Allocate convImg - donen") );

	IppStatus status = ::ippiConvValid_32f_C1R( repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(),
												kerImg.getPtr(), kerImg.getStepBytes(), kerImg.getSize(),
												convImg.getPtr(), convImg.getStepBytes() );

	_tprintf( _T("Convolve - donen") );

	std::cout << "Input width: " << repImg.getSize().width << ", Input height: " << repImg.getSize().height << std::endl;
	std::cout << "Kernel width: " << kerImg.getSize().width << ", Kernel height: " << kerImg.getSize().height << std::endl;
	std::cout << "IppStatus code for ippiConvValid_8u_C1R: " << status << std::endl << std::endl;
}

It seems that getPtr( int x, int y ) is wrong, since it calculates y * size.width instead of y * stepBytes.

Quoting Thomas JensenIt seems that getPtr( int x, int y ) is wrong, since it calculates y * size.width instead of y * stepBytes.

You have a working test-case ( Thank you, Ryan ) and you could easily verify it. Please let us know results...

Quoting RyanWhat are the results of your testing?

When the test application crashes these two 'ippi' functions are shown in the Call Stack of a Visual Studio 2005:

     ...
     ippiZigzagFwd8x8_16s_C1()               // crash was here
     ...
     ippiCopy_32f_C1R()                               // it worked
     ...

and Output Window shows

     Unhandled exception ... 0xC0000005: Access violation writing location ...

Leave a Comment

Please sign in to add a comment. Not a member? Join today