code crash during deallocate dynamic array for processor specific optimization

code crash during deallocate dynamic array for processor specific optimization

Code crashed when optimization set with processor specific switches.

I am using windows IVF 2011 XE Update4 for X86 compiler 32. I have a dynamic array of a user defined type data which include a dynamic array, such as
Type myDataType
...
real, dimension(:),allocatable:: m_fMemArray
End Type

Type(myDataType), dimension(:),Allocatable::myActualData

...
At the program end, I have a deallocation of
DeAllocate(myActualData).

This code runs fine in debug mode, in release mode. However, when I set the option of "Code Generation" as:
Enable Enhanced Instruction Set: Streaming SIMD Extension 2/arch:SSE2
Add Processor-OPptimized Code Path None (or Intel Netburst Microarchitecture and Pentium M...)
Intel Processor Specific Optimization Intel Netburst Microarchitecture and Pentium M...

The code will crash at the time when trying to deallocate the data array. Such optimization has been working in the previous compilers.

Is this a bug of compiler or possibly anything wrong on my side.

If anyone can point me to the right direction, it will be much appreciated.

17 post / 0 nuovi
Ultimo contenuto
Per informazioni complete sulle ottimizzazioni del compilatore, consultare l'Avviso sull'ottimizzazione

Can you post a complete program that exhibits this behaviour?

My first hunch at a report like yours is that somewhere in the program you have
an access violation (array subscript out of bounds or the wrong number of
arguments to a routine etc.) and this causes a corruption in the internal
data. The problem surfaces much later.

Another clue that this is happening is that it all works in debug mode but not
in release mode.

One possible mehod to hunt down the place where the actual damage is done
is to deallocate the array earlier in the program and then stop - or to comment out
parts of the program and see whether the crash disappears.

Regards,

Arjen

Ritratto di jimdempseyatthecove

I concur with Arjen. Looks like something is stomping on the internal structures.

What happens if you ALLOCATE, do something innocuous with the data (so code optimizer does not eliminate code), then DEALLOCATE. Do not assume the code you currently run is innocuous.

If this works (DEALLOCATE without crash), then remove the call to the innocuous code, remove the DEALLOCATE and insert the DEALLOCATE in your code halfway to where you experience the crash. Then depending on if DEALLOCATION error occures or not, move the DEALLOCATION either closer to the end or beginning. Keep track of where you move it (e.g. leave tracks in code as comments). What you are doing is a binary search for point in code where corruption causes (future)deallocation error.

Don't be surprised if bug dissappears while looking for it.

Jim Dempsey

www.quickthreadprogramming.com

Thanks very much for the advices provided by Arjen and Jim. They are really helpful.
Now I have resolved the problem without full comprehension. (I could be an IVF optimization bug).

The cause of the problem:
=================
There is a defined data type in the application, such as:

Type myDataType
...
real, dimension(:),allocatable:: m_fMemArray
End Type

There are declared such type data in the application, one is an allocatable array and the other is a scalar.

Type(myDataType), dimension(:),Allocatable::myActualData
Type(myDataType)::InitValue

In the middle of the application, as long as there is assignment of statement:

Do i = 1, N
myActualData(i) = InitValue
EndDo

This will crash the application when deallocate myActualData.

If I change the statement into
Do i = 1, N
Do j = 1,M
...
myActualData(i).m_fMemArray(j) = InitValue.m_fMemArray(j)
...
Enddo
Enddo

This will be fine.

The weird thing is that the application runs fine in the previous compilers. Also in current compiler, if in debug mode, or in release mode,with debug information exported.

I suspect that it might be in certain optimized modeassigning the value through reference rather than data. However, when I play around with simple application, it appears all the assigment is actually doing assignment by data. I can not replicate the problem using simple application even though the above mentioned data is not massively used in the application.

As I can not replicate in simplified application, I have not found a way to submit bug report to Intel yet.

Kind regards,

Wu_internet

Ritratto di jimdempseyatthecove

Wu,

What I suggest you do is to submit a problem report to Premier support with basically a copy of your above post.

Ask them to provide you with diagnostic code that will dump the contents of the array descriptors. Note,you want both the arraydescriptor of the array of your types and each array descriptor within each element of your array of types.Run a test producing dump of array descriptors with the method the code works and a second test with a dump where the code fails. Use WinDiff if problem isn't apparent at first glance.

I suspect something is wrong with how the array descriptors are initialized (or there is a side effect that disturbs the(an) array descriptor after initialization.

The first response you get from them is usually"Can you provide us with a small reproducer". Kindly explain to them you said in your first postyou tried and you cannot.Then you ask them "What would you use to diagnose corruption of an array descriptor? I am willing to help you to diagnose this problem, I suspect this is a case of array descriptor initialization or corruption but I need your help in dumping the contents of an array descriptor and identifying what I see. Let's work together to solve this problem."

While you have a work around, you would like the problem fixed because this problem may exist elsewhere in your code but the symptoms haven't caused the problem to be noticed (yet). One such problem with hidden symptom would be if the problem did not cause a crash but instead caused a memory leak. A worse problem would be an unnoticed corruption of the heap resulting in multiple objects allocated to overlapping addresses. I think you have identified a serious problem and it warrants a little bit of extra effort to diagnose and correct. The support person should understand this and work with you to correct this problem.

Jim Dempsey

www.quickthreadprogramming.com
Ritratto di Steve Lionel (Intel)

You don't need to submit a report quite yet. Use the attached and see if it helps you at all. Note that the "A0 offset" is no longer used for that purpose, so it will look strange. It will be zero unless the variable is polymorphic.

Allegati: 

AllegatoDimensione
Download array-descriptor.f902.27 KB
Steve
Ritratto di jimdempseyatthecove

Steve,

Thanks for attaching the array descriptor dump code. Will save it for a rainy day. It will be interesting to see if Wu discovers anything interesting.

Jim

www.quickthreadprogramming.com

I am also experimenting a hard crash when calling deallocate on an allocatable array which belongs to a derived type. I am also using SSE2 but disabling it does not change the behavior.

The type is:
type :: RPT21DARR
real, allocatable :: p(:)
end type RPT21DARR

The variable is:
type(RPT21DARR) :: sk_2s(30)

The call which crashes:
deallocate(sk_2s(1)%p,STAT=ir)

Calling LOC on sk_2s(1)%p gets: 96031592 which is 5B95368 in hex.

Dumping the array descriptor of sk_2s(1)%p just before the crach using the routine provided by Steve I am getting:

Dump of descriptor at location 05DEECA0
Base address: 05B95368
Element size: 4
A0 offset from base: -4
Flags: 00000005
Defined
Contiguous
Rank: 1
Dimension 1: LB 1 Extent 3005 Stride 4

The length is 3005 (as allocated). Do you see anything abnormal in this dump?

Thanks,
Rak

Ritratto di Steve Lionel (Intel)

And the error message for the crash is... ?

Steve

Citazione:

Steve Lionel (Intel) ha scritto:

And the error message for the crash is... ?

No error message at all. Just the Windows trace for an application crash:
Problem signature:
Problem Event Name: APPCRASH
Application Name: .emme.exe
Application Version: 0.0.0.0
Application Timestamp: 50ad4311
Fault Module Name: ntdll.dll
Fault Module Version: 6.0.6002.18541
Fault Module Timestamp: 4ec3e39f
Exception Code: c0000005
Exception Offset: 0002a43e
OS Version: 6.0.6002.2.2.0.256.6
Locale ID: 1033
Additional Information 1: fd00
Additional Information 2: ea6f5fe8924aaa756324d57f87834160
Additional Information 3: fd00
Additional Information 4: ea6f5fe8924aaa756324d57f87834160

Anyway, it seems that I have isolated the problem. It appears only when I use inside a OpenMP parallel DO a variable declared as:

type(DPPT22DARR) :: ycaddmk(MCLS_)
where MCLS_ is a parameter (30).

The above type is defined as:
type :: DPPT22DARR
double precision, allocatable :: p(:,:)
end type DPPT22DARR

The usage of the variable is either as:
ycaddmk(m)%p(i,j)=bla

or directly a matrix operation:
ycaddmk(m)%p=MyMatrix+MyScalar*ycaddmk(m)%p

I am trying now to reproduce it in a small project (hopefully I will be able to).

As a work around for the first usage, I created a temporary matrix and copy in/copy out ycaddmk(m)%p before/after the parallel DO (luckily the DO loop is over variable j). Unfortunately, I cannot do the same thing for the second example where the DO loop is over variable m :-(

Thanks Steve,
Rak

Ritratto di Steve Lionel (Intel)

The crash is an access violation. You might be exceeding the thread stack size - try defining the environment variable OMP_STACKSIZE to be some larger value (10000000 or such).

Steve

Citazione:

Steve Lionel (Intel) ha scritto:

The crash is an access violation. You might be exceeding the thread stack size - try defining the environment variable OMP_STACKSIZE to be some larger value (10000000 or such).

100% sure it is not that (I've got it in the past and the behavior was different. It has been already set to a higher value anyway, and increasing it more didin't solve the problem.

I started constructing a simple program to try to reproduce the problem but I run now into a compiler bug (depressing...). I am attaching the VC2008 project file which issues the message: "1>fortcom: Fatal: There has been an internal compiler error (C0000005)." It practically contains only one *.f file. Note that this does not reproduce the crash but I cannot even compile.

Perhaps more depressing: when I pressed upload, I've got an HTML AJAX error I will try to attach the JPEG screen-shot. Anyway, I am far more interested by the FORTRAN problem which is kind of show stopper of our current project than the HTML one.

Thanks Steve,
Rak

As expected, the upload didn't work as expected.

Allegati: 

AllegatoDimensione
Download error-intel.png229.31 KB

Project to reproduce the compiler bug.

Allegati: 

AllegatoDimensione
Download openmp-bug.7z601.71 KB
Ritratto di Steve Lionel (Intel)

I can't reproduce the compiler failure with the version you cited (2011 Update 4). But when I look in the build logs, I see you are actually using 11.1.048, which is considerably older! Please try it with a newer compiler.

Steve

Citazione:

Steve Lionel (Intel) ha scritto:

I can't reproduce the compiler failure with the version you cited (2011 Update 4). But when I look in the build logs, I see you are actually using 11.1.048, which is considerably older! Please try it with a newer compiler.

Hi Steve,

just to let you know that upgrading the compiler to the latest version solved both problems:
- the original one (mishandling arrays of allocatable arrays/pointers inside OpenMP do loops)
- the compiler crash inside VisualStudio when building a small example using allocatable arrays/pointers inside OpenMP do loops

Thanks,
Rak

Ritratto di Steve Lionel (Intel)

Glad to hear it. In the future, please double-check the compiler version you are using before writing up the report. I don't mind a report against an old version, but it saves both of us time if you name the correct version.

Steve

Accedere per lasciare un commento.