June 30, 2009 2:46 PM PDT
Automatic allocation of allocatable components of a type
When using the assignment B = A for derived types, it seems it is not necessary to allocate any allocatable components that B may contain. Is this correct?
Consider the following example:
PROGRAM MAIN IMPLICIT NONE
TYPE T_L INTEGER,ALLOCATABLE :: L(:) END TYPE T_L TYPE A TYPE(T_L),ALLOCATABLE :: SETS(:) END TYPE A TYPE(A) :: U,V
V = U WRITE(*,*) V%SETS(1)%L(:) WRITE(*,*) V%SETS(2)%L(:)
END PROGRAM MAIN
This program initializes V correctly - which puzzles me a bit, I would have expected an access violation error. This is very convenient; but are there any limitations regarding this automatic allocation for allocatable components of derived type variables that I should be aware of?
Now - if U and V are normal allocatable (and not derived types) variables, this doesn't work of course.
Another related feature of Fortran 2003 is for allocatable arrays, where you could say:
integer, allocatable, dimension(:) :: a
a = [3,4,5]
and a would get automatically allocated to the proper shape, deallocating any previous allocation. In Intel Fortran, this behavior is NOT the default and requires an option /assume:realloc_lhs. However, the use you described with the derived type components does not require the option.
Another related feature of Fortran 2003 is for allocatable arrays, where you could say:
integer, allocatable, dimension(:) :: a
a = [3,4,5]
and a would get automatically allocated to the proper shape, deallocating any previous allocation. In Intel Fortran, this behavior is NOT the default and requires an option /assume:realloc_lhs. However, the use you described with the derived type components does not require the option.
Another related feature of Fortran 2003 is for allocatable arrays, where you could say:
integer, allocatable, dimension(:) :: a
a = [3,4,5]
and a would get automatically allocated to the proper shape, deallocating any previous allocation. In Intel Fortran, this behavior is NOT the default and requires an option /assume:realloc_lhs. However, the use you described with the derived type components does not require the option.
Steve,
While obviously convenient - is this also not a bit dangerous?
What happens if you erroneously do something like
a=[3,4,5]
... a=[3,4,5,10]
where the use of a 4-element array is a bug (lets say due to bug)
Then my understanding is that F-2003 reallocates array a "behind your back", which would make it harder to catch this bug. Conversely F-95 would generate an access error and you'd have a pretty clear indication of a bug.
I suppose this comes down to safety vs convenience, I am not really criticizing, just contemplating ;-)
Well, maybe it's not a bug? I agree that many programs would not want this behavior (especially as it adds run-time cost) which is why we turn it off by default. Unfortunately, we don't offer a way to detect the error if you have the feature disabled - that second assignment of yours would have unpredictable results.
While obviously convenient - is this also not a bit dangerous?
What happens if you erroneously do something like
a=[3,4,5]
... a=[3,4,5,10]
where the use of a 4-element array is a bug (lets say due to bug)
Then my understanding is that F-2003 reallocates array a "behind your back", which would make it harder to catch this bug. Conversely F-95 would generate an access error and you'd have a pretty clear indication of a bug.
I suppose this comes down to safety vs convenience, I am not really criticizing, just contemplating ;-)
If you know the size beforehand, you can specify the dimensions in the declaration. Then, I'd expect that the "bug" will be detected.
Well, maybe it's not a bug? I agree that many programs would not want this behavior (especially as it adds run-time cost) which is why we turn it off by default. Unfortunately, we don't offer a way to detect the error if you have the feature disabled - that second assignment of yours would have unpredictable results.
Interesting idea, Jim, but that's not the way we usually handle such things, especially when the behavior is specified by the standard. I know at least one other major compiler vendor that does just what we do in this case - require a switch to get the realloc behavior.
Sorry to hijack this thread, but I have a problem regarding exactly the mentioned functionality in v11.1.035
Above mentioned automatic allocation of type members seems to be broken in cases where the original allocated array is created using (/ /) or [ ] syntax instead of using allocate().
The code below results in an access violation during the write statement. It works fine when the allocate statement is uncommented. /assume:realloc_lhs is set during compile.
Is that expected behavior or a compiler bug?
regards, Thomas
PROGRAM AUTOALLOC
!
TYPE :: tA
INTEGER,ALLOCATABLE,DIMENSION(:) :: IDATA
END TYPE
TYPE :: tB
TYPE (tA) :: A
END TYPE
!
TYPE (tA) :: A
TYPE (tB) :: B
! Allocate(IData(3))
A%IData = (/ 1, 2, 3 /)
B%A = A
WRITE (*,*) B%A%IData
END PROGRAM AUTOALLOC
The point being not so much of what the behavior is as to when/how the behavior is enabled.
The compiler switch has global ramifications whereas an attribute restricts behavior to specific instances. Who among us has taken legacy code (F95 in this case) forward in an incremental manner? It would seem to me that an attribute would be preferred over an all or nothing approach.
The standards committee goofed (at least was short sighted and didn't look backwards while looking forward).
Also, the re-allocatable lhs attribute should (maybe it is) have a required flag bit in the array descriptor to indicate if it can/cannot be automatically reallocated.
Has anyone compiled statistics on how much time was/is/will be wasted tracking down programming errors that are now hidden by the compiler? I hope, at least, there is a diagnostic report option to report on the re-allocations (both at compile time and at run time).
Thomas, I think your program illustrates a compiler bug. I have reported it as issue DPD200137625. That you use an array constructor here does not appear to be relevant.
Has anyone compiled statistics on how much time was/is/will be wasted tracking down programming errors that are now hidden by the compiler? I hope, at least, there is a diagnostic report option to report on the re-allocations (both at compile time and at run time).
Based on comments up-thread, I take it that there's no run time check in Intel Fortran for a shape mismatch now, so the times for was/is/will be likely the same. I guess now you can assert at the end of your program/procedure that SHAPE(allocatable_variable) == [ what_I_expect ]. Previously we just sat around and waited for any incoming warheads.
An alternative attribute approach strikes me as impracticable in the face of the F2003 features that tie in with automatic reallocation (consider objects with run-time variable length parameters as function return values - the calling code doesn't always know what is going to be sent back) and the need to chain that new attribute down to nested procedure calls that wanted to use the feature.
The change is incremental, in that no well formed F95 code relies on the feature. You enable the feature, and the observable behaviour of that F95 code remains the same. The downside is the overhead of a check that ALL(SHAPE(rhs) == SHAPE(lhs)) or similar for each assignment to an allocatable. I can't see that burning lots of CPU time relative to the time to actually copy the data (or, if the check failed, the additional time required for the memory allocation).
The incremental change is to then remove any explicit coding of that check and reallocation. The explicit code is now redundant. Remove it at your leisure.
The ability to introduce these new F2003 features without needing potentially confusing rules around which types/allocatables can be lhs re-allocatable (must be reasonably confusing - as Intel's taken that approach and got it wrong in 11.1.35) seems like a pretty big upside to the new feature.
If you are really worried about it, assigning to an array section (lhs(:) = rhs) would avoid the check and give you back the previous undefined behaviour/declaration of nuclear war in the event that rhs and lhs didn't match.
>>I take it that there's no run time check in Intel Fortran for a shape mismatch now
Correct, but currently there is a compile time check. I am suggesting you keep the compile time check unless the user specifies it is safe to reallocate the specific array. The suggested attribute could at compile time permit or deny assignments on a case by case basis that would cause reallocation.
>>The downside is the overhead of a check that ALL(SHAPE(rhs) == SHAPE(lhs)) or similar for each assignment to an allocatable. I can't see that burning lots of CPU time relative to the time to actually copy the data
?? Huh?? If the compiler with new feature is presented with a reallocation situation then it will generate code containing the reallocation (no test require). If the compiler is presented with a situation where reallocation cannot be determined until run time then it will insert code to performing the shape test. The only alternative to that is to force reallocation an all lhs(:) = rhs. I'll bet the compiler is not reallocating when reallocation is not warranted (i.e. is performing the shape test when information not known at compile time). So there is no redundant code.
>>confusing rules
F95: Programmer makes programming or typographical error and attempts to use a wrong shaped array - compiler error diagnostic emitted.
F2003: Programmer makes programming or typographical error and attempts to use a wrong shaped array - error hidden, will have to discover what happened during debug, or worse yet, error hidden until after release.
F2003: Programmer wants reallocation on some arrays - how can do without potential for introducing landmines in code?
Having an all or nothing approach is bad.
Also, while changing the shape of the lhs array why not change the other attributes as well (e.g. convert array of reals into array of integers or an array into a scalar).
>>If you are really worried about it, assigning to an array section (lhs(:) = rhs) would avoid the check and give you back the previous undefined behaviour/declaration of nuclear war in the event that rhs and lhs didn't match.
This is an exchange of nuclear war for subversive action.
Assume the programmer issues
Buffer = NewData
But NewData is larger than Buffer or different shape. However consider that Buffer has a pointer pointing at/into it. Auto reallocation in this case will be particularly bad for the developer/support person. This type of error may never occure during testing but may well occure once out in the field.
I like the idea of having reallocatable/rediminsioned arrays but only when specified (note the compiler options can be all/none/asindicated).
Apologies - I wasn't clear. I meant that with the feature on the compiler now puts that ALL(SHAPE(... test in implicitly (automatically generates the equivalent machine code for it) for each assignment to an allocatable, as you describe, and there is a small runtime overhead for this. The redundant code situation is if the compiler user has previously explicitly put that test in (to mimic the F2003 reallocation behaviour when playing in F95 land), for example, user code along the lines of:
new_shape = SHAPE(the_new_array)
IF (ALLOCATED(array)) THEN
IF (ANY(new_shape /= SHAPE(array))) THEN
DEALLOCATE(array)
ALLOCATE(array(new_shape(1),...)))
END IF
ELSE
ALLOCATE(array(new_shape(1),...))
END IF
array = the_new_array
In terms of the situation with diagnostics and warnings I'm still not sure you are worse off when using the Intel compiler. Consider the following example, which hopefully isn't too contrived:
PROGRAM DoSomeAllocations
IMPLICIT NONE
INTEGER, ALLOCATABLE :: destination(:)
INTEGER SOURCE(11)
INTEGER i
!***
source = [ (i, i = 1, SIZE(source)) ]
ALLOCATE(destination(10))
destination = source
WRITE (*, 100) SIZE(destination)
100 FORMAT('Size of destination is:',I4)
END PROGRAM DoSomeAllocations
"source" is too big for "destination".
Compile with ifort /warn:all /check:all and you will get no compile time (difficult to envisage a compiler supporting this - the size in the allocate statement is not required to be (and not often?) a constant) or run time warnings (the latter is a bit of a shame, compilers from other vendors can issue a run time warning). Run the program and it will tell you that the size of destination at the end of the program is 10. My understanding though, is that this program "violates" or whatever the word is, the F95 standard as source and destination are not conformable. Here be a landmine.
Compile with the F2003 behaviour (include /assume:realloc_lhs) and again, you'll get no compile time or run time warnings (and nor should you in this case). Run the program and it will tell you that the size of destination is 11. Before you didn't know (using ifort) that you were in trouble. Now at least, you can find out that something's not right, if you bother to check.
You have a valid point about pointers that unexpectedly become undefined, though of course with the F95 behaviour you are already technically in trouble because of the size mismatch between Buffer and NewData that causes the undefinition under F2003. I ran some simple tests and got some runtime diagnostics when I tried to use a pointer that's become undefined in this way, but I've no idea whether those diagnostics are reliable.
