\\heap_arrays Turn On, But Still Not Enough

\\heap_arrays Turn On, But Still Not Enough

I have asked the question here, but in a slight different tone. Here's the follow up.I have the following fortran code:

 subroutine chartest(maxncv,ldv)

  !DEC$ ATTRIBUTES DLLEXPORT::chartest
  !DEC$ ATTRIBUTES ALIAS:'chartest'::chartest
   !DEC$ ATTRIBUTES VALUE :: maxncv,ldv

 &    

   integer, intent(in)  :: maxncv, ldv

  Double precision
 &                  v(ldv,maxncv),d(maxncv,2)

   print *, 'hello'

  end

and the C# code.

  public static extern void chartest(

       [MarshalAs(UnmanagedType.I4)] int maxncv,
          [MarshalAs(UnmanagedType.I4)] int ldv


   );

In order to handle large input parameters, I turn on the \\heap_arrays option as suggested, and input 100 (kb) in the heap array item. But no matter how many I input, if maxnvc and ldv is large enough, the program will still crash.Any idea how to solve this issue?I am using version 11.0.066.For your information, I'm doing this for a structural software and the structure can go up to millions and tens of millions of degree of freedom, which means that both maxncv and ldv must be big enough to accomodate them. I need a general solution here.

9 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Unfortunately, there isn't a "general" solution. Among the needed parameters will be a decision on how big a system and how much RAM you will buy, and whether you will accept the software requirements (e.g. 64-bit OS) to make use of it.
Automatic arrays aren't recommended for this sort of usage, due to lack of error handling. Allocatable arrays will permit you to supply code to signal where a problem occurs during allocation, if allocation fails.
It's spelled /heap-arrays. No doubt you figured that out. Are you saying you set /heap-arrays:100 ? If you have many such arrays you may use excessive stack at that setting, and almost certainly will need to increase the Windows /link /stack: value well above default.
If you're trying to work within the default stack, you would reduce, not increase, the /heap-arrays switch-over value.

Tim, are you suggesting that I preallocate the arrays in C# and then pass them into the fortran code?

and whether you will accept the software requirements (e.g. 64-bit OS) to make use of it.

Also, I am compiling the above code so that they can be used in 64 bit OS. And yet, I still get the stackoverflowexception there. How does 64 bit OS suppose to help here?

No, if it makes sense to allocate in Fortran, that's best.
Perhaps we have a language misunderstanding. When you said you wanted a general solution, I assumed you meant to avoid address space limits on size.
True, if you are running in Windows, but aren't willing to use the /link /stack or similar option, 64-bit OS may not help.

Tim, if you were me, and given that I would prefer for it to be allocated in fortran, what would you do to overcome this problem?

Whichever side does the allocation, the other side may need the pointer/reference/(or your "handle") to this memory for use in subsequent calls (unless the allocation is made on a per call basis). The side that does the allocation must also be the side that performs the deallocation. As Tim suggest, it might be best to have the Fortran side perform the allocation/deallocation due to its "stronger" array descriptor management.

Also, do not pass the "value" off an array. Pass its reference or pointer.

Jim Dempsey

www.quickthreadprogramming.com

Whichever side does the allocation, the other side may need the pointer/reference/(or your "handle") to this memory for use in subsequent calls (unless the allocation is made on a per call basis). The side that does the allocation must also be the side that performs the deallocation. As Tim suggest, it might be best to have the Fortran side perform the allocation/deallocation due to its "stronger" array descriptor management.
This is exactly what I did; and I got a stackoverflowexception, which was the reason I setup this thread. How would you solve the problem here?

Also, do not pass the "value" off an array. Pass its reference or pointer.Is there any reason for passing reference? Because when I pass the value everything seems alright, or is it that there is a hidden problem?

Scalar variables passed into subroutine/function generally are passed by value.
Scalar variables returned from subroutine/function are passed by reference (or value when function return).

My stress of do not pass by value related to passing large arrays by value. Typically in Fortran (and in C/C++) you pass large objects by pointer (or reference in C++ and Fortran). However you could mistakenly assume that when you pass a scalar (e.g. integer, float, double) by value that arrays ought to be passed by value too. Passing an array by value would require the entirearray be placed on stack as opposed to pointer or descriptor.

Fortran has array descriptors (think of it as a struct containing geometry information and a pointer to the data). A call from Fortran containing an array can pass the descriptor or address of the first cell in the array. But I suppose if you have VALUE attribute on an array, the entire array is placed onto the stack.

Fortran allocations need to be done with care when the allocated object is to live past the function call. In Fortran, (newer versions) if you allocate using an array descriptor that is stack local to the fortran subroutine, the array is auto-deallocated on function return. If the array descriptor has SAVE attribute then the (only one) allocation lives past the function return. MODULE array descriptors persists too. Fortran does permit you to allocate to pointer. When allocating to pointer the allocation persists past return from subroutine. As to which technique is the best, this will depend on the applicaiton. When using specific "globla" allocation then it is best to place the array descriptors in MODULE. When using many dynamic arrays, then pointer might be advised. When single allocation then SAVE of MODULE.

Jim

www.quickthreadprogramming.com

Just restating what others above have written for clarity, for your example in the original post, what is being suggested as a first step is something like:

subroutine chartest(maxncv,ldv)     
!DEC$ ATTRIBUTES DLLEXPORT::chartest  
!DEC$ ATTRIBUTES ALIAS:'chartest'::chartest  
!DEC$ ATTRIBUTES VALUE :: maxncv,ldv  

    integer, intent(in)  :: maxncv, ldv  
    
    double precision, allocatable ::  v(:,:), d(:,:)
    integer :: stat
  
    write (*, *) 'ldv is:', ldv, ' and maxncv is:', maxncv
  
    ALLOCATE(v(ldv,maxncv), d(maxncv,2), stat=stat)
    
    if (stat /= 0) then
        print *, 'I ran out of memory :('
        return
    end if
    
    print *, 'hello'  
    
end subroutine chartest

d and v will be automatically deallocated prior to the chartest procedure returning back to c#. If you want you can deallocate them explicitly using a deallocate statement.

I can't comment with any authority on the requirements for this to be compatible with c# calling conventions (because my total experience with c# is limited to one program of about ten lines length which I wrote about two minutes ago to test the above), but this seems to work for me on a 32 bit system for maxncv and ldv that are in the thousands. If you want them to be much bigger than that, then you need to go and play in the 64 bit sandpit.

IanH

Leave a Comment

Please sign in to add a comment. Not a member? Join today