a problem about the asynchronous write

a problem about the asynchronous write

I want to test the usage of the asynchronous write statement, but the code below give an error during executing

program main
    
    use portlib
    implicit none
    integer, dimension(:,:), allocatable :: d
    integer :: i, j
    real :: ecl_time
    
    allocate(d(1000,1000))
    
    d = 3
    
    open(unit=8, file="x.dat", asynchronous = 'yes', BUFFERED = 'yes', FORM='unformatted',ACCESS='direct', status='replace', recl=1)
    
    ecl_time = timef()
    do i=1,1000
        do j =1,1000
            write(8, asynchronous = 'yes', rec=i+1000*(j-1)) d(i,j)
        end do
    end do
    ecl_time = timef()
    
    wait(8)
    deallocate(d)
    close(8)
    write(*,*) "ecl_time = ", ecl_time
    
    
end program main
    
    

error info

Microsoft Visual C++ Debug Library

Debug Assertion Failed!
...
Expression:_CrtIsValidHeapPointer(pUserData)

For information on how your program can cause an assertion failure, see the Visual C++ documentation on asserts.

Thank you for your help!

7 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Something is definitely going wrong here. We'll investigate. I don't see anything wrong in your source code.

I have escalated this to the developers as issue DPD200242778. We seem to have a problem when a large number of small asynchronous I/O operations are done without waits. Is this really what your application wants to do?

Steve - Intel Developer Support

Steve,

I seem to recall this problem a few years ago (several versions ago) where the culpret was the asychronous writes had a byte sized ID number for pending I/O operations. When you had more than 127 (or was it 255) pending operations then the app crashed or behavied badly.

To the original poster

In the WRITE statement with ASYCHRONOUS='YES' you can add ID=id-var to obrain the ID for the current write. I suggest you create a ring buffer array of IDs. The size being .LE. 127. While the FORTRAN standard does not define the values of valid ID's, you may be able to determine this via investigation. Perhaps 0 ir -1 is never used. Fill this array with a "never used" value. Then on each write, before the write, test the location of the ring buffer to see if the prior modulus write contains the never used value, if so, then issue write, obtaining next ID, insert ID into array, advance pointer (mod size of ring buffer). Should the next location in the ring buffer contain a valid ID, then perform INQUIRE with PENDING=pnd and ID=id-var specifiers. If the I/O is pending then stall, if not pending, then mark the pending I/O array entry with "never used" and loop back to the write.

Incorporating this technique should not be overly hard.

Jim Dempsey 

www.quickthreadprogramming.com

Hi Steve, I am really interested in the asynchronous io, and i want to use this tool to ensure my io operation won't block the compute statement. So I test all the circumstances to see the performence of the AIO and to determine how to use it. When I test the large number small AIO operation, it failed.

I know the extra price of the large number of small AIO is also huge. can you give me some advices to instruct me how to use the AIO?

Thank you!

Typically, AIO is best used for sequential writes of large unformatted records. Using it for direct access writes of very short records is likely to result in worse performance overall, as there is some overhead in establishing the asynchronous context. You want to use AIO when the operating system is going to take a long time to finish the request.

Steve - Intel Developer Support

>> i want to use this tool to ensure my io operation won't block the compute statement.

As mentioned in my earlier post, earlier versions of IVF had a limitation on the number of in-flight (pending) Asychronous I/O requests. The limitation appeard to be an INTEGER(1) was used to hold the ID. As to what the current version of compiler uses, I cannot say. Assuming the ID remains with the limitation of 127 or 255 pending I/O's. Is this enough? If yes, then the array of pending I/O ID's should provide sufficient protection. If you want more pending I/O's than supported internally by IVF (127, or 255, or larger in event larger ID numbers supported), then you will have to program somewhat differently. An example of which is:

Replace asychronous write with call to copy data to buffer(s) that you maintain.
Enqueue reference to buffer for seperate thread to perform I/O.
Have seperate thread dequeue I/O requests and write to file (note this can be designed to write one or more output buffers).

Your limitation on pending I/O requests (without interfering on producer thread) is a limitation on memory allocations for buffers.

Jim Dempsey

www.quickthreadprogramming.com

For a release later this year, we've made some changes that should eliminate the error you saw. We have also identified ways we can improve our implementation of asynchronous I/O and will be working on that for a future release.

Steve - Intel Developer Support

Leave a Comment

Please sign in to add a comment. Not a member? Join today