I have recently replaced synchronous read and write operations by asynchronous operations (unformatted, sequential access) in a FORTRAN application. The application is faster and output the correct results with small datasets but it crashes with large ones (it has to read and write 20GB temporary files a couple of times).
I suppose that it happens because the system is running out of memory. Indeed, I realized that the data+stack usage (DATA column of the top command) keeps increasing from the launch of the application and reaches 60GB when it crashes (48GB of RAM on the machine). To compare, only 300MB are used when running the application with synchronous calls.
I dont think that there is any logic mistake in the source code as I observe the same behavior when doing a WAIT immediately after an asynchronous READ or WRITE. Calling FLUSH right after the I/O operations has no effect on the memory use either.
I suspect that it may be due to the system buffer cache filling up but I am not sure.
Does anybody have any idea about the issue? Is there any way to limit the amount of buffer cahceused by the asynchronous I/O operations in FORTRAN?
Thanks a lot,