Porting Fortran/C from Linux to Windows

Porting Fortran/C from Linux to Windows

Hello,

I am currently working on a project where we have to port 64-bit Fortran/C code from Linux to Windows. We have chosen the Intel Fortran compiler for Windows because we had a very good experience when we used it to port from SGI to Linux. I was pleasantly surprised to see that the Intel compiler gives access to Win32 subroutines for semaphores, mapped files, etc.

My current issue is how to implement shared memory used to communicate between different processes. In Linux, we do the following:

  1. We declare variables and assign pointers to them. Example integer*4 var1(100), var2(100), var3(200),.... pointer(var1_p,var1) pointer(var2_p, var2), pointer(var3_p, var3),...
  2. After we know how much memory space we are going to need (by using the size of the variables and arrays), we call shmget and shmat. If we are successful, we get back the address at which our share memory was set, i.e. myaddress
  3. Once we know the start address of our shared memory (myaddress), we map our pointers: var1_p = iand((myaddress+7),0xFFFFFFFFF8), var2_p = iand((var1_p+4*100+7),0xFFFFFFFF8), var3_p = iand((var2_p+4*100+7),0xFFFFFFFF8), etc
  4. Then, we can access var1, var2, var3, etc directly without the need for an offset.

My question is if anybody knows if we can do something similar when we use OpenFileMapping, MapViewOfFile, etc in Windows or do we have to redesign our approach. This is one big concern I have. I do not how to setup the memory mapped file such that I can access variables independently like we do with our approach in Linux.

Any comment would really be appreciated.

Thanks,

Aldo L.

16 Beiträge / 0 neu
Letzter Beitrag
Nähere Informationen zur Compiler-Optimierung finden Sie in unserem Optimierungshinweis.

Can you do this? Yes,with similar methods. But there's a much easier option. Put your shared variables in a Fortran module in a DLL (with the variables DLLEXPORTed.)Link the DLL with the data section marked shared read-write (/section:.data,RWS) and the variables given an initial value with DATA or an initialization expression. Then link your executables which USE the module to the DLL. Windows will do the rest. We provide a sample DLL\DLL_Shared_Data that demonstrates this.

Steve - Intel Developer Support

Steve,

Thank you for your response. I will look for the sample DLL.

Aldo L.

The MapViewOfFile will return a pointer in the current process to the shared memory object, this pointer does not necessarily point to the same address in the different process virtual address.

What you describe in steps 1:4 above is what you would do with Cray pointers. I advise against using Cray pointers.

Instead, use:

USE, INTRINSIC :: ISO_C_BINDING

...

CALL C_F_POINTER(cptr, fptr[, shape])

You might want to consider passing in the front of the buffer 1) a signature to be use to verify that the contained data is what you expect it to be. 2) count of number of objects, 3) count number of object sizes, 4) blob of data.

cptr should be of type INTEGER(C_PTR) (either 4 or 8 depending on 32-bit or 64-bit process). You manipulate the offsets using integer expressions.

fptr can be something like:

REAL(4), POINTER :: var1(:)

INTEGER(C_PTR) :: SharedMemoryBuffer
INTEGER(C_INT64_T),POINTER :: Signature
INTEGER(C_INT), POINTER :: nObjects
INTEGER(C_INT), POINTER :: ObjectDimensions(:) ! 1D
REAL(C_FLOAT), POINTER :: aFloatArray(:)
...
INTEGER(C_PTR) :: NextObject

SharedMemoryBuffer = YourFunctionMappingFile()
if(SharedMemoryBuffer .eq. 0) STOP
call c_f_pointer(SharedMemoryBuffer, Signature)
if(Signature .ne. YourRequiredSignature) STOP
NextObject = SharedMemoryBuffer+sizeof(Signature)
call c_f_pointer(NextObject, nObjects)
NextObject = NextObject + sizeof(nObjects)
call c_f_pointer(NextObject, ObjectDimensions, /nObjects/)
NextObject = NextObject + sizeof(ObjectDimensions)
call c_f_pointer(NextObject, aFloatArray, /ObjectDiminsions(1)/)
NextObject = NextObject + sizeof(aFloatArray)
...

Jim Dempsey

 

www.quickthreadprogramming.com

Jim,

Thank you for your response. I am currently researching the DLL option.

I really appreciate you guys taking the time to address my issue.

Thanks again,

Aldo L.

Hello,

I am researching Steve's suggestion to put the variables in a DLL, but now I have a question related to this. I got the sample DLL program to work. I even added some Fortran structures and characters to make sure it would still work, and of course, it did. Now I am converting one of our header files that used to have the Cray pointers for setting up shared memory and adding them to a module to create the DLL. I was able to successfully create the libdis.dll (name of module within dll is dis_shared_globals). I have another project (I am using Visual Studio 2012) that is creating a static library, which will later be used to create some executable. On one of the subroutines that initializes the variables from shared memory (globals in the module inside DLL), I added the statement "use dis_shared_globals". The problem I have is that when I try recompiling the static library, it is not finding the DLL. The error I get is "Error opening the compiled module file. Check INCLUDE paths [DIS_SHARED_GLOBALS]". Where do I tell the compiler to look for my custom DLL (libdis.dll)? Recall that I am building a static library, and I do not see where to define this under the Project Properties. I think that it should work because we are utilizing USE statements in other subroutines to import methods from ifport, etc.

To summarize:

  1. Using Visual Studio 2012
  2. Name of DLL is libdis.dll, which is saved to C:\mylibraries
  3. Under Project Properties -> Fortran -> General -> Additional Include Directories, I added C:\mylibraries
  4. Since my project was setup to build a static library, I do not have any Linker options - or am I missing something?
  5. Using "use dis_shared_globals" to try to access variables from DLL.

Thanks,

Aldo L.

My sincere apologies,

Steve told me: "Link the DLL with the data section marked shared read-write (/section:.data,RWS) ". I was trying to do this when compiling the static library. I will read carefully next time.

Thanks,

Aldo L.

Sorry again,

Turns out that I had made code changes in other subroutines. When I saw the new errors, I thought I had passed the error about not finding the DLL.(I was not scrolling down my error window) Thus, I am still having this issue. I apologize for the confusion.

I noticed that under libdis\libdis\Debug\ I get dis_shared_globals.mod. Do I put this file somewhere else? How about libdis or libdis.dll?

Nothing like being in a nested loop of being wrong.

Thanks,

Aldo L.

As background: you are seeing an error from the compiler (versus the linker).  Note that the compiler doesn't "consume" DLL's.  (On Windows, neither does the linker, unlike some other operating systems.)  The compiler is complaining about not being able to locate the mod file (the one that you found) in response to a USE statement referencing that module.  The mod file is the compiler's method of communication between separate invocations of the compiler, of the relevant parts of a MODULE through to where the module is USE'd.

Within Visual Studio, the Additional Include Directory property is used by the compiler to search for mod files.  This property corresponds to the /I command line option.  This command line option adds the specified directory to the INCLUDE paths for the compile.  Environment variables also influence this list of paths.

(For whatever historical reason, mod files and source code fragments included by an INCLUDE line are assumed to be in the same set of directories, even though they are not really that similar conceptually.)

Within Visual Studio, if Fortran project A depends on Fortran project B (right click on the project name for project A in the solution explorer, select "Project Dependencies", select the checkboxes for the projects that the subject project depends on) then the directory where mod files from project B are typically written is automatically added to the INCLUDE path for project A.  This is the easiest solution to the error.

Setting project dependencies in this manner may also automatically include relevant final outputs from project B in the link step of project A - say if project B is a static library then that library will automatically be included in the link step for project A.
 

In message #1, I see stuff like IAND(x,0xFFFFFFFF8).  Does ifort really accept C BOZ syntax as an extension?  Those BOZ constants are hard to check for spelling in 64-bit code and some of them are incorrect in your post.  My feeling is that you just shouldn't have to count the number of F's.  If you changed this to

integer(kind(var1_p)), parameter :: align8_mask = -8

then you could write IAND(x,align8_mask) and have code that is much less error-prone.

Thanks IanH, it is seeing the module now.

Aldo L.

Zitat:

Repeat Offender schrieb:

In message #1, I see stuff like IAND(x,0xFFFFFFFF8).  Does ifort really accept C BOZ syntax as an extension? 

No.

Steve - Intel Developer Support

Hi,

I am changing my code to use the DLLs in order to implement the shared memory in Windows. The question I have is that we have structures, for example:

TYPE:: kdat
  SEQUENCE
    real*4 high_val, low_val, cent_val, tolerance_val
end TYPE kdat

TYPE(kdat) :: kdatin(max_kdat)

!dec$ attributes dllexport :: kdatin

The question is: In order for my tasks to see kdatin, do I need to initialize each structure member to non-zero?

Thanks,

Aldo L.

No, it is sufficient to initialize just part of the variable. It can be zero, it just has to be initialized.

Steve - Intel Developer Support

To everyone who has posted an answer/comment to my questions, thank you very much. Thanks to your help, we are very close to finish porting Fortran code from the 70's that used to run in Unix systems to run in Windows 7 (SGI -> Linux -> Windows). Out of 7 processes that are supposed to be running, we have 6 of them running with semaphores, shared memory, sockets, and signal handling. The last barrier has to do with cray pointers. Right now, whenever we try to access a variable associated with a cray pointer, the process aborts even though we are using Qsafe-cray-ptr when we compile. I went back to my notes when we used to compile in Linux with the Intel Fortran compiler, and this is the only flag we used to make cray pointers work. I am including an example of how we use the cray pointers (this is defined in a header file):

TYPE :: mheader
         SEQUENCE
         integer*1    var1, var2, var3, msgtype
         integer*4    timstamp
         integer*2    length, var5
end TYPE mheader

TYPE :: state
         SEQUENCE
         TYPE(mheader) header
         integer*2    id(3)
         integer*1    var10, var11
         real*4       var12(3)
         real*8       var13(3)
         real*4       var14(3)
         integer*4    var15
         integer*1    var16, var17(11)
         real*4       var18(3), var19(3)
         integer*1    var20
         character*11 name
         integer*4    var21
         integer*4    var22(219)
end TYPE state

TYPE :: action1
         SEQUENCE
         TYPE(pdu_header) header
         integer*2    id1(3), id2(3), var100(3), var101(3)
         integer*4    var102
         real*8       var103(3)
         integer*2    var104(4)
         real*4       var105(3), var106
end TYPE action1

TYPE(state) :: eventout1
TYPE(action1) :: eventout2

common /allevents/ eout_p

pointer (eout_p, eventout1)
pointer (eout_p, eventout2)

This is only an example of 2 structures, there are 30 or more. The reason for doing this, is that the msgtype inside the header structure is used to determine the message type. Once this is determined, the pointer is used to address specific locations in memory for byte manipulation (pointer arithmetic). The pointer arithmetic varies depending on the message type.

We are wondering if we can still do this in Windows. If not, that is OK. We just want to make sure there is no other way before we start rewriting large sections of code. Just like we did not know about implementing shared memory with DLLs, not sure if there are alternatives.

Once again, thanks for spending some time looking at this. Help/comments are always appreciated.

Thanks,

Aldo L.

 

Zitat:

Aldo L. schrieb:

To everyone who has posted an answer/comment to my questions, thank you very much. Thanks to your help, we are very close to finish porting Fortran code from the 70's that used to run in Unix systems to run in Windows 7 (SGI -> Linux -> Windows). Out of 7 processes that are supposed to be running, we have 6 of them running with semaphores, shared memory, sockets, and signal handling. The last barrier has to do with cray pointers. Right now, whenever we try to access a variable associated with a cray pointer, the process aborts even though we are using Qsafe-cray-ptr when we compile. I went back to my notes when we used to compile in Linux with the Intel Fortran compiler, and this is the only flag we used to make cray pointers work. I am including an example of how we use the cray pointers (this is defined in a header file):

TYPE :: mheader
         SEQUENCE
         integer*1    var1, var2, var3, msgtype
         integer*4    timstamp
         integer*2    length, var5
end TYPE mheader

TYPE :: state
         SEQUENCE
         TYPE(mheader) header
         integer*2    id(3)
         integer*1    var10, var11
         real*4       var12(3)
         real*8       var13(3)
         real*4       var14(3)
         integer*4    var15
         integer*1    var16, var17(11)
         real*4       var18(3), var19(3)
         integer*1    var20
         character*11 name
         integer*4    var21
         integer*4    var22(219)
end TYPE state

TYPE :: action1
         SEQUENCE
         TYPE(pdu_header) header
         integer*2    id1(3), id2(3), var100(3), var101(3)
         integer*4    var102
         real*8       var103(3)
         integer*2    var104(4)
         real*4       var105(3), var106
end TYPE action1

TYPE(state) :: eventout1
TYPE(action1) :: eventout2

common /allevents/ eout_p

pointer (eout_p, eventout1)
pointer (eout_p, eventout2)

This is only an example of 2 structures, there are 30 or more. The reason for doing this, is that the msgtype inside the header structure is used to determine the message type. Once this is determined, the pointer is used to address specific locations in memory for byte manipulation (pointer arithmetic). The pointer arithmetic varies depending on the message type.

We are wondering if we can still do this in Windows. If not, that is OK. We just want to make sure there is no other way before we start rewriting large sections of code. Just like we did not know about implementing shared memory with DLLs, not sure if there are alternatives.

Once again, thanks for spending some time looking at this. Help/comments are always appreciated.

Thanks,

Aldo L.

 

It appears you now have something working on Windows.  Can you now consider moving to standard Fortran instead of using extensions, obsolete coding constructs, or limiting features:

  • Use PROCEDURE pointers and standard Fortran pointers instead of Cray pointers,
  • Use "defined" kinds from ISO_FORTRAN_ENV such as REAL32, INT32 and declare variables as REAL(KIND=REAL32) :: var12(3) instead of the obsolete style of real*4.  This applies to all your program variables.
  • Consider applying BIND(C) attribute to your "structs" i.e., derived types in Fortran instead of SEQUENCE,

and so forth?  Note you have a need to port from Linux to Windows now; tomorrow you may need to port to Android (!): using standard Fortran features will truly maximize code portability.  Consider /stand:f08 compiler option.

2 cents,

Melden Sie sich an, um einen Kommentar zu hinterlassen.