Forum Jump

Select Group :
Select Forum :
Sorted By :
Sort Order :
From The :
 
Thread Tools  Search this thread 
desiramoobranz.co.nz
June 10, 2009 9:48 PM PDT
Coupling two programs with shared memory
Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs) performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by  some linkage.

Thank you in advance for your help.

Desi 
 
jaredkeithwhite
Total Points:
320
Status Points:
270
Green Belt
June 11, 2009 12:41 AM PDT
Rate
 
#1

You should look at something like MPI.  To my knowledge, modern operating systems are not going to offer you much support in the way of two programs accessing one another's memory.  You can find some good resources about MPI here:

http://www.mcs.anl.gov/research/projects/mpi/

Also:

http://www.open-mpi.org/

Additionally, the Intel Compiler (for both C++ / Fortran) support OpenMPI.

Regards,
-Jared



Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs) performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by  some linkage.

Thank you in advance for your help.

Desi 
Raf Schietekat
Total Points:
16,765
Status Points:
16,765
Black Belt
June 11, 2009 2:22 AM PDT
Rate
 
#2

POSIX support takes the form of shm_open(3) and shm_unlink(3), which build on top of mmap(2).
pvonkaenel
Total Points:
3,580
Status Points:
3,080
Brown Belt
June 11, 2009 5:33 AM PDT
Rate
 
|Best Answer
#3 Reply to #2

Under Windows I 've used a block of named shared memory to pass data between applications.  You use CreateFileMapping() and MapViewOfFile() on one side and OpenFileMapping() and MapViewOfFile() on the other.  However you need to be very careful about unsynchronized access into the memory from multiple applications and must use the heavy weight named synchronization objects which are quite expensive.

In my case it work very well in a pipelined system where events were triggered in one app when data was available for consumption in other apps.

Peter
adunsmoor
Total Points:
240
Status Points:
190
Green Belt
June 11, 2009 9:16 AM PDT
Rate
 
#4
The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI.

IPC:  http://www.boost.org/doc/libs/1_39_0/doc/html/interprocess.html

MPI: http://www.boost.org/doc/libs/1_39_0/doc/html/mpi.html

I haven't used either of these but happened to read the documentation yesterday.   The interprocess library seemed well put together and hid some of the intracacies of using the lower levels interfaces.  Especially if you have to do any cross platform work.
Raf Schietekat
Total Points:
16,765
Status Points:
16,765
Black Belt
June 11, 2009 10:08 PM PDT
Rate
 
#5 Reply to #4
"The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI."
Going with boost is rarely a bad idea, I was thinking of amending my suggestion with a reference myself.

"Especially if you have to do any cross platform work."
Without boost or even if I don't really need it (all targets would have to provide it), I would probably try to implement shm_open and shm_unlink as adapters in terms of the native API (which was not specified), instead of inventing yet another layer.

desiramoobranz.co.nz
June 14, 2009 3:55 PM PDT
Rate
 
#6 Reply to #5
Many thanks for your replies. I have implemented the, CreateShareBuffer(),OpenShareBuffer(),CloseShareBuffer(), DeleteShareBuffer() approach with success. Problem solved now.

Cheers

Desi


Intel Software Network Forums Statistics

8445 users have contributed to 31553 threads and 100398 posts to date.
In the past 24 hours, we have 10 new thread(s) 30 new posts(s), and 43 new user(s).

In the past 3 days, the most popular thread for everyone has been Lost in MKL The most posts were made to TBB on linux segfaulting The post with the most views is Hi,if you were using imsl yo

Please welcome our newest member nonamez