Forum Jump

Select Group :
Select Forum :
Sorted By :
Sort Order :
From The :
 
Thread Tools  Search this thread 
desiramoobranz.co.nz
June 10, 2009 9:48 PM PDT
Coupling two programs with shared memory
Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs) performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by  some linkage.

Thank you in advance for your help.

Desi 
 
jaredkeithwhite
Total Points:
320
Status Points:
270
Green Belt
June 11, 2009 12:41 AM PDT
Rate
 
#1

You should look at something like MPI.  To my knowledge, modern operating systems are not going to offer you much support in the way of two programs accessing one another's memory.  You can find some good resources about MPI here:

http://www.mcs.anl.gov/research/projects/mpi/

Also:

http://www.open-mpi.org/

Additionally, the Intel Compiler (for both C++ / Fortran) support OpenMPI.

Regards,
-Jared



Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs) performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by  some linkage.

Thank you in advance for your help.

Desi 
Raf Schietekat
Total Points:
16,565
Status Points:
16,565
Black Belt
June 11, 2009 2:22 AM PDT
Rate
 
#2

POSIX support takes the form of shm_open(3) and shm_unlink(3), which build on top of mmap(2).
pvonkaenel
Total Points:
3,495
Status Points:
2,995
Brown Belt
June 11, 2009 5:33 AM PDT
Rate
 
|Best Answer
#3 Reply to #2

Under Windows I 've used a block of named shared memory to pass data between applications.  You use CreateFileMapping() and MapViewOfFile() on one side and OpenFileMapping() and MapViewOfFile() on the other.  However you need to be very careful about unsynchronized access into the memory from multiple applications and must use the heavy weight named synchronization objects which are quite expensive.

In my case it work very well in a pipelined system where events were triggered in one app when data was available for consumption in other apps.

Peter
adunsmoor
Total Points:
240
Status Points:
190
Green Belt
June 11, 2009 9:16 AM PDT
Rate
 
#4
The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI.

IPC:  http://www.boost.org/doc/libs/1_39_0/doc/html/interprocess.html

MPI: http://www.boost.org/doc/libs/1_39_0/doc/html/mpi.html

I haven't used either of these but happened to read the documentation yesterday.   The interprocess library seemed well put together and hid some of the intracacies of using the lower levels interfaces.  Especially if you have to do any cross platform work.
Raf Schietekat
Total Points:
16,565
Status Points:
16,565
Black Belt
June 11, 2009 10:08 PM PDT
Rate
 
#5 Reply to #4
"The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI."
Going with boost is rarely a bad idea, I was thinking of amending my suggestion with a reference myself.

"Especially if you have to do any cross platform work."
Without boost or even if I don't really need it (all targets would have to provide it), I would probably try to implement shm_open and shm_unlink as adapters in terms of the native API (which was not specified), instead of inventing yet another layer.

desiramoobranz.co.nz
June 14, 2009 3:55 PM PDT
Rate
 
#6 Reply to #5
Many thanks for your replies. I have implemented the, CreateShareBuffer(),OpenShareBuffer(),CloseShareBuffer(), DeleteShareBuffer() approach with success. Problem solved now.

Cheers

Desi


Intel Software Network Forums Statistics

8285 users have contributed to 31229 threads and 99107 posts to date.
In the past 24 hours, we have 7 new thread(s) 35 new posts(s), and 47 new user(s).

In the past 3 days, the most popular thread for everyone has been comparison cilk++, openmp, pthreads first results The most posts were made to comparison cilk++, openmp, pthreads first results The post with the most views is Very amusing...  Escalated as

Please welcome our newest member tvinni