Using the Intel® MPI Library in a server/client setup

Overview

In some instances, it can be advantageous to have an MPI program join a job after it has started. Additional resources can be added to a long job as they become available, or a more traditional server/client program can be created. This can be facilitated with the MPI_Comm_accept and MPI_Comm_connect functions.

Key Functions

  • MPI_Open_port - Creates the port that is used for the communications. This port is given a name that is used to reference it later, both by the server and the client. Only the server program calls MPI_Open_port
  • MPI_Comm_accept - Uses the previously opened port to listen for a connecting MPI program. This is called by the server and will create an intercommunicator once it completes.
  • MPI_Comm_connect - Connects to another MPI program at the named port. This is called by the client and will create an intercommunicator once it completes.

Notes

  • The programs must use the same fabric in order to connect, as the port is dependent on the fabric.
  • The programs must be on the same operating system in order to connect. Different versions/distributions of the same operating systems could work, this has not been tested and is not supported.
  • The method of getting the port name from the server to the client can vary. In the sample provided, a text file is written containing the port name.

Example

A very simple example is attached to this article. The server opens a port, writes the name of the port to a file, and waits for the client. The client will read the file and attempt to connect to the port. To verify that the two programs are connected, each sends a pre-defined value to the other. To compile and run the example, download the files and place them in the same folder. Open two terminals and navigate to the folder where the files are located.  In the first terminal, use:

mpiicpc server.cpp -o server
mpirun -n 1 ./server

And in the second terminal:

mpiicpc client.cpp -o client
mpirun -n 1 ./client

In Windows*, change mpirun to mpiexec. With the code as provided, the server should show:

Waiting for a client
A client has connected
The server sent the value: 25
The server received the value: 42

And the client should show:

Attempting to connect
Connected to the server
The client sent the value: 42
The client received the value: 25
For more complete information about compiler optimizations, see our Optimization Notice.

3 comments

Top
Bingwei C.'s picture

I just write a simple program to test mic,but it meet some errors.

My code is so simple:

#include <mpi.h>
#include <omp.h>
#include <stdio.h>
#include <stdlib.h>

#define N 20000

void offload_check(int rankID)
{
#ifdef __MIC__
  printf("RankID %d running on MIC\n", rankID);
#else
  printf("RankID %d running on host\n", rankID);
#endif
}

void VecAdd_omp(float* A, float* B, float* C, int size)
{
#pragma omp parallel for
        for(int i=0;i<size;i++)
                C[i] = A[i] + B[i];
}

int main( int argc, char** argv)
{
        int i,M;
        int myrank, root=0, totalrank;
        MPI_Status status;

        MPI_Init(&argc,&argv);
        MPI_Comm_rank(MPI_COMM_WORLD,&myrank);
        MPI_Comm_size(MPI_COMM_WORLD, &totalrank);

        if(myrank == root)
                printf("total rank is:%d\n",totalrank);
        M = N / (totalrank-1);

        if(myrank == root)
        {
                float *A, *B, *C;
                int size = N * sizeof(float);
                A = (float*)malloc(size);
                B = (float*)malloc(size);
                C = (float*)malloc(size);

                srand(2013);
                for(i=0;i<N;i++)
                {
                        A[i]=rand()%10;
                        B[i]=rand()%10;
                }

                for(i=1;i<totalrank;i++)
                {
                        MPI_Send(A+(i-1)*M, M, MPI_FLOAT, i, i, MPI_COMM_WORLD);
                        MPI_Send(B+(i-1)*M, M, MPI_FLOAT, i, i, MPI_COMM_WORLD);
                }

                for(i=1;i<totalrank;i++)
                {
                        MPI_Recv(C+(i-1)*M, M, MPI_FLOAT, i, i, MPI_COMM_WORLD, &status);
                }
                for(i=0;i<N;i+=10000)
                {
                        printf("%6d: %4.2f + %4.2f = %4.2f\n",i,A[i],B[i],C[i]);
                }
                free(A);
                free(B);
                free(C);
        }
        else
        {
                float *A, *B, *C;
                int size = M * sizeof(float);
                A = (float*)malloc(size);
                B = (float*)malloc(size);
                C = (float*)malloc(size);

                MPI_Recv(A, M, MPI_FLOAT, 0, myrank, MPI_COMM_WORLD, &status);
                MPI_Recv(B, M, MPI_FLOAT, 0, myrank, MPI_COMM_WORLD, &status);

                offload_check(myrank);
                VecAdd_omp(A, B, C, M);

                MPI_Send(C, M, MPI_FLOAT, 0, myrank, MPI_COMM_WORLD);

                free(A);
                free(B);
                free(C);
        }
        MPI_Finalize();
}

The errors are showing int the following text:

-bash-4.1$ mpiexec.hydra -host 172.31.1.254 -n 2 -env I_MPI_DEBUG 3 ./vectoradd_cpu_mic_symmetric : -host 172.31.1.1 -n 2 -wdir /tmp /tmp/vectoradd_cpu_mic_symmetric.out
[1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx4_0-1
[1] MPI startup(): DAPL provider ofa-v2-mlx4_0-1
[1] MPI startup(): shm and dapl data transfer modes
[0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx4_0-1
[0] MPI startup(): DAPL provider ofa-v2-mlx4_0-1
[0] MPI startup(): shm and dapl data transfer modes
Fatal error in MPI_Init: Other MPI error, error stack:
MPIR_Init_thread(658).................:
MPID_Init(195)........................: channel initialization failed
MPIDI_CH3_Init(104)...................:
dapl_rc_setup_all_connections_20(1272): generic failure with errno = 671092751
MPID_nem_dapl_get_from_bc(1239).......: Missing port or invalid host/port description in business card
Fatal error in MPI_Init: Other MPI error, error stack:
MPIR_Init_thread(658).................:
MPID_Init(195)........................: channel initialization failed
MPIDI_CH3_Init(104)...................:
dapl_rc_setup_all_connections_20(1272): generic failure with errno = 671092751
MPID_nem_dapl_get_from_bc(1239).......: Missing port or invalid host/port description in business card
-bash-4.1$

Nikolay P.'s picture

My MPICH2 working code doesn't work with impi:

login1$ mpirun -n 1 ./iclient 'TEST2'
iclient server_name TEST2
[0] iclient Error in Lookup_name: "Invalid service name (see MPI_Publish_name), error stack:
MPID_NS_Lookup(73): Lookup failed for service name TEST2"
[0] iclient MPI_Lookup_name param serv_name TEST2, port_name ?@
Fatal error in PMPI_Comm_connect: Invalid port, error stack:
PMPI_Comm_connect(139)...........................: MPI_Comm_connect(port="?@", MPI_INFO_NULL, root=0, MPI_COMM_SELF, newcomm=0x7fffed21f410) failed
MPID_Comm_connect(206)...........................: 
MPIDI_Comm_connect(579)..........................: Named port ?@ does not exist
MPIDI_Comm_connect(380)..........................: 
MPIDI_Create_inter_root_communicator_connect(134): 
MPIDI_CH3_Connect_to_root(311)...................: 
MPID_nem_dapl_get_from_bc(1224)..................: Missing port or invalid host/port description in business card

#include <stdio.h>
#include "mpi.h"
#include <string.h>
main(int argc, char **argv) 
    {
        MPI_Comm iserver;
        MPI_Status mpi_status;
        char port_name[MPI_MAX_PORT_NAME];
    char errmsg[MPI_MAX_ERROR_STRING];
        char server_name[1024]; 
    int merr,integer,size_mpi,msglen;
    int id;
    if(argc > 1)
        {
        strcpy(server_name, argv[1] );/* assume server's name is cmd-line arg */
        printf("iclient server_name %sn",server_name);
        }
        MPI_Init(0, NULL);
        MPI_Comm_size(MPI_COMM_WORLD, &size_mpi);
    MPI_Comm_rank(MPI_COMM_WORLD, &id);    
        merr=MPI_Lookup_name(server_name,MPI_INFO_NULL,port_name);
        MPI_Error_string(merr,errmsg,&msglen);
        printf( "[%i] iclient Error in Lookup_name: "%s"n", id,errmsg );fflush(stdout);
        printf("[%i] iclient MPI_Lookup_name param serv_name %s, port_name %sn",id,server_name,port_name);
        merr=MPI_Comm_connect(port_name,MPI_INFO_NULL, 0, MPI_COMM_SELF, &iserver);
        if(merr==MPI_SUCCESS)
        {
        integer=88;
        printf("Sending integer %i via MPI communication from the client. n",integer);
        MPI_Send(&integer,1,MPI_INT,0,1,iserver);
        }
    else
        {
        printf("CANT ESTABLISH CONNECTION TO THE PORT %s",server_name);
        }    
    MPI_Finalize();
}

Nikolay P.'s picture

Where are the mentioned code examples?

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.