I am evaluating communications between MICs and host using MPI. I found that the bandwidth results are not bad between MICs, especially between two MICs that attached to different hosts (only ~2.45MB/s). Is it expected?
Besides, I notice that there are some lower level communication APIs like SCIF. Will this increase the communication performance in terms of throughput? Normally in what case will people think about using SCIF? I tired to google some sample codes of using SCIF, but only found Intel User Guide. So it seems to me that very few people are really using this.