Hi Folks - I've been working on getting a high performance scientific application working with the intel compiler suite (icc (ICC) 11.1 20100806) on a 20 core Intel box running RedHat 6. Program has been run successfully for years using the Gnu compiler suite. The code is already heavily threaded, but I thought I'd add the -parallel compiler option and see what else could be had. However, this has caused the program to hang at random iterations. It's always in the same function. This function is executed by multiple threads simultaneously via a thread pool. No synchronization occurs between the simultaneous instances of the function. When the program hangs, all but one of the instances of this function return, and top shows 100% CPU for the project, indicating that that thread is still running. The code is a large loop, and the hang never occurs at the same iteration of the loop.
If I reduce the size of the thread pool to 1, the run always completes, 2 it ussually completes. Increasing the size of the thread pool leads to more consistent & earlier hangs. And finally, after systematically removing compiler flags, I've found that removing the -parallel flag appears to have eliminated the problem.
So it seems like the optimizations introduced by the -parallel flag are interacting badly with this manually threaded code. Any suggestions on what I might do to resolve this? It's not a show-stopper, since I can just leave out that flag. But I'm curious what's going on and what this option could do for the performance.
Compiler command line now looks like this:
icc -xHOST -O3 -no-prec-div -c -O3 -Werror -DNDEBUG <snip lots of includes> -MMD -MP -MF build/Release_intel/GNU-Linux-x86/_ext/1472/sieve.o.d -o build/Release_intel/GNU-Linux-x86/_ext/1472/sieve.o ../sieve.cc
Link looks like this:
icc -xHOST -O3 -no-prec-div -o dist/myprog <snip lots of object files> -L/home/user/lib -L/usr/lib/mysql -L/usr/lib64/mysql -lmysqlclient -lpthread -lz -lgfortran -ltoollib_nopvm_i -lboost_iostreams -lifport -lifcoremt
There is one small bit of fortran in there - just noticed that I'm still linking against the Gnu fortran library - will go take that out now.