Intel Software Network - <![CDATA[ Intel Fortran debugger and MPICH ]]> feed http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512 en-us Re: Intel Fortran debugger and MPICH There was a thread several years ago about idb (now named idbc) and mpich:
http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/50526/
Is there nothing about updates to this on the mpich2 support sites?  Did it change that much?
Your command line appears to invoke debugging of mpiexec, not of your program.

]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Thu, 29 Oct 2009 15:48:50 -0700 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools Re: Intel Fortran debugger and MPICH
Quoting - tim18
There was a thread several years ago about idb (now named idbc) and mpich:
http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/50526/
Is there nothing about updates to this on the mpich2 support sites?  Did it change that much?
Your command line appears to invoke debugging of mpiexec, not of your program.

I guess there have been some changes. I have tried out what was suggested in that thread and it didn't work either:
neither mpirun nor the now recommended mpiexec recognize -dbg=idb as a valid option, whereas invoking it as
mpiexec -machinefile machines -n 1 idb stagyympi
yields:
problem with execution of idb  on  compute-0-0.local:  [Errno 2] No such file or directory
So, no luck there. Any other ideas?
Thomas
]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Thu, 29 Oct 2009 20:08:48 -0700 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools Re: Intel Fortran debugger and MPICH
Quoting - truedas
Hi,

$ idb -parallel mpiexec -machinefile machines -n 4 ./stagyympi

Any ideas what may be wrong here?
Thomas

Hi Thomas,

Thank you for the interest to Intel products.

The command line looks very strange. It should look like:
"xterm -e idb -pid <pid_of_mpiexec> -mpi2 -parallel /usr/bin/python"

Also you should set environment variables:
IDB_HOME=<where_idb_is_located>
LD_LIBRARY_PATH=<where_idb_libraries_are_located>
IDB_PARALLEL_SHELL=<full_path_to_ssh> - you don't need this var if you use rsh connection.
Probably, MPIEXEC_DEBUG=1

I hope this helps.

Best wishes,
Dmitry

]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Thu, 29 Oct 2009 22:57:50 -0700 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools Re: Intel Fortran debugger and MPICH

Hi Thomas,

Thank you for the interest to Intel products.

The command line looks very strange. It should look like:
"xterm -e idb -pid <pid_of_mpiexec> -mpi2 -parallel /usr/bin/python"

Also you should set environment variables:
IDB_HOME=<where_idb_is_located>
LD_LIBRARY_PATH=<where_idb_libraries_are_located>
IDB_PARALLEL_SHELL=<full_path_to_ssh> - you don't need this var if you use rsh connection.
Probably, MPIEXEC_DEBUG=1

I hope this helps.

Best wishes,
Dmitry

Hi Dmitry,
I finally got to trying that out, but setting the IDB_PARALLEL_SHELL and MPIEXEC_DEBUG as you said made things hang. When I have only IDB_HOME and LD_LIBRARY_PATH set, I can somehow start the debugger and it seems to try to attach somehow to the process (although I don't understand why the last argument is the python executable and not the actual program executable), but then it says:
Intel(R) Debugger for applications running on IA-32, Version 9.1-28, Build 20070305
Reading symbolic information from /opt/rocks/bin/python2.4...No debugging symbols found
Attached to process id 27609  ....
stopped at [<opaque> _dl_sysinfo_int80(...) 0x008ca7a2]   
Symbol "MPIR_being_debugged" is not defined.
Symbol "MPIR_Breakpoint" is not defined.
Symbol "MPIR_Breakpoint" is not defined.
::MPIR_Breakpoint has no valid breakpoint address
Make breakpoint pending on future shared library load? (y or [n])
Warning: Breakpoint not set

and hangs, no matter whether I have chosen y or n in the breakpoint question.

So, as that failed, I was prompted to look back into my MPICH installation again and notice that for some reason, it says in the configure log:
configure: WARNING: unrecognized options: --enable-debuginfo

although according to the installation guide, this is the option I should give to "provide access to the message queues for debuggers". In the configure log, it also says later:
checking debugging support... no
although I'm not sure whether that refers to this kind of debugging or rather to the debugging options for MPICH2 developpers themselves (which would not be of interest for me).
So basically, I'm rather confused about the error messages and wonder if I have to build MPICH2 in a different way to make the Intel debugger work (and if so, how).
May the problem lie here already, and how do you build MPICH2 then?
Thomas
]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Fri, 30 Oct 2009 15:45:45 -0700 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools Re: Intel Fortran debugger and MPICH
Quoting - truedas
Hi Dmitry,
I finally got to trying that out, but setting the IDB_PARALLEL_SHELL and MPIEXEC_DEBUG as you said made things hang. When I have only IDB_HOME and LD_LIBRARY_PATH set, I can somehow start the debugger and it seems to try to attach somehow to the process (although I don't understand why the last argument is the python executable and not the actual program executable), but then it says:
Intel(R) Debugger for applications running on IA-32, Version 9.1-28, Build 20070305
Reading symbolic information from /opt/rocks/bin/python2.4...No debugging symbols found
Attached to process id 27609  ....
stopped at [<opaque> _dl_sysinfo_int80(...) 0x008ca7a2]   
Symbol "MPIR_being_debugged" is not defined.
Symbol "MPIR_Breakpoint" is not defined.
Symbol "MPIR_Breakpoint" is not defined.
::MPIR_Breakpoint has no valid breakpoint address
Make breakpoint pending on future shared library load? (y or [n])
Warning: Breakpoint not set

and hangs, no matter whether I have chosen y or n in the breakpoint question.

So, as that failed, I was prompted to look back into my MPICH installation again and notice that for some reason, it says in the configure log:
configure: WARNING: unrecognized options: --enable-debuginfo

although according to the installation guide, this is the option I should give to "provide access to the message queues for debuggers". In the configure log, it also says later:
checking debugging support... no
although I'm not sure whether that refers to this kind of debugging or rather to the debugging options for MPICH2 developpers themselves (which would not be of interest for me).
So basically, I'm rather confused about the error messages and wonder if I have to build MPICH2 in a different way to make the Intel debugger work (and if so, how).
May the problem lie here already, and how do you build MPICH2 then?
Thomas

Hi Thomas,

I was talking about Intel MPI Library and IDB.
I don't know how MPICH supports IDB, but invocation looked incorrectly. Probably MPICH was configured incorrecly during installation.
An application hangs because of MPIEXEC_DEBUG variable.

IDB attaches to a python script first to get information about running processes from mtv.so. Mpiexec should wait for a debugger.

BTW: your IDB is very old... March 2007 - I'm not sure that IDB could work correctly with MPI-2 applications at that time. Could you get idb 10.1 somehow?

Regards!
Dmitry
]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Sun, 01 Nov 2009 23:04:59 -0800 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools Re: Intel Fortran debugger and MPICH
The correct start up command is:
mpirun -dbg=idb -np <number of processes> <binary name>

Two pre-requisites:
1. The environment variable IDB_HOME needs to be set to the directory where IDB resides.
2. The file mpirun_dbg.idb, which is included in the IDB 9.1 distribution, needs to be put in the "bin" directory of the MPICH installation.

Please try this out and let me know if this helps.

Best wishes,
Dmitry

]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Mon, 02 Nov 2009 22:03:55 -0800 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools Re: Intel Fortran debugger and MPICH
Some clarifications about idb 9.1 usage with mpich:

The correct start up command is:
mpirun -dbg=idb -np <number of processes> <binary name>

Two pre-requisites:
1. The environment variable IDB_HOME needs to be set to the directory where IDB resides.
2. The file mpirun_dbg.idb, which is included in the IDB 9.1 distribution, needs to be put in the "bin" directory of the MPICH installation.

Please try this out and let me know if this helps.

Best wishes,
Dmitry

Hi Dmitry,
thanks for the info, and sorry for being slow with the response.

I guess it will be difficult to get my hands on a more recent version of the compiler on that architecture, but I tried it now with the version, and it still doesn't work.
$ echo $IDB_HOME
/opt/intel/idb/9.1.045/
I also copied mpirun_dbg.idb into the bin directory where the MPICH executables live, but neither mpirun nor mpiexec recognize the option -gdb:
$ mpirun -gdb=idb -np 4 stagyympi
invalid "local" arg: -gdb=idb

I have looked in the installation guide if there is a way to "teach" MPICH the use of compilers other than gdb, but have found none.

Thomas
]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Thu, 05 Nov 2009 21:54:53 -0800 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools Re: Intel Fortran debugger and MPICH
Quoting - truedas
I have looked in the installation guide if there is a way to "teach" MPICH the use of compilers other than gdb, but have found none.

Thomas

I've looked into the MPICH2 mpiexec.py script and seems they support only GDB (-gdb) and TV (-tv) debuggers. -gdb option doesn't have parameters.
Unfortunately it's not so easy to modify mpiexec.py so that it would work with idb. If you wanna use IDB you probably need to install Intel MPI Library.

Regards!
Dmitry

]]> http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Fri, 06 Nov 2009 06:42:11 -0800 http://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/69512/ Intel® Cluster Ready Tools