Compiling with OpenMP directives : debugger provides wrong line numbers

Compiling with OpenMP directives : debugger provides wrong line numbers


I have some difficulties due to OMP directives.  I am compiling on linux64 with ifort build 20120212 and following flags:
 -g -O0 -fp-model precise -openmp  -Dmkl -ftz -extend-source -DFCCI -check uninit -fpe0 -ftrapuv  -traceback

. I am stepping into a subroutine during debug (idbc or gdb). But both debuggers provides exactly the same line number (888) which does not correspond to the called subroutine. The called subroutine is written in a source file which contains 5 subroutines. I am calling the 5th one which start at line 1567. Line 888 is the middle of the 3rd subroutine.

Each subroutine in this source file uses OMP directives :
[fortran source]

The problem disappear if I remove these OMP directives. Line number is also consistent with source file.
Is it expected or a known issue ?

Thanks for any help.


15 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

It's possible.  We have had some issues in the past with the line number correlation being incorrect when debugging.

I don't see where we've fixed any of those since 12.1, so updating to a new compiler is not likely to help.

I appreciate that you can't share the full project, but can you characterize the program better?  Is line 888 legitimately somewhere in the backtrace?   If not, is that third subroutine (where line 888 lives) also called in the same OMP region as the fifth subroutine?  Any other clues? 

In general, the previous problems were caused by a "stale" line number being reused, and that's what I'd expect was happening here too.

                   Thanks --


Hi Lorri,

Thank you for this feedback.

The situation occurs with 1 threads. Subroutine_5 is called first. but i step directly on line 888.

Furthermore even if i create the breakpoint before running the program with idbc, the line number is not correct (888 again) :

[schwoertzig]$ idbc --args ./my_program -nthread 1
Intel(R) Debugger for applications running on Intel(R) 64, Version 12.1, Build [77.329.14]
object file name: ...
 Reading symbols from ...done.
(idb) break SUBROUTINE5
Breakpoint 1 at 0x656f38: source.F, line 888.
(idb) bt
The "backtrace" command has failed because there is no running program.

Here are line details from source.F :
Subroutine 1 : line 1-361
Subroutine 2 : line 368-722
Subroutine 3 : line 729-1226
Subroutine 4 : line 1233-1560
Subroutine 5 : line 1567-1688

Here is an extract of the flowchart.

 |   +--SUB_A
 |    |   +--SUB_A
 |    |    |   +--Subroutine 5
 |    |    |   +--Subroutine 4
 |    |   +--SUB_C
 |    |    |   +--Subroutine 3
 |    |   +--Subroutine 1
 |    |   +--Subroutine 2

Do you need more information ?

Thank you and Best Regards




Moving Subroutine5 to a new source file also fixes the issue. Debuggers are able to locate the correct line number corresponding to the subroutine beginning.
Does anybody have an explanation taking into account this new clue ?

Thanks and Best Regards

Are you telling us in-lining affects your debugging?

How about

-debug inline-debug-info



if you don't want to compile procedures separately?


Tim Prince wrote:

How about
-debug inline-debug-info

Thank you for this idea.
I just tested thses compilation options without success :
  -debug inline_debug_info
  -inline none

  • Idbc –args ./myprogram –nthread 1

(idbc) break SUBROUTINE_5

Breakpoint 1 at 0x5f6b08: source.F, line 888

This is still the unexpected line number located in the middle of SUBROUTINE3 (in the same source file).



Are these subroutines in a module? If so, and I've experienced this myself (and I imagine others here have too), it is possible that the output directory for your .mod and .o files is not the input folder for the link phase of the .o files. In this situation you end up compiling the new code and linking the old code. A quick test for this is to insert a "write(*,*) 'new code'" somewhere in the new code that gets executed early. If this write does not occur, then check your paths as this would indicate you are linking old object files.

Inlining and any optimizations will affect line numbering. A technique I often use is to insert a CALL to a subroutine that is explicitly stated as to not be inlined (and additionally not visible to IPO as it aggressively inlines). The called routine doesn't need to do anything. Usually, the line number where the call is made is always available. Once at break point, you can then single step (and disregard information presented as to line number).

Jim Dempsey


jimdempseyatthecove wrote:

Are these subroutines in a module?
Jim Dempsey

Hi Jim,
Thank you for this information.
In my case subroutines are not in a module. I also compiled from scratch when i tested the inlining flags.

Using your tick, I added CALL VOID() (on line 1615) at the beginning of the SUBROUTINE5 (which beginns on line 1567).  I made a breakpoint inside VOID SUBROUTINE which is empty. Using "up" command to move in the backtrace debugger says to be on line 935 which is again somewhere in the middle of SUBROUTINE3 which is not related to SUBROUTINE5. This is really enigmatic.




Looks like compiler optimization issue (rearranging code but not line numbers).

One other potential issue is: does the statement following the CALL VOID() contain a "variable" than is a statement function?

Some older programs made use of statement functions. These appear in the data declaration section and when immediately in front of code, are often confused with code.

AVG(A,B,C) = (A+B+C)/3.

The above, when appearing in the data section is a statement function (A,B,C must have been declared).
If the above appears after the first code statement, then it is a statement (not a "statement function").

When debugging, the debugger might hop to the location of the declaration of the statement function (then hop back on next step).

Jim Dempsey

Hi Jim,

Thank you for the idea. There are only Data statments before and no such situation after that. (Replacing data statments does not change the behavior.)

I found a second workaround :
In SUBROUTINE_2 there are 2 !$OMP DO block :

      DO IG = 1, NG
      END DO
      DO IG = 1, NG
      END DO

Removing $OMP END DO directives also fixes the issue : Breaking into SUBROUTINE_5 leads to the correct line number (1567 instead of 888).



That should be sufficient information for Intel support to go on, but they are hesitant to do anything without a simple reproducer. If you can put together a simple reproducer then they will likely fix the problem.

Jim Dempsey

I created a much shorter test program based on all the details in this thread but was unable to reproduce the failure so my test is not imitating enough of the original code to produce the line number confusion. IDB is finds/sets the break at the correct line number for subroutine 5 in my case with 14.x and older 12.1 compilers.

Is there any chance at getting a reproducer?

You can provide those in private via a private message or submitting an issue under Intel Premier Support.

Thank you Jim - couldn't have said it better myself!  :-D

Yes, Thierry, I tried to create my own reproducer given the patterns you've described, with no luck.
I don't need a runnable reproducer, just something I can compile and look at our internal tool that "tracks" line number<->source correlations.

And, you can use the "Private Message" utility to send it along for further security.

           Thank you --

                                     -- Lorri


I built a small example using OMP directives. This source code structure is a litle bit different from the original description but the consequences are similar ones. Debugger has some trouble to match source lines.

I am not sure if I did something wrong or if it is highlithing a compiler bug. You will find below all needed information.

           --- Thanks and Best regards.

                          --- Thierry


      CALL SIGEPS51()

SOURCE FILE : sigeps51.F

      TFEXT  = 0.0D00
      TFEXTT = 0.0D00      
      J = 2
      I = 0
      IF (J==1)THEN
#include ""
          TFEXT = TFEXT + TFEXTT
#include ""
        DO I=1,2
#include ""
          TFEXT = TFEXT + TFEXTT
#include ""
#include ""
          TFEXT = TFEXT + TFEXTT
#include ""
      print *, "1-UN"
      print *, "2-DEUX"
      print *, "3-TROIS"            
#include ""
          TFEXT = TFEXT + TFEXTT
#include ""


#if defined(CPP_mach)
         PRINT *, "CPP_mach=", CPP_mach
!         PRINT *, "CPP_mach NOT DEFINED"          

#if defined(_OPENMP)
         PRINT *, "OPENMP=", _OPENMP
!         PRINT *, "OPENMP - NOT DEFINED"          



My Compiling command is the following one :

[schwoertzig]$ ifort -debug inline-debug-info -g -O0 -fp-model precise -openmp  -Dmkl -ftz -extend-source -DFCCI -check uninit -fpe0 -ftrapuv  -traceback -c main.F;ifort -g -O0 -fp-model precise -openmp  -Dmkl -ftz -extend-source -DFCCI -check uninit -fpe0 -ftrapuv  -traceback -debug inline-debug-info -c sigeps51.F;ifort -g -O0 -fp-model precise -openmp  -Dmkl -ftz -extend-source -DFCCI -check uninit -fpe0 -ftrapuv  -traceback -debug inline-debug-info sigeps51.o main.o -o main
[schwoertzig]$ ifort -v
ifort version 12.1.3


The debugger behavior is the following one :

[schwoertzig]$ idbc ./main
breIntel(R) Debugger for applications running on Intel(R) 64, Version 12.1, Build [77.329.14]
object file name: ./main
aReading symbols from /home/schwoertzig/tmp/tests_fortran/bug_OMP/main...done.
(idb) break sigeps51.F:11
Breakpoint 1 (sigeps51.F:11) pending
(idb) break sigeps51  
Assertion failed: "!"should not be possible"" ./src/st/mem/semanticevents.C:435
This is an unexpected condition and may indicate the presence of a defect.
If you wish to report this, please include the stack trace that follows.
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN15IDBAssertFailed3runEPKcS1_j+0xe) [0x60dca2]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN14DTLU_namespace12assertFailedEPKcS1_j+0x1e) [0xb51f8e]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb() [0x8e3e3d]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_Z17nextSemanticEventP14ReadableSymtab13RelocatedAddr+0x2c9) [0x8e4b6f]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN38PositionWithRespectToProlog_Calculator9calculateEP9AddrSpaceRK13RelocatedAddrbmbmRP14SymbolEntrydefR27PositionWithRespectToPrologRS2_SA_SA_SA_+0x80f) [0x8f9d03]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN22ProcedureDescriptorDef27positionWithRespectToPrologEP9AddrSpaceRK13RelocatedAddrRP14SymbolEntrydefR27PositionWithRespectToPrologRS2_SA_SA_SA_+0xa5) [0x8fa507]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN14IfOccursInExpr9getRAListEP6VectorI13RelocatedAddrE+0x106f) [0x648a53]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN21AbstractConditionExpr9getRAListEP6VectorI13RelocatedAddrE+0x2a) [0x9433c2]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN19AbstractConditionPC7realizeEv+0x9d) [0x942ad9]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN20AbstractHandlerTable6createElbP17AbstractConditionP3CmdbbN7Handler15HandlerDurationES3_bP9ConditionNS4_18UserConditionStyleElNS4_14PassCountStyleE+0x473) [0x94a1cd]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN11CmdGdbBreak5do_itER19CmdExecutionContextRN10BaseForCmd9CmdResultE+0x274) [0x64e432]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_ZN10BaseForCmd7executeEb+0xfc5) [0x6598e5]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb() [0x60b74c]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_Z15ProcessCommandsv+0x54) [0x60ba5c]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(__gxx_personality_v0+0x41d) [0x6099e5]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(_Z7idbMainiPPKcS1_+0x180) [0x609c76]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(main+0x3a) [0x60998a]
/lib64/ [0x3d2de1ed1d]
/opt/intel/composer_xe_2011_sp1.9.293/bin/intel64/iidb(__gxx_personality_v0+0x232) [0x6097fa]



Downloadapplication/octet-stream source_code.tar.gz748 bytes

Here is the simpliest reproducer I found. For debuggers (idbc/gdb) line 6 does not exist. It should be "A = A + B". But Replacing include statments by the corresponding directive seems to work. How to get rid of this issue ? Does anyone know a workaround ?



      INTEGER :: A, B
      A  = 0
      B  = 0     
#include ""  !contains only single line !$OMP ATOMIC
          A = A + B
#include "" !contains only single line !$OMP END ATOMIC 

[schwoertzig]$ ifort -debug inline-debug-info -g -O0 -openmp  main.F
[schwoertzig]$ idbc ./a.out
Intel(R) Debugger for applications running on Intel(R) 64, Version 12.1, Build [77.329.14]
object file name: ./a.out
Reading symbols from /home/schwoertzig/tmp/tests_fortran/bug_OMP/source_code/a.out...done.
(idb) break main.F:6
Breakpoint 1 (main.F:6) pending


Leave a Comment

Please sign in to add a comment. Not a member? Join today