offload error: cannot find offload entry?

offload error: cannot find offload entry?

hi

 when I try to run  a fortran progran(compiled by mpiifort with options -offload-build -openmp) on MIC,

it appears two error :

offload error: cannot find offload entry __offload_entry_aoint_2e_dscf_F90_183aoint_2e_dscf_

offload error: process on the device 1 unexpectedly exited with code 1

I  sure the program works well with openmp ,

and I test some MIC examples on my machine to make sure the MIC works.

now I cannot find the problem, can any one help me?

thank a lot.

32 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hi,

From your question, it is unclear if your OpenMP code that worked was a host only or a host+offload code.

From what you have described, I suspect your app's build procedure creates or uses static archives and one must contain your routine with the offload section; however, if the archive containing that routine was not built using xiar -qoffload-build then the required MIC specific archive is not created. While your app links, the MIC-image is missing the MIC-instance of the routine, so at execution time the host-side app reaches the offload, loads the MIC-image, but the run-time cannot find the routine because it does not exist.

You can visit this compiler reference on how to create offload libraires using xiar and xild. 

Thanks, 

Sumedh

 

hi, sumedh

my code is a host-offload code. I want to compute a loop on the MIC with openmp, the openmp code works well on the CPU, but when I add the instruction"!DIR$ offload target(mic) in(......)"  (I have simplify the instruction to prevent other problems), it turns out the error : cannot find offload entry.

And I recompile the linking library like Global Array with mpiifort,  but it didn't work.

I will try xiar and xild to create offload libraries and check whether it works.

thanks a lot.

 

 

hi  sumedh

thanks for your reply.

I am sorry that I have replied to you earlier but I submited to the furoms failed twice

 and all the feedbacks I've write to you were gone..

Now I follow your suggested, and an example I writed can run on the MIC.

 With the xiar, it created the MIC library, but my code still cannot work on the MIC,

 it turns out that "On the remote process, dlopen() failed. The error message sent back from the sink is /tmp/coi_procs/1/5598/load_lib/ifortoutUqcXvl: undefined symbol: exchange_ On the sink, dlopen() returned NULL. The result of dlerror() is "/tmp/coi_procs/1/5598/load_lib/ifortoutUqcXvl: undefined symbol: exchange_"

offload error: cannot load library to the device 0 (error code 20)"

I think that there are still some other libraries that my code needed ,

and they are not compiled nor on the MIC, I will try to find them out.

 

I have another question :can I call subroutine in an offload section, does the subroutine affect the MIC to run the program?

I am confused about this.

whatever, I have made some progress, I will update the problems.

thanks very much

wan bo

 

 

Hi, 

I am not sure what can cause that particular error. Let me look around a little more and I'll get back to you with what I find. 

In the mean time, I have some answers. Yes, you can call a subroutine from an offloaded section. However, for a subroutine to be callable from an offload region, it should be marked with attributes offload. This lets the compiler know that this function is going to be called from an offload and hence the function is compiled for both the host as well as the coprocessor. You can find more about attributes offload at this compiler reference. 

-Sumedh

First, please drop usage of -offload-build  That is not a valid compiler option.

If the offload directive added for the loop is not in code resident in a static library then using xiar/xild w/-qoffload-build is not necessary.

The undefined symbol exchange_ indicates there no instance of the routine within the offload image. That suggests the routine was not declared for offload via a !DIR$ ATTRIBUTES OFFLOAD or equivalent. Calling a subprogram in an offload section requires declaring the subprogram for offload within the calling scope and at the point of definition using the !DIR$ ATTRIBUTES OFFLOAD.

If you have not already browsed the Fortran MIC samples (/opt/intel/composer_xe_2013/Samples/en_US/Fortran/mic_samples), there are samples demonstrating openmp with offload and calling subprograms within offload code that may help with your interests.

引文:

Kevin Davis (Intel) 写道:

First, please drop usage of -offload-build  That is not a valid compiler option.

If the offload directive added for the loop is not in code resident in a static library then using xiar/xild w/-qoffload-build is not necessary.

The undefined symbol exchange_ indicates there no instance of the routine within the offload image. That suggests the routine was not declared for offload via a !DIR$ ATTRIBUTES OFFLOAD or equivalent. Calling a subprogram in an offload section requires declaring the subprogram for offload within the calling scope and at the point of definition using the !DIR$ ATTRIBUTES OFFLOAD.

If you have not already browsed the Fortran MIC samples (/opt/intel/composer_xe_2013/Samples/en_US/Fortran/mic_samples), there are samples demonstrating openmp with offload and calling subprograms within offload code that may help with your interests.

Hi kevin, I have declared the subprogramwithin the calling scope and at the point of the definition using the  !DIR$ ATTRIBUTES OFFLOAD.

Then I  declare the subprograms that I used in offload code just as follows 

interface
 !DIR$ OPTIONS /offload_attribute_target=mic
 subroutine leoF09_subprograms_sub(ival)
 integer, intent(in) :: ival
 end subroutine leoF09_subprograms_sub
 !DIR$ END OPTIONS
end interface

But when I run the code on the MIC,  it turns out the same error massage as before:

On the remote process, dlopen() failed. The error message sent back from the sink is /tmp/coi_procs/1/36074/load_lib/ifortout9qYrzO: undefined symbol: exchange_
On the sink, dlopen() returned NULL. The result of dlerror() is "/tmp/coi_procs/1/36074/load_lib/ifortout9qYrzO: undefined symbol: exchange_"
offload error: cannot load library to the device 0 (error code 20)

I can't find the symbol exchange_, I just use a subroutine exchange() to change two variables' position. 

I still don't know the library in "cannot load library to the device 0"   indicate which library.

Hi Wan, 

Each routine that you call within an offload should be declared for offload using !DIR$ ATTRIBUTES OFFLOAD. My guess is that you are calling the exchange routine in the offloaded section of the code but the exchange routine at its definition has not been declared for an offload. If you declare the exchange routine for offload, I think the error should go away. 

-Sumedh

I agree w/Sumedh. The "library" in the error is simply referring to the mic executable which is a shared library. All the symbols referenced in the mic executable (library) must be resolved when loading to the card. The case here is exchange_ is not resolved.

Hi Wan.  In a private message to me you asked about the techniques I published in another forum thread to diagnose mute failures to execute the offload code segment and indicated that you'd tried the published methods but they did not work.  You did not explain what happened.  I would recommend that you try again adding the following string to your compiles:  -offload-option,mic,compiler,"-z defs"

With that as an option in the compile, you should get all the undefined symbols remaining in your program before the offload occurs.  Then you might see any functions that have not been attributed as an offload function and follow the advice Sumedh and Kevin have given above to ensure that all the procedures and functions needed for the offload segment are generated for the offload segment.

use the following option to get the detect the unresolved symbols during link time

-offload-option,mic,ld,"-z defs"

With Fortran it is a bit easier to determine because names are not as mangled.

Wan, you should find your routine exchange requires decoration for offload as Sumedh indicated.

Robert’s form requires the option to be: “-Wl,-zdefs” (since it goes through the compiler driver). Ravi’s form directs the option to the linker (and not the compiler). Just wanting it to be clear.

thanks so much for the help from  Sumedh, Kevin, Robert-reed and Ravi, I'm too sorry that I give the feedbackso late. The MIC platform I use upgraded hardware these days.

I declare the routines, but it still comes out that "undefined symbol: exchange_", but I have rewrite the subroutine so there is no toutine of exchange. This happened every time before I delete the libpscfMIC.a. It's the offload library when I followed Sumedh's advice and used the xiar to compile and it creates this. After I deleted it, the unknown exchange_ disappeard. 

So when I  link without xiar(just  scr), it displays

aoint_2e_dscf.F90:(.text+0x64d): undefined reference to `__offload_target_acquire'

aoint_2e_dscf.F90:(.text+0x887): undefined reference to `__offload_offload'

when I link with xiar, I displays many undefined reference error that I need to add attributes offload

 

so I want to know whether I need to create the offload library?  And I find little information about  `__offload_target_acquire'  and `__offload_offload'

 wan bo

 

I am having the same error. So I used -offload-option,mic,ld,'-Wl,-zdefs'.  I see the my first undefined refernce .../fun3d-12.2-63280/Linux-mpi-mic/main.F90:1196: undefined reference to `multigrid_defs_mp_fmg_levels_request_'

I trace it down to the directory and library. I see that I built the object files with ifort (no offload in for library) and used xiar

I see that when ever I use xiar, it indicated that it is using 'ar'.

/usr/local/intel/Composer/composer_xe_2013.1.117/bin/intel64/xiar cru a1.o a2.o ... an.o

xiar: executing 'ar'

Or when I do a which xiar ...

../FUN3D_90/$which xiar
/usr/local/intel/Composer/composer_xe_2013.1.117/bin/intel64/xiar

This is expected, or is there another location for xiar for this purpose.

-dana-

To clarify, when I use xiar (or even xiar -V), the resulting output indicates "xiar: executing 'ar'".

../LibF90/$/usr/local/intel/Composer/composer_xe_2013.1.117/bin/intel64/xiar -V
xiar: executing 'ar'
GNU ar (GNU Binutils; SUSE Linux Enterprise 11) 2.20.0.20100122-0.7.9
Copyright 2009 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.

So is xiar working as expected?  Can I use a ranlib on a .a resulting from xiar?

-dana-

I see now that I have to use the -qoffload-build on th xiar command

xiar -qoffload-build cru libCapDefs.a

then I see two

xiar: executing 'ar'
xiar: executing 'ar'

and I get two libraries

-rw------- 1 dphammon g0761  362082 2013-04-01 11:43 libCapDefs.a
-rw------- 1 dphammon g0761       8 2013-04-01 11:43 libCapDefsMIC.a

Let me try throughout my libraries.

Another quick way to mark all the routines and data in your code with offload_attribute is using the command line option

-offload-attribute-target=mic

This is usefule especially if you want to build libraries code to support both host and MIC

Any further information on the missing "__offload_target_acquire"  and "__offload_offload"?

I'm not sure what information you are asking about. Could you elaborate?

The missing routines mentioned in earlier replies suggest the offload run-time library was somehow not included in the link.

hello,

my program has the similar error.The error message is:

on the remote process,dlopen() failed. The error sent back from the sink is /tmp/coi_process/1/162729/load_lib/ifortoutnLEt27:Undefined symbol:pmgrid_mp_masterproc_ .

offload error:cannot load library to device 0(error code 20)

The difference from wan.b's  error is that pmgrid_mp_masterproc_ is not a function name or a subroutine name.It is made up of a module name pmgrid and a variable masterproc declared in module pmgrid.what's more ,I also don't know what the character "mp" in pmgrid_mp_masterproc_  is.

The following is my added offload section:

!dec$ offload target(mic) in(arr)

!dec$  omp  parallel do

 do j=1,5000

      do i=5000

       arr(i)=i*3.0/x

     end do

end  do

!$omp end parallel do

 

"mp" is part of the compiler provided name decoration associated a module procedures/functions/variables.

The error suggests a reference of masterproc inside offloaded code where the variable lacks declaration for such sue although the code snippet shown doesn't have evidence of that so its hard to explain with only seeing the snippet. Were there no other edits made related to offloading?

In module pmgrid, near the declaration for masterproc, try adding:

!DIR$ ATTRIBUTES OFFLOAD : mic :: masterproc

Thank you Kevin Davis for your immediate response.I will try it you suggested

when  I add the offload option(-offload-option,mic,ld,"-z defs"),the program turn out many undefined refereence errors,such as:

radinpMIC.o:In function 'radinp_':

/home/yyz/fbh/.............../cam1/radinp.F90:(text+0x25):undefined reference to 'time_manger_mp_get_curr_calday_'

/home/yyz/fbh/.............../cam1/radinp.F90:(text+0x74):undefined reference to 'shr_orb_mod_mp_shr_orb_dec1_'

hello everyone,

         I  want to know whether the following examples have the potential for causing memory leaks.

 1.        

   integer iarray(len)         

        do i=1,len  

         !dec$ offload target(mic:0)  in(iarray)

           !$omp parallel do

                do j=1,len

                    iarray(j)=j

                 end do

           !$omp end parallel do

         end do 

 

2.

 integer,allocatable::iarray(:)

  allocate(iarray(len))     

        do i=1,len  

         !dec$ offload target(mic:0)  in(iarray:length(len) alloc_if(.true.) alloc_if(.false.) )

           !$omp parallel do

                do j=1,len

                    iarray(j)=j

                 end do

           !$omp end parallel do

         end do 

Sorry, I missed your reply back in December. About that, I’m guessing the numerous errors may relate to the target-side link not including the object files (.o) for routines declared in modules. Let us know if you have not resolved that issue to date.

I do not completely follow your latest question/issue related to memory leaks. What sort of behavior are you experiencing to suspect this?

I think it was just a typo and meant to be free_if, but the compiler will reject double “alloc_if” in the allocatable example.

It would really help us understand and help us offer other ideas if you can provide a complete reproducer.

Hello,

        I have solved the question asked on December .Actually,The question I asked last time exactly have a typo. The book  Intel Xeon Phi Coprocessor High Performance Programming  written by  Jeffers and Jim evokes the idea whether the following examples have the potential for causing memory leaks. What’s more,I know the mic card has more than 6G memory,however, the variables and arrays on the in/out/inout/nocopy of target clause I written is much less than 6G,but I often face the problems that segmentation faults and the mic memory is not enough。so,I suspect whether there is memory leak when I use alloc_if and free_if.

The book said:

Pointers  to static data on processor are special-cased. The alloc_if and free_if modifiers are

ignored when the following are both  true:

1.    The processor address used during creation of a target memory association points to statically allocated data.

2.    The variable is also available in the target binary because it has __declspec(target(mic)) or ATTRIBUTES OFFLOAD:mic

The statically allocated memory on the coprocessor is used as the destination of the transfer.

This target memory is not dynamically allocated and never freed.

     There is only one block of target memory associated with a processor address. It is an error to call alloc_if(1)[Fortran:alloc_if(.true.)] to create a second association for a processor address before freeing the existing one . The new association overwrites the earlier one,which has the potential for causing memory leaks on the target.

     It is an error to call free_if(1)[Fortran:free_if(.true.)] for a transferred pointer if a matching association is not found . The attempted removal of an association is silently ignored.An association can be made with a processor address, and a certain length, and another association made with a different processor address within that range.Since origin addresses are different;you can use alloc_if and free_if to create distinct target allocations.

 

Example 1:        

   integer iarray(len)         

        do i=1,len  

         !dec$ offload target(mic:0)  in(iarray)

           !$omp parallel do

                do j=1,len

                    iarray(j)=j

                 end do

           !$omp end parallel do

         end do 

Example  2:

 integer,allocatable::iarray(:)

  allocate(iarray(len))     

        do i=1,len  

         !dec$ offload target(mic:0)  in(iarray:length(len) alloc_if(.true.) alloc_if(.false.) )

           !$omp parallel do

                do j=1,len

         end do 

                                                         Thank you

 

 

                    iarray(j)=j

                 end do

           !$omp end parallel do

 

 

 

hello,

    when I use a global variable defined in a head file by including the global variable in a offloaded function,I found the value is different from the value of the global variable without ofloading.How to solve the problem ,can you give me some suggestion? Thank you.

     Following is a example routine:

Attachments: 

AttachmentSize
Download example_0.docx16.19 KB

The initial value of the global sslp (assigned in example3.F90) has not been transferred to the coprocessor prior to the offload in example1.F90. Insert an offload_transfer in example3.F90 after the assignment to sslp to transfer the current value, e.g.:

!dec$ offload_transfer target(mic) in(sslp)

Be mindful that if you are running on a multi-card system then you need to offload_transfer to each card.

 hello Kevin Davis ,

               thank you for your reponse.Now I face a new problem about MPI+offload routine(a code segment of a MPI procedure is offloaded to MIC ),can you give me some information about  MPI+offload routine.thank you

That is a very different question that what this thread is about. The last question was off topic also so can I please request that you start a new thread about using MPI + offload and provide more specifics about what exactly your interests are?

I have not tried MPI + offload so I must defer to experts in our forum here who have for help answering those questions so please start a new thread and with any specific questions/interests that you have.

Thank you,

hello,

      today I face a problem of  'mpirun: No match',any suggestions will be appreciated

Attachments: 

AttachmentSize
Download trapezoid.c.docx16.26 KB

   I faced another problem is that mic0 can't be recognised by the procudure,anyone,can  you give me some suggestion? 

Attachments: 

AttachmentSize
Download test_offload.docx16.68 KB

Leave a Comment

Please sign in to add a comment. Not a member? Join today