Problem linking when offloading STL

Problem linking when offloading STL

We have some offload code that doesn't run because dlopen() fails with undefined symbols. One of the suggestions that I found was to add "-Wl,-zdefs" to the offload linker options, which might bring the undefined symbols to our attention at link time.  However, when I do this, even the most simple offload code won't link.

For example:


#include <stdio.h>

#pragma offload_attribute(push target(mic))
#include <vector>

struct Foo
{
    std::vector<int> bar;
};
#pragma offload_attribute(pop)

int main(int argc, char * argv[])
{
#pragma offload target(mic:0)
    {
        printf("Hello world");
        Foo* s = new Foo();
    }
}

I get the following output:

1>------ Build started: Project: TestOffloadLinkage (Intel C++ 14.0), Configuration: Release x64 ------
1>Build started 3/18/2014 10:54:55 AM.
1>InitializeBuildStatus:
1>  Touching "x64\Release\TestOffloadLinkage.unsuccessfulbuild".
1>MessageBuildingWithCompiler:
1>  Building with Intel(R) C++ Compiler XE 14.0
1>ClCompile:
1>  ***** ClCompile (x64 - Intel C++)
1>  Main.cpp
1>Link:
1>  C:\Users\Me\AppData\Local\Temp\ipo_7292.o: In function `std::_Bit_iterator_base::_M_bump_up()':
1>  C:\Users\Me\Documents\Sources\TestOffloadLinkage/Main.cpp:17: undefined reference to `__kmpc_global_thread_num'
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Platforms\x64\PlatformToolsets\Intel C++ Compiler XE 14.0\Microsoft.Cpp.x64.Intel C++ Compiler XE 14.0.targets(977,5): error MSB6006: "xilink.exe" exited with code 1.
1>
1>Build FAILED.
1>
1>Time Elapsed 00:00:01.87

This is really puzzling to me, as I'm not using any threads in this code. I do have /Qopenmp set, and that does seem to be important as it links fine without it. Is this a bug?

8 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

I've been unsuccessful in finding out what is the distinction between "-Wl,-zdefs" and -offload-option,mic,compiler,"-z defs" although I could guess that the former makes more sense when building with -mmic.

Setting /Qopenmp causes your code to initialize OpenMP structures even though you don't have any OpenMP directives, so the dependence on openmp library is expected, but not the failure to include it in the link step (unless you didn't use it consistently).  I haven't attempted Windows hosted MIC, but usually the option /# is used to get a verbose echo including steps used in link.

 

This is a defect with xilink. I reproduced this when building under the MSVS IDE only and I have not been able to find a work around under the IDE. I reported this to Development (see internal tracking id below) and will keep the post updated with status of a fix.

If there's any possibility that you can link outside the IDE using the icl driver then you can add the /Qoffload-option,mic,link,"-zdefs" option to help identify those unresolved externals you noted occur at run-time.

> icl /Qopenmp /GR- *.obj /Qoffload-option,mic,link,"-zdefs"
Intel(R) C++ Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 14.0.2.176 Build 20140130
Copyright (C) 1985-2014 Intel Corporation.  All rights reserved.

bar2MIC.o: In function `bar1()':
bar2.cpp:(.text+0x6): undefined reference to `bar2()'
icl: error #10104: unable to open 'bar2MIC.exe'

(Internal tracking id: DPD200254589)

Actually, toying with a bit more, I was wrong earlier. This is appears to be working as designed and the xilink failure is expected because the -zdefs disallows unresolved externals to exist in the MIC-side executable; therefore, the MIC-side link is failing which is what xilink reports.

The unresolved references are actually identified before the xilink failure. For the Release build, disable IPO and (Configuration Properties > C/C++ / Optimization [Intel C++] > Interprocedural Optimization) and it becomes obvious the unresolved references are found in the <file>MIC.o object file as noted below.

1>------ Rebuild All started: Project: ConsoleApplication2, Configuration: Release x64 ------
1>  t1.cpp
1>  x64\Release\t1MIC.o: In function `main':
1>  C:\Temp\kddavis\ConsoleApplication2/t1.cpp:17: undefined reference to `__kmpc_global_thread_num'
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Platforms\x64\PlatformToolsets\Intel C++ Compiler XE 14.0\Toolset.targets(997,5): error MSB6006: "xilink.exe" exited with code 1.
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

For the Additional Options for MIC Offload Linker, only specify: -zdefs

Something else to try is, instead of compiling with /Qopenmp use /Qopenmp_stubs  (Configuration Properties > C/C++ / Language [Intel C++] > OpenMP Support) just to help identify the unresolved reference and avoid the unresolved reference to the __kmpc_global_thread_num.

Let me know whether this helps.

Hi Kevin,

The reason I started using this option is because everything would compile fine, but when I tried running the offloaded code I would get errors like:

On the remote process, dlopen() failed. The error message sent back from the sink is /var/volatile/tmp/coi_procs/1/6745/load_lib/driverMIC.out: undefined symbol: ...

So, I am looking for a nice way to detect (and correct) this through the IDE rather than at runtime.  What I'm trying to understand is why the MIC-side link is failing with -zdefs. Removing this option, it compiles/links and runs properly, so there evidently isn't anything wrong with the build but perhaps rather a difference between the environment at build time vs. runtime.

Perhaps you can suggest a better way to detect (truly) undefined symbols in the MIC offload regions?  Thanks for your help,

And maybe it isn't 100% clear from my comments: the sample code I provided runs fine.  I encounter the dlopen() failure periodically during the development process, and would like to catch it during the build process.

I'm sorry if I created more confusion with my earlier replies.

The -zdefs option should provide what you are looking for. All unresolved references in the MIC-side image will be reported at link time and the link will fail if any exist.

When using the -zdefs (I also verified the -Wl,-zdefs form works) under the IDE in the Additional Options for MIC Offload Linker setting, xilink invokes the associated Linux linker (ld) to perform the link for the MIC-side. By design, ld's -zdefs option disallows the resulting MIC-side image from being created with unresolved references. So, it reports all unresolved references and returns an error code to xilink which in turn reports the failure under the IDE. This is the expected behavior when unresolved references exist. If none exist, the link will succeed.

I'm still trying to understand why with /Openmp the OpenMP routine is not resolved in your example but I believe you should be able to use -zdefs despite that routine being unresolved. What I was trying to say in my last reply was that if that OpenMP routine is hindering using -zdefs then if you use /Qopenmp_stubs, all Openmp references should be resolved in the MIC-side image and any unresolved references in within only your code should only be reported.

Thanks for the explanation, it is much clearer now.

Leave a Comment

Please sign in to add a comment. Not a member? Join today