fpic and mcmodel ifort 11 clarification, static libs >2GB data

fpic and mcmodel ifort 11 clarification, static libs >2GB data

I've read though other posts that seem to indicate a lot of
confusion on the use of -fpic/fPIC and mcmodel= flags especially
with static libs. I understand that -fpic is needed for shared libs.

We are using ifort 11, 64bit on CentOS, building/linking
against static libs.

Application depends on several libraries and it is not
clear to me when to use the -fpic or mcmodel flags.
(ex: =mcmodel=medium -shared_intel)
How "deep" do you have to go with this?

Originally, we used -mcmodel=medium -shared_intel on all
app and libraries but my understanding is that is only needed
on the main application for link time. If all libs are built with
-fpicthan they can be moved around appropriately in mem.

Our application does use arrays that add up to >2GB.
Some individual arrays may be >2GB. One of the libs
will manipulate these arrays and need access to them.

Application (-mcmodel=medium -shared_intel)
Depends on libs that depend on other libs.
Some libs will access those large arrays.

Example: App (-mcmodel=medium -shared_intel) uses >2GB array space
-> LibA (built with -fPIC ok?) uses >2G arrays
-> HDF (built with -fPIC)
-> szip (-fPIC needed or not?)
-> -lm (system)

Question, does LibA need to be built with mcmodel=medium -shared_intel ?

side note, with ifort 10 we used to use the -mcmodel on App, LibA, HDF but since moving to
ifort 11 we can only avoid the link time relocation errors if we use -fpic on the HDF. Did
the behaviour change?

A pointer to a any clear documentation would be very helpful.


4 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Let me see if I can make this easy.
-mcmodel=medium set the size of data pointers to 64bits but leave instruction addresses at 32bits. the default is -mcmodel=small (32bit data and address pointers). If your data will exceed 2GB you will need to set mcmodel=medium. Also, ALL objects and libraries need to have a similar setting. So other Fortran objects and libraries need to be built with this option. C/C++ should have the appropriate switches set for LP64 (or ILP64 but be careful of passing those big integers to Fortran) to be compatible with these 64bit pointersThis should be intuitive: the size of pointers needs to agree across the whole application.Now, you might think that settles it. Not quite. static data, such as data in COMMON, will be located in the TEXT data segment by the compiler and linker. On many linux distributions (but not all) TEXT is limited to 2GB even for 64bit applications (x86_64). Here is a code sample:real*8 MY_BIG_ARRAY(1000000000)COMMON / foo / MY_BIG_ARRAY

If you attempt to compile and link this, you get a link error "relocation truncated to fit: R_X86_64_PC32" because the common data plus compiled code does not fit in 2GB IRREGARDLESS of your MCMODEL setting. The TEXT segment is 2GB for this linux distribution, period.

To try to reduce the amount of data stored in TEXT, a common trick is to build with -dynamic-intel. This moves all the code from the Intel runtime libraries out of TEXT by linking against the dynamic libraries that will be loaded into a separate part of memory and out of the TEXT section. BUT you can see, this will ONLY work if your data is under 2GB and you just need a LITTLE more space. So it's not a guarantee.Another time that -dynamic-intel is used is if your code+static libraries are HUGE. There are about a dozen applications that we've seen with truely huge code sections - we're talking hundreds-of-thousands to millions of lines of code.
Another trick is to compile your code to a shared library using -fpic. Again, this is to move your code out of the TEXT section. But likewise, it's an attempt to free up more space in the 2GB text section, allowing some marginal more space for your COMMON blocks. Again, if your COMMONs add up to over 2GB this is no help.
So let's summarize:

large COMMON data will not fit if all the common data exceeds 2GB on many Linux distributions ( incidently, this holds true for Windows x64 programming model as well. Mac OS X 10.6 does not have that restriction on TEXT for x64 applications. And some Linux distros will allow larger TEXT sections. )
If you data is over 2GB, you will need -mcmodel=medium to use 64bit pointers necessary to address the larger data arrays AND you must use data NOT allocated in COMMON blocks. If you have large COMMON blocks you should recode this into MODULE DATA and remove the common blocks.
-fpic is only really needed if you want to create shared libraries. It's use to free up TEXT space is a bit of a hack and is not it's intended purpose.

Thanks for the response, appreciate you taking the time.
There still seems to be some conflicting information. From what I read above you recommend that
mcmodel=medium should be used for compiling libraries that will get linked in to
the main that also needs mcmodel=medium. But only the main
has data >2GB.

In this thread http://software.intel.com/en-us/forums/showthread.php?t=43717

Martyn Corden (Intel) states:
I think the confusion arises because libraries that have not themselves been built with -mcmodel medium, (because they do not contain > 2 GB of static data), must instead be built with -fpic so that they use full addressing and can be linked with objects that were built with -mcmodel medium. Otherwise, you get the dreaded "relocation error" from the linker.

So, do the libs need mcmodel=medium regardless of language (C and/or Fortran) or
-fpic to be relocatable?

We are still at ifort v11.1.073.
Is there any change to this behavior to be expected with XE?

Thanks for any clarification.

I tried the above sample program (using both ifort 12.1 and 11.1). When I don't use "-mcmodel=medium -shared-intel" it complains about the relocation errors as expected. However, when I only use the "-shared-intel" flag it works fine. I didn't expect that since the size of common block array MY_BIG_ARRAY is > 2gb.

Leave a Comment

Please sign in to add a comment. Not a member? Join today