internal threshold was exceeded

 

Reference Number : Q511410 / Q459928


Version : Intel C++ and Fortran Compiler  versions 11 and later


Operating System : Windows, Linux, Mac OS X
 
Architectures:   IA-32, Intel 64


Problem Description : The compiler contains certain internal limits intended to prevent excessive memory usage or compile times for very large compilation units. When such a limit is exceeded, some optimizations are skipped to reduce the memory footprint and compile time. For example, loops preceded by OpenMP directives may not be threaded. Optimizer messages may not be seen for such loops, even if the optimization report is enabled. Some compilers may emit a remark such as "An internal threshold was exceeded".


Resolution Status : From the 11.0 compiler, the switch /Qoverride-limits (Windows*) or -override-limits (Linux* or Mac OS* X) may be used to disable the internal limits and continue optimizing (-qoverride-limits for version 15.0 and later on Linux or Mac OS X). This is likely to increase memory usage and compile time; in some cases, especially on 32 bit operating systems, there is a risk that the compilation may fail due to insufficient memory. 
Some debugging features cause additional code to be generated and may increase the likelihood that internal limits are exceeded. If a "threshold exceeded" message is encountered, consider temporarily disabling such features as array bounds checking. (Note that bounds checking is enabled by default for debug builds within Intel Visual Fortran).
In some cases, reducing the optimization level for the affected function may help. For very large, complex functions, it may be preferable to split them into smaller components.

For diagnostic messages emitted when compiler internal limits are exceeded in the version 15 and later compilers, and for actions that may be taken, see the article at  https://software.intel.com/en-us/articles/fdiag25464 

[DISCLAIMER: The information on this web site is intended for hardware system manufacturers and software developers. Intel does not warrant the accuracy, completeness or utility of any information on this site. Intel may make changes to the information or the site at any time without notice. Intel makes no commitment to update the information at this site. ALL INFORMATION PROVIDED ON THIS WEBSITE IS PROVIDED "as is" without any express, implied, or statutory warranty of any kind including but not limited to warranties of merchantability, non-infringement of intellectual property, or fitness for any particular purpose. Independent companies manufacture the third-party products that are mentioned on this site. Intel is not responsible for the quality or performance of third-party products and makes no representation or warranty regarding such products. The third-party supplier remains solely responsible for the design, manufacture, sale and functionality of its products. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. *Other names and brands may be claimed as the property of others.]
For more complete information about compiler optimizations, see our Optimization Notice.

6 comments

Top
anonymous's picture

sir. what should I do if the internal disk of my laptop is at risk?? help me pls

Martyn Corden (Intel)'s picture

/Qoverride-limits (-override-limits) is present in the 11.1 compiler, but it is not documented in the main compiler documentation, since it is a high risk switch that may cause your compilation to fail or to be very slow. If it wasn't present at all, you would get a warning from the driver about an unrecognized switch.
It is more likely to be useful on a 64 system with plenty of memory than on a 32 bit system or one with limited memory.

There are a few limits in the compiler that are not hooked up to this switch, see my previous comment. You should normally only see the message for very complex and/or extremely large codes. If your code doesn't seem to meet that description, and you are able to provide an example, you might like to submit a support issue, either to Intel Premier Support, or to the compiler user forum. Note that the complexity of the generated code is sometimes not immediately apparent from reading the local source. If your code is in the "very complex" category, you might also like to try building at a lower optimization level, e.g. at /O2 instead of /O3, or with interprocural optimization (/Qipo) disabled.

In general, if you build with /Qoverride-limits, you should measure whether it actually makes a significant difference to performance, and not use it unless it does.

servil's picture

Looks like /Qoverride-limits is not present, using ICL 11.1 EM64T. Get this warning eve if compiling with /Qoverride-limits

invoking icl /? lists nothing like /Qoverride-limits

anonymous's picture

I have exactly the same "internal threshold" message even if -override-limits is adopted.. I'm running CentOS 5.4 32-bit, 2GB RAM, enough disk space, using IA-32 Compiler Professional Edition for Linux and trying to compile PHP 5.3. Is there any chance to resolve that?

Martyn Corden (Intel)'s picture

The compiler does not itself have a high memory threshold, that is imposed by the operating system.
If you are running on IA-32 or using the compiler for 32 bit apps, there are various thresholds of about 2 GB that you can't work around.
If you are running on Intel64 or IA-64, the compiler (which allocates memory dynamically) can use as much real and virtual memory as is available on the system (though on Windows, there are still limits on the sizes of generated object files and executables). However, the compilation of very large, complex programs can sometimes become very slow, especially if there is heavy paging of virtual memory. The compiler has certain internal limits in order to prevent excessive memory usage and/or extremely long compile times. Examples are a maximum number of statements (in the intermediate code representation), limits related to branches, numbers of temporaries, disambiguation, etc. /Qoverride-ride limts should override most of these, but there may be some that are not overridden (for example, if they protect against going outside array bounds).

If you are able to provide in a Premier Support issue an example of a source file that still gives these warning messages when /Qoverride-limits is specified, I'll be happy to investigate. Please quote this article.

yanqing-wang's picture

I had met this trouble even if /Qoverride-limits was adopted. After checking all available physical memory when 'interval threshold' was popped up, I found that physical memory was enough(about 2GB was free in system).
1. Are there other compiler's options in connection with this issue?
2. Can I force compiler to constantly allocate system memory in any cases? My machine has enough system memory to do that.
3. Can I manually configure the high memory threshold of compiler to overcome this issue?

thanks!

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.