idb error: system open-file limit

idb error: system open-file limit

Hi-

I'm trying to use idb with ifort v11 on Mac OS X on a fairly large code. The executable is producing an error not seen on other platforms, so I wanted to look at it with idb. I get the following error stream.

prompt> idb RadHyd
Intel Debugger for applications running on Intel 64, Version 11.0, Build [1.1510.2.69]
------------------
object file name: RadHyd
Reading symbols from /Users/Shared/Work/Codes/CHIMERA/trunk/RadHyd/Execute/build1D/RadHyd...
The system open-file limit has been reached.
So the debugger cannot open files it needs to read debug information.
The debugger's understanding of the program will be limited.

Unrecognized binary file format for file /Users/Shared/Work/Codes/CHIMERA/trunk/RadHyd/Execute/build1D/unpack_eos_keys.o
Could not read debugging information from '/Users/Shared/Work/Codes/CHIMERA/trunk/RadHyd/Execute/build1D/unpack_eos_keys.o'.

... and it continues for several pages of files.

Any way around the "system open-file limit"?

Raph

16 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Quoting - Kevin Davis (Intel)

Unclear if this is a shell imposed limit or not. The shell imposed open files limit defaults to 256 on my system.

What happens if you increase the open files limit under the shell and then start idb ?

For bash/ksh use: ulimit -n and for csh/tcsh use: limit descriptors

Choose to reflect the number of object files you expect will be opened.

Thanks! Didn't think about shell limitations. Setting that to 1024 solved the open-file limit problem, but revealed another, which I saw previously, butascribedto the open-file limit issue.

prompt> idb RadHyd
Intel Debugger for applications running on Intel 64, Version 11.0, Build [1.1510.2.69]
------------------
object file name: RadHyd
Reading symbols from /Users/Shared/Work/Codes/CHIMERA/trunk/RadHyd/Execute/build1D/RadHyd...Assertion failed: "found" src/st/mem/pil_loadablefileformatMachO.C:1780
This is an unexpected condition and may indicate the presence of a defect.
If you wish to report this, please include the stack trace that follows.
Could not start process for RadHyd
No image loaded ... Recovering ...

I don't think "loadablefileformatMachO" is one of mine.

Raph

Unclear if this is a shell imposed limit or not. The shell imposed open files limit defaults to 256 on my system.

What happens if you increase the open files limit under the shell and then start idb ?

For bash/ksh use: ulimit -n and for csh/tcsh use: limit descriptors

Choose to reflect the number of object files you expect will be opened.

Agreed, src/st/mem/pil_loadablefileformatMachO.C:1780 is an IDB related file/location of where the assert occurred.

I do not know what is causing this nor do I have any prescriptions on how to avoid it. I will inquire with the IDB developers to see if they can discern the cause and offer any work around.

The IDB Development team fix this assert and others in the latest IDB shipped with the 11.0.059 update (package id: m_cprof_p_11.0.059.dmg).

Can you please try installing the 11.0.059 update package and try the newer idb?

The newer IDB is version "1.1510.2.96", the full sign-on is:

Intel Debugger for applications running on Intel 64, Version 11.0, Build [1.1510.2.96]

You currently have "1.1510.2.69"

Thanks Kevin, this gives me a reason to expedite getting my support renewal processed.

Raph

Quoting - Kevin Davis (Intel)

The IDB Development team fix this assert and others in the latest IDB shipped with the 11.0.059 update (package id: m_cprof_p_11.0.059.dmg).

Can you please try installing the 11.0.059 update package and try the newer idb?

The newer IDB is version "1.1510.2.96", the full sign-on is:

Intel Debugger for applications running on Intel 64, Version 11.0, Build [1.1510.2.96]

You currently have "1.1510.2.69"

I just learned about an additional side effect related to this assert that is not fully addressed in the newer IDB.

From the IDB developer's investigation, we understand this assert relates to the linker not issuing symbolic definitions for all local variables in the executable's debug information. The changes in the newer IDB avoids the assert but you still may experience issues debugging, specifically involving access of some local variables.

From instances we have investigated, this specific assert involved Fortran local arrays that are allocated in the _bss segment. If you obtain the newer IDB and experience problems with local arrays not being accessible/visible, undefined, or they contain incorrect values, then try making the local array global via a COMMON as a work around.

I will keep this thread updated on the status of the additional debugging issue as I learn it.

(Internal tracking ids: DPD200111050, DPD200136264)

IDB Version 11.1, Build [1.2097.2.222] included in the latest 11.1.046 update contains a fix for additional related asserts from tracking id DPD200111050.

The variable access/visibility issue tracked under id DPD200136264 continues to be worked.

Hello Kevin and Raph;

I wonder if it is possible to restart a discussion that has been dormant for some months. Here's hoping.

I get the same problem that Raph first mentioned, namely the "open file-limit" problem. I too get rid of this problem by typing "limit descriptors 1024" at mny console prompt (I am running MacOS X 10.5.8).

That got rid of the open-file limit but, like Raph, uncovered the same problem he cited, namely

Reading symbols from /Users/dclarke/nmacdonald/test/debug2/xdzeus36...Assertion failed: "!" - SCATTERED!"" src/st/mem/pil_loadablefileformatMachO.C:1596
This is an unexpected condition and may indicate the presence of a defect.
If you wish to report this, please include the stack trace that follows.
Could not start process for xdzeus36
No image loaded ... Recovering ...

and the solution for Raph was to upgrade his IDB from1.1510.2.69 to1.1510.2.96. Well, alas, I have the latter, and I still get this assertion failure.

The last I saw of this stream was that with the fix to Raph's problem (but doesn't fix mine), there were still issues with accessing arrays accurately, etc., and that we were to stay tuned. So...

1. How do I fix the assertion failure in version1.1510.2.96, and

2. What is the update on the array issues?

Thanks.

Cheers, David.

The asserts happen at a different line numbers within pil_loadablefileformatMachO.C, (e.g. 1780 vs. 1596), so they may have some relationship but do not appear to be the same.

For item 1:

As I noted on 7/22/09, there were additional asserts fixed in the later IDB Version 11.1, Build [1.2097.2.222] so its likely yours if fixed in the newer IDB. I say that with some confidence because the details of the internal tracking id DPD200111050 include the assert you showed.

For item 2:

The array issue is not yet resolved so the only advice is to try the work around mentioned of making the local array global via COMMON.

I will add that development has worked extensively with Apple to root-cause the issue with relocation addresses related to the BSS segment. Currently, there is compiler debug emission changes needed, after which debugger changes may be necessary. This turned out to be a complex issue. I'll keep the thread updated as I learn more.

Thanks Kevin.

Sorry for asking a very basic question, but how do I upgrade toIDB Version 11.1, Build [1.2097.2.222] (and presumably ifort as well)? Is this something I have to pay for, or are fixes such as these given to anyone with a "current" version, whatever that may mean.

Thanks.

regards, David.

No apology necessary.

The IDB version tracks the compiler release version. IDB 11.1 (and updates) is included in 11.1 Compiler Pro release packages.

The Support Subscription included in the compiler purchase entitles the user to 1-year of free upgrades to the product. So, they're given to anyone with a non-expired support subscription.

You may know to obtain product updates from the Intel Registration Center. Itlimits access to downloads based on the expiration of your support subscription.

The current 11.1 Compiler Pro release for Mac OS is 11.1 update 4 (11.1.080) which contains IDB version:

Intel Debugger for applications running on Intel 64, Version 11.1, Build [1.2097.2.319]

Let us know if you have other questions.

Kevin (or whoever knows):

I have sort of answered my last question myself. My technician forwarded me an e-mail from INTEL sent last December informing me (through him) of an upgrade to 11.1. He gets hundreds of these messages all the time, and rarely forwards them unless someone asks for them in particular, like I just did now for ifort. It would be good to get me as the person such upgrade notices are sent to, not him.

So, I download this 456 MByte file (all I want is an update, not the whole thing again!), and after about 15 minutes, up comes the installer. After I figure out how to link it to my license, it and I are stuck on dealing with XCode.

I don't want XCode. XCode is a piece of junk, in my view. I just want ifort and idb to work at the terminal prompt. Yet it seems to think it has to link up with XCode, and I cannot for the life of me find what directory to link it with. My finder is of no help here; it gives me something to click on to fire up XCode (which works), but it doesn't tell me where it is so I can tell the ifort installer. So, lots of questions:

1. Do I really need all 456 MBytes (no doubt compressed, so it's gonna blow up once the installation begins) just to get an upgrade? What will happen to all the files it replaces or obviates. Do they just stay put clogging up my disk, or is the installer smart enough to clean up after itself, and remove all the stuff I no longer need? If it isn't, how do I effect this purge?

2. How do I get past this Xcode hurdle? And are there more hurdles along the way?

3. Why aren't these things straight forward? On the site that was e-mailed to me, there should have been a button for upgrading, that doesn't need me to go through all this again. I will *always* forget these install details as I only have to do this every 6-12 months. Thus, relying on my memory of how these things work will never be successful, thus the need for an "upgrade" button that simply assumes there will be an existing license, finds it, then just updates the files needing updating without having to download Gbytes of unwanted data. Or did I miss the magic "upgrade only button" somewhere?

Thanks.

regards, David.

Thread is getting off topic. You might consider starting new forum threads with related titles. You might be asking questions of interest to others who may miss them given the unique title of this thread.

Hopefully I covered all the questions/comments.

Update notices are emailed to the address provided when the serial number is registered and only to users with current support licenses. Various notifications also appear in this Forum when new updates are available. Sounds like your tech submitted his email address when registering the S/N. We can change the email address of your Mac OS Fortran subscription to yours if you wish. Please submit a request for this in our Intel Software Development Products Download, Registration & Licensing forum.

You shouldn't have to locate your license with each install. The installer locates and offers a default choice of using the existing activation unless what it finds is expired. The installation needs to know the location of Xcode because of the dependency on the associated GNU binutils that ship as a part of Xcode, and also to install the Intel Xcode integration components (whether Xcode is used or not). You can still use the compiler via the command-line but you cannot avoid needing to have Xcode installed or the need for the installation knowing where Xcode is. The default installation location for Xcode is /Developer.

1. Currently, our updates are complete releases, so yes you need the whole file. This could change in the future. Our packages do not install over an existing installation but instead install side-by-side under /opt/intel/Compiler/11.1/. It is true on the Mac only that common links /usr/bin/ifort and /usr/bin/idb are updated to refer to the last installed compiler/debugger. You must manually run uninstall_cprof.sh (under the specific folder) to uninstall versions you no longer want.

2. Touched on this above. All I can recommend is installing Xcode in the default location (/Developer) which our installer provides as the default selection, wo when prompted during the install, know that you can always use the default.

3. You didn't miss any magic upgrade button. We're continually trying to make improvements in the installation/upgrade/download areas. There plans to improve and make easier "checking for updates" in future versions. There also ideas about offering a "caf-style" download/upgrade and just breaking up packages. All I can say is stay tuned and if you have specific feedback then by all means please share that with us in our Intel Software Development Products Download, Registration & Licensing forum. Hopefully some of the changes planned will help make upgrading easier.

Kevin (or whoever knows):

I have sort of answered my last question myself. My technician forwarded me an e-mail from INTEL sent last December informing me (through him) of an upgrade to 11.1. He gets hundreds of these messages all the time, and rarely forwards them unless someone asks for them in particular, like I just did now for ifort. It would be good to get me as the person such upgrade notices are sent to, not him.

So, I download this 456 MByte file (all I want is an update, not the whole thing again!), and after about 15 minutes, up comes the installer. After I figure out how to link it to my license, it and I are stuck on dealing with XCode.

I don't want XCode. XCode is a piece of junk, in my view. I just want ifort and idb to work at the terminal prompt. Yet it seems to think it has to link up with XCode, and I cannot for the life of me find what directory to link it with. My finder is of no help here; it gives me something to click on to fire up XCode (which works), but it doesn't tell me where it is so I can tell the ifort installer. So, lots of questions:

1. Do I really need all 456 MBytes (no doubt compressed, so it's gonna blow up once the installation begins) just to get an upgrade? What will happen to all the files it replaces or obviates. Do they just stay put clogging up my disk, or is the installer smart enough to clean up after itself, and remove all the stuff I no longer need? If it isn't, how do I effect this purge?

2. How do I get past this Xcode hurdle? And are there more hurdles along the way?

3. Why aren't these things straight forward? On the site that was e-mailed to me, there should have been a button for upgrading, that doesn't need me to go through all this again. I will *always* forget these install details as I only have to do this every 6-12 months. Thus, relying on my memory of how these things work will never be successful, thus the need for an "upgrade" button that simply assumes there will be an existing license, finds it, then just updates the files needing updating without having to download Gbytes of unwanted data. Or did I miss the magic "upgrade only button" somewhere?

Thanks.

regards, David.

David,

Xcode is required for the install. The product updates are full replacements. You can uninstall the old version first if you wish.

For a complete list of the system requirements, please see the compiler release notes.

Steve - Intel Developer Support

Leave a Comment

Please sign in to add a comment. Not a member? Join today