TBB For MinGW

TBB For MinGW

This problem seems to have been asked before without a satisfactory solution. There is no way of building for windows and mingw currently distributed with TBB.

I've tried to make my own inc files to get it working but I am, how you say, bad at it.

I have an angle on how it could be done, but I'd appreciate it if a more skilled person either helped or did it himself. If I knew what TBB required of the compiler and how to satisfy those needs via GCC, this would be trvial. Alas, the inc files are a little opaque to my amateur eyes.

I know of one issue, that is that the windows inc file called detect.js. The thing is, this script gets information like the runtime version by calling the VC++ compiler. Not good. I'm not sure how to get such info for MinGW and the linux gcc inc isn't very helpful.

Basically the inc files have to be spliced together; part linux gcc, part windows cl. I don't know how to do it.

I do know that mingw uses the same runtime as VC++, so it should be possible with a little work.

"Halp."

31 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Please consider starting with the "Additions to atomic" patch, which is for a large part concerned with porting.

This is not the first discussion we've had here regarding minGW. Did you know about this apparently successful attempt? It seems that what they did was pretty rudimentary and certainly less than a port, but maybe it would work for you?

In it's current form, it's impossible to build TBB natively with anything else than MSVC or Intel compiler on windows. And it's FAR from a build system issue. (Cygwin is another story, it's not native).

Even if you could write the necessary detection stuff (which you can actually skip by explicitly setting the necessary arch and compiler env vars on the command line), and the relevant .inc files, the build is still going to fail when compiling the sources. If you just take a quick look at the sources, there are #ifdefs and the like all over the place, some of which test for the platform (WIN32, __GNUC__, MSC_VER, etc.), and currently, all these tests assume, that "windows = msvc or intel". The compiler (intel, gcc, msvc, digital mars, watcom etc.) and platform (windows, linux, mac, etc.) specific settings should have been decoupled from the start. But for windows, it currently implicitly assumes msvc or intel.

And even if the build succeeds with mingw (that's a whole lot of work), there is still the question whether it actually works...

To be able to fully support MINGW, someone who is already familiar with the internal needs and workings of the library should help. Otherwise, we won't see any mingw support for at least a few years.

--
Greets,
B.

I'm trying stuff, and report if I succeed.

--
Greets,
B.

You can lead a horse to water, but you can't make it drink...

Well, after modifying some of the sources and build files, I was able to build TBB with MinGW (gcc 4.2.1-sjlj precisely), and the unit tests as well. Some of them work, some seem to livelock, some crash.

For now, I won't get to look what might be the problem, perhaps sometime next week.

Here's a test run log (though I suspect it would be different on a second run):
http://digitus.itk.ppke.hu/~oroba/tbb/tests.log

--
Greets,
B.

After a bit more testing, it seems that everything works, except exception handling.
I don't know if it has anything to do with SEH, because MinGW doesn't support it (the relevant keywords __try, etc. are a noop -- #defined to be empty).

I don't yet know how exceptions are dealt with in a MT environment.

For a start, can someone explain what exactly is the purpose of the __try and __finally block in task.cpp around line 2673? Does it have anything to do with C++ exceptions or is it only for SEH (or both)?

Thanks in advance.

--
Greets,
B.

Dear B., thanks for exploring the possibility of MinGW port. I am glad you have made a good progress there. We have identified the problem with MinGW being use of _WIN32 to process compiler-specific portions of code, but have not yet put resources to improve it. If you contribute the patch, we will be glad to merge it back to TBB.

The problem, however, is how to ensure that any subsequent changes do not break it again, having inmind that we do not officially support MinGW and do not test with it. So it would be good if youor someone else provides some advice to follow in order to keep MinGW support healthy. Even better if someone could keep an eye on TBB updates and make sure MinGW support is not broken, once implemented.

As for the question: the try-finally block is there to catch any exception (as C++ exceptions are based on SEH in MS implementation). It is necessary to perform proper cleanup in case an exception is thrown in a worker thread and not intercepted.

Hello,

I would be very interested if you could tell or send me how you managed to compile with mingw32-gcc.exe.
You can contact me at spatlabor at gmail.com

If you prefer sending me the .a and .dll would be fine also.

Thanks

What I have actually done, is that I took the low-level assembly implementations from linux_ia32.h, and put it inside windows_ia32.h with a #ifdef __MINGW32__ guard (well, more-or-less). Other modifications were necessary in other files, but basically, that's the core of it.
This is of course not the best solution, as it is redundant. Whatever modification is made on the linux low-lever routines, also have to be made on the windows MINGW routines, as they are the same. Perhaps some common file, something like "gcc_ia32.h" would be better, which contains common GCC stuff, which is included by both windows_ia32.h and linux_ia32.h. The same goes for other architectures.

I'll pack up a zip with the modified sources and description soon, so others can also try it, and someone with better TBB understanding can begin to ponder on how to integrate it into official TBB (currently my solution feels a bit hackish).

Stay tuned :)

EDIT: Almost forgot: How could that __try/__finally block be fixed for MinGW in task.cpp? I'm sure that's the source of some of the crashes. How is it done on linux? What happens exactly if one of the threads throw an exception (linux/windows)? Does it propagate "out" from the thread?

--
Greetings,
Balazs

MADakukanov:

Dear B., thanks for exploring the possibility of MinGW port. I am glad you have made a good progress there. We have identified the problem with MinGW being use of _WIN32 to process compiler-specific portions of code, but have not yet put resources to improve it. If you contribute the patch, we will be glad to merge it back to TBB.

This is something I honestly don't understand. MinGW is GCC for Windows, that is, the most popular open source compiler for the most popular platform. And Intel treats it just as some kind of toy compiler, not worth the time or the work to support it. Intel, the biggest computing company, worth I don't know how many billion dollars, fails to recognize priorities and allocate resources to support what users keep asking. How long is it going to take before you finally decide that maybe MinGW is something Intel needs to support, instead of asking users with no clue of the TBB internals to port it and keep supporting it for you? There are things that users/volunteers can contribute, but major support should come from the main developers.

Look at Trolltech/Qt and their support for compilers and platforms. Smaller company than Intel for sure, with maybe more developers than TBB? OK, but Qt is a much much (much) larger project than TBB, with all the due respect, so things should even out on that, leaving the MinGW support problem on Intel/TBB's shoulders.

Pasquale

If so many people use MinGW, maybe one of them can base a port on the latest "Additions to atomic" patch (which is largely concerned with porting, not just with tbb::atomic), currently version 2008-09-15? Last intervention here, I promise, but the earlier suggestion of gcc_ia32.h looks eerily like gcc_x86.h.

balage:How could that __try/__finally block be fixed for MinGW in task.cpp? I'm sure that's the source of some of the crashes.

I discussed it with Arch Robison and we agreed the best thing is to change it over to RAII instead of __try/__finally. I.e. create a local object which destructor does the necessary cleanup.

How is it done on linux? What happens exactly if one of the threads throw an exception (linux/windows)? Does it propagate "out" from the thread?

On Linux (or more correctly, with POSIX threads) you could specify a cleanup handler with pthread_cleanup_{push,pop}.

Exception handling changed a lot in TBB 2.1, while the code in question dates probably back to 1.0. To keep it short, now we try to intercept exceptions in tasks executed by a worker thread, andpropagate those to the master thread that called a TBB algorithm. For safety reasons, however, I would stillkeep that old cleanup code.

First of all, sorry for the long delay, this week has been exceedingly heavy.

To tricaric:
Don't be too surprised. Most bigger projects received MinGW support only later, I'm talking about wxWidgets, the OGRE 3D engine and lots of others, but eventually, all of them got pretty good MinGW support, so I'm confident TBB will have excellent MinGW support sooner or later, just stay tuned :)

As to the progress:
At latest tomorrow I'll pack the things together and upload somewhere with instructions.

As for the exception handling:
The problem is, MinGW doesn't have SEH at all (__try/__finally is noop), and because it's not posix, there are no cleanup handlers either. What could be the solution? What's exactly the RAII solution? Does that mean, that the exception is simply not catched at all?

--
Greetings,
Balzs

I'm glad somebody has decided to look into this seriously. Much appreciation for you!

Now, MinGW doesn't support SEH, but couldn't you use it via inlined ASM? It's just a thought. I'm going to do some research and see if such a solution has been successfully implemented in other projects.

balage:As for the exception handling:
The problem is, MinGW doesn't have SEH at all (__try/__finally is noop), and because it's not posix, there are no cleanup handlers either. What could be the solution? What's exactly the RAII solution? Does that mean, that the exception is simply not catched at all?

In simple, RAII is the C++ technique to deal with the kind of problems that __finally is intended to help, i.e. to execute some code independently on whether an exception was caught or not. Read the Wikipedia article for RAIIto get acquainted with it. The idea is simple: put cleanup code into a destructor of an object in the scope of interest, and this destructor will be called. In the particular case, __try/__finally should be removed, a special class (probably local tothe worker_routine function) should be written in a way that its destructors does all required cleanup, and an object of this class should be created in the scope where __try/__finally had resided.

You might as well leave this exercise to us in the TBB team if you do not feel confident about what should be done. If I have little more time to implement it later I can post it here.

I'm familiar with RAII, it's just that I forgot, that there is no catch now either, so the exception is not catched now either :)
I'll try this then, and see if it works...

--
Greets,
Balzs

Okay, sorry for the delay again.
As promised, here's the patch, attached to this post.
To build using MinGW:

  • First rename (or copy) mingw32-g++.exe and mingw32-gcc.exe in the in directory to g++.exe and gcc.exe respectively.
  • Copy over the attached archive's contents to the TBB source tree (tbb21_20080605oss) overwriting files.
  • Launch a cmd prompt in the TBB root, then type:

set tbb_os=windows
set arch=ia32
set compiler=gcc
set runtime=unknown
mingw32-make

That will build the libs, to build and run the tests, type:
mingw32-make test
Of course, some tests will live-lock, others simply crash. The source of problems is centered around exception handling issues I believe.

Other issues I encountered so far:

  • Some tests use __declspec(thread) in WIN mode for TLS. MinGW doesn't support this, yet only issues a warning, hence the resulting program will be incorrect. I modified one of these (namely test_task_scheduler_observer.cpp) to use the WinAPI Tls*() functions.
  • _M_IX86 (and others similar) are defined by the compiler under windows by MSVC and Intel, but NOT by MinGW. However, the MinGW winapi headers DO define these. The problems occur, when sometimes the windows.h header gets included before _M_IX86 is used, so sometimes it works, but when windows.h is not included, compilation fails. I added __i386__ next to where _M_IX86 is used to correct this.
  • __m128 needs some special alignment, and GCC (or just MinGW, I don't know) does alignment differently. I didn't fully research the subject, but if you do a google on "__m128 mingw align" you'll see what I mean. Also, here's a short discussion: http://readlist.com/lists/lists.sourceforge.net/mingw-users/0/2675.html

Well, that's it so far. Whoever can, experiment with this stuff.

I used gcc 4.2.1-sjlj for compilation.

--
Greets,
Balzs

Attachments: 

Thanks Balzs for the great job done. Could you please also officially contribute your patch to the project? For that, open http://www.threadingbuildingblocks.org/contribution_first.phpand follow the instructions. If the official contribution is not made, we can not incorporate the patch to the main TBBtrunk.

What I plan to do, is to get the latest development version, and see if I can do the necessary modifications.

Anyway, the "patch" can be downloaded from here (I cannot see my attachment to my prev post - do they work?):

http://digitus.itk.ppke.hu/~oroba/tbb/tbb21_20080605oss_mingw_patch_1.zip

Again, I have to stress, that my solution is a bit hackish, and also requires a lot of testing. So please, whoever has MinGW and some time, experiment with this stuff. Or just report, I'm curious if anyone found this thing useful.

--

Greets,

Balzs

Quoting - balage
What I plan to do, is to get the latest development version, and see if I can do the necessary modifications.

Anyway, the "patch" can be downloaded from here (I cannot see my attachment to my prev post - do they work?):

http://digitus.itk.ppke.hu/~oroba/tbb/tbb21_20080605oss_mingw_patch_1.zip

Balzs, it would be great if you could contribute your patch via the official TBB contribution page: http://www.threadingbuildingblocks.org/contribution_first.php. Without that, we couldnot use your work to improve MinGW support in TBB. Thank you!

Quoting - Alexey Kukanov (Intel)

Balzs, it would be great if you could contribute your patch via the official TBB contribution page: http://www.threadingbuildingblocks.org/contribution_first.php. Without that, we couldnot use your work to improve MinGW support in TBB. Thank you!

Whatever happened to the MinGW patch? Did it get submitted and then included into mainline TBB?

Pasquale

No it did not get submitted :(

Quoting - tricaric

Whatever happened to the MinGW patch? Did it get submitted and then included into mainline TBB?

Pasquale

Forgetting all about last year's promises: how far do you get with atomic.Raf_Schietekat.20081206.zip plus this last-minute replacement for its include/tbb/atomic.h?

Raf: could you please elaborate on how your atomic patch would improve MinGW support? I'm not sure I'm following that.

Alexey: that's really too bad. I've now tried to contact him directly via email (googled him), I really hope he gets my message and has a chance to submit his patch(es) officially to TBB. While we wait for that, is there any plan for the TBB team to start to work at MinGW support anyway? If not, do you think that the submission of the patch by balage will trigger dedicated MinGW support at some level?

Regards, Pasquale

Quoting - tricaric
Raf: could you please elaborate on how your atomic patch would improve MinGW support? I'm not sure I'm following that.

[...]

Support for atomics means getting involved with assembler, which is the biggest part of porting, Atomics are also by far the biggest part of assembler, so it was relatively easy to also do the non-atomic parts. The patch now supports three times as many architectures as the original: x86(-64), POWER/PowerPC, Itanium (these are now in TBB, and even there it has been confirmed that more platforms work with the patch than TBB out-of-the-box), Alpha, ARM, ESA/390-z/Architecture, MIPS, PA-RISC, SPARC (most of them still need to be validated and perhaps corrected, but they're fairly complete). Of course, that's only the architecture side of it, there's also the compiler side, but MinGW is about the GNU compiler, which is the one used for all of these platforms, exceptions notwithstanding. The third part is operating systems, and I did add a couple of those, too. Maybe MinGW needs additional work, but forgive me for thinking that this might be a good base for it, and of course I wouldn't mind adding this to its list of features.

And then, again, there was silence. Did you try or not? What were your findings?

I will have to try to find a solution in about a couple months, and I'll report then.

Given the general amount of (non)-support and (non)-interest, I don't plan on spending much time on it. Even if I have invested already months on writing TBB-based parallel code, results so far are less than exciting, and if all stays like that, I'm sooo out of here. Strip away all TBB nonsense and forget about the multicore (illusion)-(delusion)-revolution. Maybe Intel would have been better off staying an hardware-only company... Just release processors, and let serious developers write libraries...

P.

"I will have to try to find a solution in about a couple months, and I'll report then." See #5.

So, still no MinGW support for TBB =(

Leave a Comment

Please sign in to add a comment. Not a member? Join today