Anyone who has tried to find a problem or make sense of someone else's heavily threaded parallel code knows how cumbersome it can be to identify where a run-time problem really comes from.
What compounds this problem is that the advent of more and more “multi”- and “many” core processors requires extensive use of parallel programming models to best take advantage of the benefits these systems provide to program execution flow and thus to get the best performance out of these systems. Using more and more advanced parallel programming techniques makes it however more and more difficult to keep track of all active threads and their possible data inter-dependencies.
In short the problem of identifying and fixing run-time problems in parallel code is only going to get more unpleasant.
Many of you may have already heard of the Go-Parallel Initiative or may even already have been to http://www.go-parallel.com. Others may have heard the announcement of the Intel® Parallel Studio that was made at the fall Intel Developer Forum in San Jose. This is a new product that is now available for early beta program sign-up at http://www.intel.com/go/parallel. It's intention is to provide Microsoft Windows* developers with the tools and insights they need to streamline the concurrent programming experience and reduce development and bug-fix cycles. I really recommend that you have a look, give it a try and see for yourself what it can do for you. Perhaps you can tell me what you think is the best way to enable developers to translate the immense computing power of modern multi-core processors into application performance and an improved end-user experience.
One component of the Intel® Parallel Studio is the Intel® Parallel Debugger Extension. It provides great insight into thread groups and concurrency especially if you use OpenMP*.
Additionally it can be used to detect data sharing events and in conjunction with the Intel® C++ Compiler to force code execution to be halted as soon as such a data sharing event occurs. If you have a run-time issue in a specific source code area, this can be a very powerful capability to point you to the exact location of a hidden data sharing event or a data dependency you weren't aware of.
Similarly you can monitor function reentrancy, making sure that whenever reentrancy occurs this actually is the intended runtime behavior and all your variables are handled in a thread-safe manner.
Last, but not least one of my favourite OpenMP* specific features is that you can force a parallel section to be executed in single-thread mode. Thus you can verify algorithmic code correctness before trying to identify a concurrency issue like data sharing violation.
In summary, it provides enhanced visibility of
- the data shared between multiple threads in Intel® C++ Compiler built applications,
- the vectorized data streams processed in Single Instruction Multiple Data (SIMD) registers,
- the use and dependencies of re-entrant function calls,
- Active OpenMP tasks lock information and thread team hierarchies.
If you are interested in more details there is a white paper and article on the Intel® Parallel Debugger Extension I posted at /en-us/articles/parallel-debugger-extension.
I intend to continue commenting on issues and solutions with debugging concurrent code, whether it be on Windows* or on Linux*. Hopefully I have updates on a regular basis. This blog entry is intended as a starting point for discussion. I am looking forward to your comments and suggestions. Thanks for reading and participating. If you know of a debug problem that you would like to look into or you know of a great utility that facilitates debugging and analysis of parallel code, please let me know.
As always - the opinions expressed on this site are mine alone and do not necessarily reflect the opinions or strategies of Intel Corporation or its worldwide subsidiaries.