4,391 Posts served

10,712 Conversations started

Top 10 challenges in parallel computing

By Michael Wrinn (Intel) (13 posts) on December 31, 2008 at 11:02 pm

In the spirit of yearend, top 10 lists, here is one for parallel computing. A version appeared earlier this year in a paper with Tim Mattson, who's talked about this in other forums and gets full credit for its collation; it reflects thinking by some of the folks here, and is worth posting for discussion. Note the list is for parallel computing, so encompasses hardware, tools, and design considerations as well as programming. In priority order:

1. Finding concurrency in a program - how to help programmers “think parallel”?
2. Scheduling tasks at the right granularity onto the processors of a parallel machine.
3. The data locality problem: associating data with tasks and doing it in a way that our target audience will be able to use correctly.
4. Scalability support in hardware: bandwidth and latencies to memory plus interconnects between processing elements.
5. Scalability support in software: libraries, scalable algorithms, and adaptive runtimes to map high level software onto platform details.
6. Synchronization constructs (and protocols) that enable programmers write programs free from deadlock and race conditions.
7. Tools, API’s and methodologies to support the debugging process.
8. Error recovery and support for fault tolerance.
9. Support for good software engineering practices: composability, incremental parallelism, and code reuse.
10. Support for portable performance. What are the right models (or abstractions) so programmers can write code once and expect it to execute well on the important parallel platforms?

Reactions, please! Some things to consider:

• We make no mention of power consumption - is that reasonable? How would the power challenge be phrased, and where would it slot into the list?
   
• Data locality (#3) is intimately tied to scalability (#4). As systems become heterogeneous, with an assortment of different latencies among elements, how can we reconcile this tuning to portability (#10)?
   
• Are we underestimating the importance of composability, so innocently tucked into "good software engineering" (#9)? And by the way, how would you teach that concept to university students, and in which course?

Lots more -- let's pick it up in 2009. Happy New Year to all!

Categories: Academic, Parallel Programming

For more complete information about compiler optimizations, see our Optimization Notice.

Comments (1)

January 8, 2009 8:48 AM PST David Brooks

I am a CS student at the University of New Orleans. I think #1 is the biggest things I am trying to wrap my mind around. It seems to me all of this needs to be taught early on in the life of a university student since it takes practice to think in parallel (for me anyways). So far it seems to best understand this problem I needed to understand how GCC turns normal C code into assembly, and how that assembly can be executed in a parallel way. I do not know of a guide to any high level language that gives best practices on writing code that the compiler can easily optimize for more than one core but such a guide would help. I guess that's all for this post. Cheers! David Brooks

Trackbacks (0)

Leave a comment