Parallel Programming using Stone Knives and Bearskins

At this year's SIGCSE conference, I sat in on the last half of the Birds of a Feather session on Curricula in Concurrency and Parallelism. Toward the end of the session, someone made a comment like "We are teaching parallelism with such low level-languages, methods, and tools." I kept my mouth shut, but the first thought I had was "You say that like it's a bad thing."

Recall the historical progression of computer programming. We started with patch cords connecting components. Eventually we figured out how to store programs like data and could program in machine language. To make this simpler and more easily understood by humans we devised assembly language programming.  To make this even simpler, high-level languages were created. The parade of HLLs has become more expressive with each generation, from Algol and Fortran to C++ and Scheme.

Not too many practitioners use assembly language these days. It is still available, but is considered too low-level. It was an early attempt to make programming easier. With advances in programming language theory and the advent of program translation tools (e.g., compilers), we were able to come up with a more natural programming interface.

Right now we are in the infancy of parallel programming. Yes, it's been around and practiced for decades before this. However, it was only used by specialists in the HPC community. Now, with the advent of multi-core processors, parallel programming can no longer remain in the realm of a select few. Everyone that will be programming computers will need to understand and practice parallelism. But, you may complain, the choices for parallel programming aren't really "ready for prime time."

True.  Threads seem like the assembly language of parallel programming. Why isn't there anything in wider usage that is easier, better, and safer to use? For the answer to this questions, go back and read the first sentence of the previous paragraph.

When new technology is getting started (or opens up to a much larger audience) we take baby steps and need to do things that will seem overly difficult 2, 5 or 10 years later. Right now we don't know the best way to do parallel programming.  (Heck, I'd say we don't even know the best way to do serial programming.) Through adversity and pain and trial-and-error we will find those things that work, those that don't work, and implement better methods to do it all. It will take time, though. You can't go from stringing together 1's and 0's to writing polymorphic object classes overnight.

The next time your threaded code deadlocks or you take hours to track down and tune a load balance issue, don't curse your tools. Think of it as a story that you will be able to tell your grandkids. Of course, they won't believe that you were able to get even the simplest codes to execute correctly using such crude tools. Then they'll tell you about how difficult it is for them and complain they don't have something more high-level to use.