Getting From Zero to Thread Monkey: Book Recommendations

I keep getting asked about what someone might do to get some background and information on multithreaded programming if they were starting from scratch. This is a list that I send out in response to this question about books I've read or have some opinioin about. (From now on when I get such a request, I can point the querant to my blog entry and boost my hit count. :-)) I've tried to annotate each book with a publisher's website. Where I couldn't find a publisher page, I've used the page. [Note: this is not an endorsement of, but I do find I can get a lot of book information from them, even when things are out of print.] If you find a broken link, let me know.

A good start would be the "Programming with Hyper-Threading Technology" book by Gerber and Binstock from Intel Press. This covers some basic concepts and offers introductions to the major threading models. Programming for HT versus MC shouldn't be all that different, at least at the level most programmers will be dealing with.

Recently, Intel Press has published "Multi-Core Programming." This is a book I try to warn others away from. There are too many inconsistencies, specious topics, and just plain wrong information to be make this book worth the effort, especially for beginners. I've passed along some of my major concerns about this book to the publisher and the authors. I can only hope that a revised edition might set things right.

In the back of the HT book, there is a recommendation for "Threads Primer: a Guide to Multithreaded Programming" by Bil Lewis and Daniel Berg that appears to no longer be in print. I've not read this one, but I have the book by these same authors dealing with POSIX Threads. The latter one is dreadful, but the former might be just the thing for those starting out even though it is over 10 years old. Caveat Emptor.

I've found "Windows System Programming, Third Edition" by Johnson Hart to be a first-class introduction to Windows programming. There are four chapters (over 100 pages) devoted to Windows Threads programming.

Others, with more in-depth Win32 coverage, are "Multithreading Applications in Win32" by Beveridge and Wiener and "Win32 Multithreaded Programming" by Cohen and Woodring. I've been told that there are some inconsistencies with what the former book contains and the actual state of the current API. The latter one is out of print according to the O'Reilly website. Since both of these are almost ten years old and 64-bit computing is more prevalent, be sure to exercise caution when reading either of these.

For Pthreads, I highly recommend "Programming with POSIX Threads" by David R. Butenhof; the O'Reilly book, "Pthreads Programming," is good, too.

As for OpenMP, the first book published on that topic was "Parallel Programming in OpenMP." I've always found that the current OpenMP specification ( is an excellent reference and an adequate tutorial for many people. There are several coding examples to illustrate the pragmas, directives, and APIs. Since there is nothing addressing how to develop parallel or OpenMP threaded apps, you may wish to seek additional resources, though.

Michael Quinn wrote "Parallel Programming in C with MPI and OpenMP," but this seems a tad pricey if you're only looking for the few chapters on OpenMP. However, If you're also interested in MPI programming (which can be done on multi-core platforms), this is a great book with excellent examples taken from many different computational areas.

A more advanced program design book for multithreaded programming is "Patterns for Parallel Programming" by Mattson, Sanders, and Massingill. I'm impressed with the first few chapters I've read (even though my name is spelled wrong in the reference section :-( ).

One area of text books that I've not had much need to keep up with is the more theoretical or survey books about parallel computing and algorithms. The biggest reason for this is that I no longer get free copies of text books like I did when I was in academia. If I'm going to fork over $50-$125 for a technical book, it's going to be about something I don't know too much about. However, if you're looking for something more theoretical with regards to concurrent programming, I can't recommend two books by M. Ben-Ari enough. Specifically, the 1982 classic "Principles of Concurrent Programming" and the most recent "Principles of Concurrent and Distributed Programming, 2nd Ed." are both worth it.

If you've got other recommendations, I'd like to hear about them. Also, if you've got a favorite book that covers parallel computation in a general way or as an introduction to the field, let me know what it is and what you like about it. While I've got quite a few books on parallel algorithms, too, many of these were written in the heyday of distributed memory architectures or are based on theoretical models of computation. (Anyone know what a CREW PRAM is?) If there are more accessible or general parallel design/algorithms books, I know my library could stand to be updated. Besides, once you've mastered a threading API, if you can't design and implement parallel algorithms in your codes, you might just as well have studied underwater basket weaving or benchmark mining.

Para obtener más información sobre las optimizaciones del compilador, consulte el aviso sobre la optimización.