Software Concurrency for undergrads? Panel discussion at IDF

By Michael Wrinn (Intel) (12 posts) on August 13, 2008 at 10:52 pm

The Intel Developer Forum, in San Francisco August 19-21, brings this year a series of talks and workshops of particular interest to the academic community: a chalk talk on research collaboration in parallel computing, a technical session on expressing parallelism, a threading self-paced lab - details of these and more may be found here.

One of these is a panel discussion, Academic Community Multi-core Programming Roundtable, where we'll address the positioning of multi-core (more broadly, parallel) computing in undergraduate CS courses. We are fortunate to have two panelists with uniquely broad points of view, each from his own school, and from national-level efforts:

Dan Garcia  teaches at UC Berkeley, and has developed curricula to teach concurrency to *introductory* students, in the first 3 CS courses; topics range from a functional implementation of MapReduce, right through pthreads and MPI. Part of these UCB efforts are documented in the paper, Infusing Parallelism into Introductory Computer Science Curriculum using MapReduce. Beyond his immediate campus, Dan is also active in education: he is a member of the ACM Education Board, and has presented every year at SIGCSE since 2001.

Tom Murphy teaches at Contra Costa College, where he chairs the department and covers a range of courses in theory and practice. In addition to this role, Tom is seriously involved, as organizer and instructor, with the SC Education Outreach (a part of NCSI, the National Computational Science Institute), delivering summer workshops in parallel computing to college professors.

Our third panelist is your humble blog author here, Michael Wrinn. I'll join this distinguished cast as a voice from industry, where I've enjoyed a role in recent years of working with academia on fostering parallel computing in education. Scot Buck, Intel's Director of University Relations for Intel Research, will provide referreeing services, and keep things (more or less) civil.

While coming from different points on the computing education spectrum, the panelists are all concurrency "evangelists", in the sense that we practice and teach parallel programming, and suspect that everyone else should too. To me, a CS degree based solely on sequential, von Neumann techniques is starting to resemble a degree in CS history: interesting, somewhat important, but ignorant of the key design of modern computing platforms. Of course, there are other points of view -- no less than Donald Knuth states: "I won’t be surprised at all if the whole multithreading idea turns out to be a flop...a pipe dream".

So, my academic friends, where do you stand on this matter, and how does that influence your teaching content? How soon should it be introduced, and how much is enough? How important is the choice of implementation model?

Please join us at this IDF Panel discussion - to challenge us, to share your experiences, plans or frustrations, and to let us know what we could to do help, in contending with the computing switch to concurrent platforms. There's a free, 3-day IDF pass available for professors interested in attending; I look forward to seeing you there!

Categories: Academic, Parallel Programming

Comments (4)

August 15, 2008 3:00 PM PDT


Charles E. Leiserson		 Multicore is mostly about performance, and educators have done a pretty good job of drumming processor performance out of the curriculum. Ten years ago, you had C for performance programming, but languages promising high levels of abstraction (Java, Scheme, ML, Python, etc.) have since been preferred for teaching, pushing C out of many mainstream curricula. Conventional wisdom says performance doesn't matter, arguably because semiconductor manufacturers offered ever-increasing performance with faster clock speeds, a trend which ended in 2003. In addition, in industry, most performance-critical applications have migrated from C to C++, and educators have rightly been wary of teaching such a huge and hairy language as C++. Consequently, students are left today without a good linguistic context within which they could be easily taught performance programming -- with or without concurrency. Crossing the linguistic gap in the curriculum can be difficult, because it generally means teaching students a new language (probably C/C++) just to deal with performance.

My experience in teaching parallelism to undergraduates at MIT indicates, however, that there are two _theoretical_ notions with enormous practical impact that should be taught early. The first is the quantification of parallelism as the ratio of work and span, and the second is the notion of a determinacy race. (My e-book http://www.cilk.com/multicore-e-book presents these two concepts in an elementary fashion.) One doesn't need a linguistic basis to appreciate these concepts, but for students to be well educated in multicore computing, they also need a framework in which they directly experience these notions and in which performance can be measured and applications tuned. Unfortunately, even at MIT, many of my colleagues don't see the real value in the teaching of performance over the teaching of easy abstractions, and thus our classes on multicore must spend lots of time just bringing students up to speed on basic C/C++ language issues. This adverse state of affairs also limits the degree to which multicore can become pervasive in our curriculum.

Lest I sound too pessimistic, a group of us at MIT are working hard to change things, and I'm optimistic we'll have impact sooner than later, but I hate to quantify what that means in years. I'm also concerned that with the nationwide drop in computer-science enrollment, even the few students we are able to educate well in multicore won't be nearly enough to fill the demand, thereby adversely affecting the economy, which is habituated to ever-increasing application performance.
August 17, 2008 11:44 AM PDT

Ami Marowka
Ami MarowkaTotal Points:
40
Registered User
 Unfortunately, I will not be able to attend the Intel Developer Forum, in San Francisco. However, I will share with you some of my thoughts regarding teaching multicore programming in academia and outside the academia.

First, I do not agree with Prof. Charles E. Leiserson that "Multicore is mostly about performance". Parallel computing calls, first of all, for a new kind of thinking – parallel thinking. I believe that if we will find the way to teach our students to think in parallel then the performance issues will be understood more easily. The way I choose to guide my students to think-in-parallel is by teaching them the data dependency problem by using examples (see my paper). Moreover, a parallel-thinking is needed if we are expecting for new breakthroughs in designing more creative parallel algorithms and parallel programming models.

I participated in the last IEEE IPDPS conference in Miami. The topics in the conference panel were the same topics that will be introduced in the IDF roundtable. All the panelists in IPDPS shared the same thought – the academia does not do enough to teach parallel computing. However, they did not know to answer the question – why the academia does not do enough to teach parallel computing? Moreover, they mentioned two books that did not contribute to the parallel computing discipline. The first one is the famous book written by Prof. Leiserson et al,, "Introduction to parallel algorithms". The first edition of the book dedicated a full chapter for parallel algorithms but was taken out in the second edition. The second series books are "Computer Architecture" by John L. Hennessy, David A. Patterson that claimed during the years that parallelism performance will be solved by hardware alone.

Michael Wrinn mentioned the last ITiCSE conference. Unfortunately, I submitted a paper to this conference but the paper was not accepted. 3 reviewers of 5 ranked my paper "strong accept". All the reviewers thanked me a lot for let them know that the concurrency revolution has started … From this feedback I learned how much people in the academia are not aware to the parallel computing revolution.
The paper was accepted to IEEE DSO and you are invited to read it at
http://dsonline.computer.org/portal/site/dsonline/menuitem.6.....du.xml&;


In the last two years I have tried to convince people from Intel Software College in Europe to help me, to help them, to expand the knowledge regarding multicore programming. Till now without any success.
I asked them to help to open multicore school in my country as they did in China. I did not ask for money, software or hardware. I just ask them their name. I asked them to give me permission to give my students that attended my course a formal certification from Intel. Microsoft and Cisco usually give their students such certifications. I told Intel that I ready to use their courses or mine. I told them that I ready to be trained in Intel for this purpose. I told them that I ready to use only the compilers and tools from Intel. But till now they have not raised the gloves. Moreover, I told them that in the industry nobody knows parallel programming simply because they were not thought parallel computing in the university. I told them that I am also ready to give courses for the industry based on Intel courses/software/tools but I did not receive an answer.

Without attracting the students with something valuable, it is very hard to pull them in.

The bottom line.
Debra Goldfarb asked "Where is SAP for the Rest of us?" (http://www.hpcwire.com/blogs/26967564.html), and argues that we need HPC "ERP". Berekly researchers are talking about the same thing, but call it the "13 Dwarfs" (http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.html). Prof. Leiserson concerned about the nationwide drop in computer-science enrollment. So, if I connect all these points, sooner or later we will have HPC "ERP" and multicore programming will remain by the hands of experts from Intel and Microsoft only, unless something drastic will change this vision.

Ami Marowka
http://us.geocities.com/amimar2/
August 26, 2008 9:56 PM PDT


John Phillips		 The offer for a free pass is appreciated, but obviously not useful by now.

More important to me is the question of parallel programming in academia and what to do with it. A quick survey of the blog postings shows this entry and a number of others that are discussing (and mostly lamenting) the state of parallel and distributed education in academia with the resulting lack of needed skills in industry. I have spent nearly 20 years teaching physics, mathematics, computational science and computer science at a few different universities in Ohio, and so far my experiences are not very positive.

Ohio is fortunate to have a number of state and federal funded organizations that are trying to expand the available skill set and catalyze change in curricula. I have worked with a couple of them, and with a private university that has tried to teach parallel computing as a required course, and high performance computing as a later elective. Calling it a hard sell is an understatement. Even most of the physical, biological and computer science faculty members in the state don't see a reason to support the work. As part of their mission, the Ohio Supercomputer Center generously offers impressive numbers of processor hours to anyone affiliated with a school in state for free, but most schools have never even expressed potential interest in taking advantage of the program. The University where I taught parallel and distributed computing is discussing dropping the class from the requirements and the curriculum, while high performance computing didn't get the enrollment to run the last time it came up on the schedule. This is very similar to patterns at other schools mentioned in a different blog entry.

To reverse this trend, a few things are needed.

First, there need to be books written at the level of interested faculty members that provide inspiration and a firm foundation for the field. A friend of mine once referred to his first introduction to Knuth's classic books as "a religious experience." I know of no such books currently available for parallel, distributed and high performance work. Without this foundation and excitement, it is easier to follow the status quo and model the curriculum after what it has always been. As anyone who has ever opened Knuth's books knows, creating an equivalent would be a daunting task.

Given a firm foundation, a more clear idea of how to create good parallel programs dependably using tools that are already familiar for other reasons and that do not need excessive modification from the tools accepted by industry can be constructed. Current discussions almost always descend into the murky realm of atomic operations details, memory models and processor instructions. While some of us want to know those details and there is a need for those skills in the population that designs useful tools, most programmers will not be willing to use any method that requires such considerations. Nor will most companies agree to let such be part of the development process. Programmers want to do what is familiar to solve common problems. Companies don't want to be the only ones sitting on the risk of an unproven tool.

With tools in place, it is possible to add parallel ideas into the flow of the curriculum with minimal resistance. Since industry adoption will increase when companies are more comfortable with the tools, the pressure from potential employers and incoming students will increase. This will also provide a platform for high quality introductory textbooks to be written. I have owned and read a fairly large fraction of the texts currently available, and I can't claim to be satisfied with any of them. I have taught from some, and pulled some ideas from others, but I have never completely liked any. No friends I have asked have claimed to be satisfied, either. Many make their own materials, or just try to avoid teaching the class. The advanced text hoped for in the first point is not a good choice for this. In physics, the Feynman Lectures are a deep and inspiring look at the topics covered in a typical introductory class. They are truly classic in their scope and presentation. And no one with any sense would ever consider using them for an introductory class. (When Feynman tried, it failed miserably.) The advanced text would have the same problem.

This is the path that seems necessary to me, but it is neither quick nor easy. In fact, I don't think that the depth of understanding needed to produce a Knuth-like text has yet been identified for parallel and distributed work, so the initial investment in time and effort would be very large. However, without it there will be little progress in teaching students to think in parallel, because for the most part their teachers also don't know how and don't see it as important that they learn.

John Phillips
August 27, 2008 7:13 AM PDT


Michael A, Gottlieb		 You write that "When Feynman tried, it [using The Feynman Lectures on Physics for an introductory physics class] it failed miserably.

First of all, Feynman never taught using The Feynman Lectures on Physics (books) at all. When Feynman taught Phys 1 and 2 (the 2-year freshman and sophomore physics course required of all students at Caltech) the books did not exist! They were created, later. What the students got initially were lecture summaries and outlines (plus homework problems, lab manuals, etc.) handwritten by graduate student T.A.s.

Secondly, it is a widely held misconception that Feynman's undergraduate course was a failure, or that The Feynman Lectures were not useful as textbooks. They were used as texts for Phys 1 & 2 for almost 20 years at Caltech (accompanied by exercise books that were originally published by Addison-Wesley, and later self-published by Caltech). The Feynman Lectures has also been translated into over a dozen languages, and they are used in the classroom more often in translation than in the original English. For example, FLP is very popular in the former Soviet States - it is believed that millions more copies of FLP have been printed in Russian than in English, though the exact number is unknown.

Feynman is himself to blame for the misconception that his course was a failure, as it is largely due to comments he made in an AIP interview with Charles Weiner in 1966, in which he basically reiterated his self-deprecating preface in FLP. In the same interview (which spanned several days) Feynman told Weiner that "People who were directly in contact [with the students] would tell me that I was underestimating them, and that it wasn't as bad as I thought.", and that he had "better feelings" about The Feynman Lectures since writing his preface. (Years later, in interviews with Mehra, Feynman said that The Feynman Lectures on Physics was one of the best things he did. So, his feelings about FLP seem to have improved with age.)

About Feynman's FLP preface, Matthew Sands writes the following:

"After the completion of the second year of lectures—near the beginning of June, 1963—I was in my office assigning the grades for the final examinations, when Feynman dropped in to say goodbye before leaving town (perhaps to go to Brazil). He asked how the students had performed on the exam. I said I thought pretty well. He asked what was the average grade, and I told him—something like 65 percent as I recall. His response was, "Oh, that’s terrible, they should have done better than that. I am a failure." I tried to dissuade him of this idea, pointing out that the average grade was very arbitrary, depending on many factors such as the difficulty of the problems given, the grading method used, and such—and that we usually tried to make the average sufficiently low that there would be some spread in grades to provide a reasonable "curve" for the assignment of letter grades. (This is an attitude, incidentally, that I wouldn't approve of today.) I said that I thought that many of the students had clearly got a great deal out of the class. He was not persuaded.

I then told him that the publication of the lectures was proceeding apace and wondered whether he would like to provide some kind of preface. The idea was interesting to him, but he was short of time. I suggested that I could turn on the dictating machine I had on my desk, and that he could dictate his preface. So, still thinking about his depression over the average grade on the final exam of the second year students, he dictated the first draft of "Feynman's Preface", which you will find in front of each volume of the Lectures. In it he says: "I don’t think I did very well by the students." I have often regretted that I had arranged for him to make a preface in this way, because I do not think that this was a very considered judgment. And I fear that it has been used by many teachers as an excuse for not trying out the Lectures with their students."

from 'On the Origins of The Feynman Lectures
on Physics, A memoir by Matthew Sands,'
Feynman's Tips on Physics (2006)

And, finally, here's what Feynman said (verbatim) to Mehra in a January 1988 interview (only about a month before Feynman died, so these, I believe, are Feynman's last words on this subject):

"At the end of the two years [1961-63] I felt that I had wasted two years, that I had done no research during this entire period and I was muttering to this effect. I remember Robert Walker saying to me: "Someday you will realize that what you did for physics in those two years is far more important than any research you could have done during the same period." I said,"You're crazy!" I don't think he's crazy now. I think he was right. The books [Feynman Lectures on Physics] are popular, they are read by a lot of people, and when I read them over [I find] they're good, they're all right. I am satisfied; rather, I am not dissatisfied with them. I am am just dissatisfied with the system - whether it would transmit. But when you have a book and somebody from far away writes that he is learning from it... then I feel that I may have done something to a large number of other people, to people everywhere.

They have the books on the shelves. They are used all the time. They are twenty-five years old, and they are still on the shelves. Undergraduate and graduate students use them. They look them up for fundamental ideas behind advanced subjects. There is all kinds of stuff there, more basic physical points of view, and so apparently they are useful. I must admit now that I cannot deny that they are really a contribution to the physics world."

from The Beat of a Different Drum,
by Jagdish Mehra, 1994
(by far my favorite Feynman biography)

Trackbacks (1)


Leave a comment  

To obtain technical support, please go to Software Support.
Name (required)*

Email (required; will not be displayed on this page)*

Your URL (optional)


Comment*