Sequential programming is dead. So stop teaching it!

By Paul Steinberg (Intel) (13 posts) on October 22, 2008 at 3:49 pm

Sequential programming is dead.  So stop teaching it!

This teasing diktat points out an important truth – All major manufactures of CPUs, GPUs and ASICs have moved to a many core design, yet universities and colleges are not training engineers in the parallel and concurrent disciplines needed to efficiently program on such systems. To make matters worse, many of the computer science and engineering faculty are often not yet informed enough to effectively teach the subject, even were it to be offered. The sad fact is that unless a more vigorous program of parallel computing is inserted into the curriculum, most engineering and computer science graduates will not have the skills to provide competitive advantage on multicore and many core platforms.

 

So start teaching parallelism!

Undergraduates need to be exposed to parallel programming techniques starting in CS1 and then need to build on the skill in every (relevant) course.  This is not the case at most institutions; when they teach parallel computing at all, they often relegate it to advanced topics or elective courses.  That said, there are a number of colleges and universities that have found that it is not all that difficult to incorporate it into their existing curriculum.

Intel Software College course architect Michael Wrinn spoke  with Professor Daniel Ernst ,University of Wisconsin – Eau Claire,   on the topic recently at ITiCSE in Spain where Professor Ernst delivered his paper Concurrent CS: Preparing Students for a Multicore World .  Professor Ernst employs what he calls an integrated approach wherein he mixes concurrency in the already established curriculum.  This lightweight tactic is readily comprehended by the students and does not entail a wholesale rewrite of existing course material.  For an in depth discussion with many examples, view the webinar Daniel led on the subject last month.

 

Professors Matt Wolfe and Ada Gavrilovska of Georgia Tech will be leading (or have led, depending on when you read this) a webinar on this topic as well Thursday, October 30, 9:00 AM PST.  It is titled Multi-core in Classroom, the view from Georgia Tech.  Register for this webinar and join in the conversation.  The same link will bring up the archive webcast as well.

 

Ride! Ride, ride!  Wait not for the dawn! Let not the swift wait for the slow!  Ride!

All right, the Nazgul are not coming, but we are facing a sea change in computing. Single core is gone and will not return.  Scaling will be based on our ability to efficiently program for many (and then many many) compute cores.  We have to train the next gen of engineers to think parallel or we will not have done our jobs.  Join in the discussion by attending joining the Intel Academic Community, attending our webinars, contributing to parallel course development on the Academic Community Wiki or even hosting a webinar of your own.  Contact me and I’ll see what we can do.

We are not all mouth, we actually do things

Just a teaser for my next blog entry on our plans for a cross indutry and academia working group on curriculum developement for concurrency and parallelism.  We will be kicking off at Super Computing 08 in Novemeber.  More soon.

Categories: Academic, Parallel Prog. & Multi-Core

Comments (23)

October 22, 2008 5:59 PM PDT

Aaron Tersteeg (Intel)
Total Points:
15,080
Status Points:
15,080
Community Manager
 Sequential programming is alive and well. Long live sequential programming!
Another view is that parallel programming needs to be taught on top of sequential programming not instead. I look forward to hearing what the experts have to say at Super Computing 08. See you there.
October 22, 2008 7:07 PM PDT

Paul Steinberg (Intel)
Total Points:
6,434
Status Points:
6,434
Community Manager
 Sequential programming lives -Le Roi est mort, Vive le Roi. Yet the point remains that parallelism and concurrency must be taught as part of the undergraduate curriculum. Currrently this is not be done enough/comprenhisvly/at all.
October 22, 2008 10:48 PM PDT


EdwardOCallaghan		 Bring Unix back into schools then you may seen the threading trend follow back in..
Intel should do its part by making sure all of its hardware devices work with OpenSolaris, and work well !
Regards,
October 23, 2008 12:36 AM PDT


UX-admin		 That's right - bring OpenSolaris back to academia, and then you'll have a chance at raising awareness of parallel programming. I second that.

(Back when I was at the university taking a course in operating systems, we went through Dijkstra as well as a side by side comparison of Windows NT microkernel and Solaris's own kernel. It did wonders for my understanding of parallel programming. This was almost a decade ago, when most systems were single core.)
October 23, 2008 7:18 AM PDT

Asaf Shelly
Total Points:
1,430
Status Points:
930
Brown Belt
 Sequential programming is only one type of parallel programming. In sequential programming the flow is maintained by a stack and there is only a single flow. Parallel programming can implement many types of flow-control methodologies. I agree that we should not focus all attention to the single-stack process methodology for beginners.
Our experience in my courses about parallel computing is that students that have a degree in exact science are about 5 times faster to understand the concepts of parallel programming than students that have a degree in computer science. My guess is that computer science graduates are already 'locked' on single task methodologies and their understanding is built of those foundations. Everything new that they learn is first translated in their mind to what they know. In a way it is like teaching manual transmission to someone that is used to automatic transmission (for over 10 years).

Asaf
October 23, 2008 10:42 AM PDT

Gael Holmes (Intel)
Total Points:
14,363
Status Points:
14,363
Black Belt
 FORTRAN ROCKS!
October 23, 2008 10:50 PM PDT


Ben Metcalfe		 I call bull**** (a little on this).

Firstly, (as disclosed) you're a manufacturer of multicore processors and so clearly it's in your interest to be getting people to build software that utilizes the features... especially as it's pretty hard to up sell value much above the current 3ghz cores. Clearly you're a little biased around the benefits of parallelism as it requires cpus that your company enjoys selling.

However, the opportunity of multicore architecture can be looked at in two ways. Yes for certain high end ups, it's more ideal to incorporate parallel threads, but at the same time there is still a lot of be said for the benefits of being able to run multiple sequential programs across different cores to create true multitasking.

I have a 8 core twin Xeon machine sitting under my desk, but I utilize it by operating multiple virtual instances dedicated to various cores within the system. If a given process I wanted to run *forced* me to use parallelism I'd have to dedicate more cores to that given VM and as such either see a reduction of backround resource or have to move this onto dedicated equipment.

So what I'm saying is yes, there is a value in parallelism programming where appropriate, but at the same time I would hate to see a world where every new piece of software (eg my word processor) required a multi-core processor just to run.
October 24, 2008 2:18 PM PDT


Dave Wheat		 Back in the mid 1980's (seems like ancient history), a professor told me one truth about universities -- and I think it certainly applies here. Much of the curriculum universities teach is about 10 years behind the times. New technologies, approaches, and theories change so rapidly (especially in high-tech), that most universities cannot hope to keep up. So they stick with the basics until such time that a new technology becomes main-stream (and business that hire grads are actually looking for those specific skill sets).

Given time, universities will see the light and catch up. We are early movers in this rapidly changing industry. Once businesses and software vendors internalize the benefits of utilizing multiple cores to their maximum potential and begin demanding recent college graduates have those skills, universities will come around.
October 29, 2008 5:33 PM PDT


Terry		 I'm self taught with most of the stuff I know. I've been out of school for 25 years and things that I learned in school are a bit dated. I think the universities should try to keep up with the times but if the student doesn't take initiative and keep on learning on their own, then in five years it won't matter to that particular person anyway.

Multi threading is a very important subject and it should be understood by all programmers. People who say it's not needed or necessary, don't really understand the power that it provides. If developers understand how to implement multiple threads, they can make a program start in 3 seconds instead of 15. That might not seem like a big deal but use the program 1,000 times and you've saved 12,000 seconds or 200 hours. It's a productivity improvement for businesses.

It's not a difficult subject to understand and implement. But if it's not implemented properly, you can slow down the application by forcing flushes of the processors cache.
October 30, 2008 5:34 AM PDT


pesho		 DEAD!? as if all on the world is INTEL!? what about cell-phones, embedded devices, Wii-console.
and let me ask you ONE big question: is in it so that parallel programming is nothing more than sequential programming at the same time? to put parallelism against sequential programming is redicilous to begin with. this article is nothing but an intel hype. and ONE more thing. parallelism wasnt even introduced by Intel in main stream. it was NVIDIA & ATI !!!!
November 13, 2008 8:35 AM PST


Tim		 OK, so ... how?

Our languages suck at it. Do you want us to start using languages designed for parallelism and concurrency, like SISAL or Erlang or FORTRESS? Almost nobody uses them in industry, and I've heard serious complaints about them as languages, too. And when we teach languages that aren't big in industry (e.g., Scheme), students think we're doing it to be cruel. Just as you want to be practical by having students learn this because you think it'll be useful in industry, students want to be practical and learn things that they see *are* used in industry.

Intel has a C compiler. Intel seems to write most of their shipping code in C. Students have Intel chips in their computers. When they want to write cool programs, it's no surprise they look to C.

I suppose it's possible to write parallel code in C, but it's pretty painful. People get into programming to scratch an itch. Intel's itch may be "you've got 64 cores...", but virtually no students have this itch. Unless parallelism is the default, why would I bother?

I suspect we'll start teaching it about the time when big computer companies (like Intel) start shipping parallel compilers and parallel code.
November 13, 2008 8:52 AM PST


Daniel Bradberry		 Yes parallel programming should be taught in Universities but not in CS1. Many of the people I know started a computer science degree having never written a line of code in their life and it was bad enough trying to explain Object Oriented methods let alone how to distribute a computational task across a 64-core processor.
November 13, 2008 9:51 PM PST


Joe		 Until you show me how parallel programming is useful for anything other than math simulations and graphics, I'll remain unconvinced. Time and time again the people who push parallel programming fail to provide examples beyond solving problems that nobody in the real world has. Take the top 50 applications actually in use right now (starting with a web browser) by real users and not physicists and tell me how parallel programming is going to significantly improve them.
November 13, 2008 10:14 PM PST


Stephan Schmidt		 Algorithms are inherently sequential. Thinking is sequential. Because people are sequential and not not very good at thinking and doing things in parallel.

It's sad that in 2008 the tools are not good enough to parallize algorithms and programs automatically. What are the compiler people doing all the time? Why do I need to use Erlang and message passing to write parallel software? That has nothing to do with my business tasks I need to solve. Why should I need to think about cores? People want to deliver business value.

Cheers
Stephan
November 14, 2008 12:46 PM PST

wolfmurphy
Total Points:
805
Status Points:
805
Black Belt
 I teach CS. Fundamental to my teaching are two seemingly conflicting directions. With CS being a scientific discipline, my students better have a firm theoretical background. At the same time I want my students to be able to quickly cut beautiful clever code.

I spent over 30 years programming on multimillion dollar rarified architectures. Those architectures are now everywhere, and those skills ones helping to exploit those architectures. I don't want my students to be surprised in a job interview, or on the job with not being versed in the basics. We do need how to weave thread, MPI, and Cuda into the standard curricula.

And to respond to Joe, all that counts to me in CS are math simulations and graphics. The mathematical models and related visualizations are what is most interesting to me in todays world. If I had my way, I'd morph what CS stood for into Computational Science. I want to see people who have the science background to know what problems need to be solved, the math background to know they are being solved right, and the CS background to know they are being solved efficiently. Many branches of science are reinventing themselves in using these mathematical models to forecast what to look for in.

I want to see parallelism and computational science taught at least starting in high school, using tools like Alice, Vensim, and the Second Life metaverse.

... Tom
November 14, 2008 2:27 PM PST

Michael Wrinn (Intel)
Total Points:
2,199
Status Points:
1,699
Brown Belt
 During Intel's Asia Academic Forum in Taipei, a couple of weeks back, participants (all professors from a selection of top Asian universities) voiced an observation similar that expressed here by Dave Wheat: given the incommensurate rates of change of technology and curricula, how best can a committed educator respond?

In the spectrum of academic subjects, we have at one extreme the static fields such as maths, or classical physics, where little changes beyond notational style; a prof could almost use the same textbook on which he himself learned. At the other end are the sciences which are changing constantly -- certain areas of biology come to mind -- and the best schools are reconciled to constant updates of their courseware. Let me submit that Computational Sciences fall somewhere in the middle of this span: a background of static concepts (sorting algorithms, compiler technology, data structures etc), an occasional shift in programming language paradigm or dialect, and technology improvements impacting not so much the content itself as its convenience and scope. If we were teaching 5 years ago, or even 10, we haven't had to update either our courses or skills too dramatically.

Now comes this hardware shift to concurrent systems, and Computational Sciences is getting shoved, rather abruptly, toward the busy end of that staticdynamic spectrum of disciplines. This is not our comfort zone. For all the gloating about Moore's Law and orders-of-magnitude enhancements, it cannot be claimed that the conceptual side of the science has kept pace. Suddenly, there is some hard work to do.
November 17, 2008 10:50 AM PST

Paul Steinberg (Intel)
Total Points:
6,434
Status Points:
6,434
Community Manager
 As the instigator of the offending post, I am delighted to see the discussion. I am down at Supercomputing 08 with Tom and others and we will be discussing this topic on a panel this evening,. The discussants include NVIDIA and Intel (and SUN and IBM and AMD) so clearly this is an issue of concern to ALL computer manufacturers. If you are in Austin - come to room 10B this evening (Monday, Nov 17). If you are not, we are doing a live webcast on the subject http://is.gd/7Rvz on Thursday Nov 20.

I would love to be able to carry discussion on this topic further. One idea would be an open forum using some kind of internet voice/text app. Not sure which yet. I know I could drag some Intel folks and I'm pretty sure I could get a few folks from elsewhere in the industry and academia to join in. Maybe we could even make this a semi-regular and continuing discussion on broader topics – you guys clearly have both the opinions and the savvy. So let me know -post something here if you like the idea http://is.gd/7RyX and we'll figure out how to make it happen.
December 31, 2008 11:15 PM PST


jhon		 I agree with but some extend the Sequential programming is only one type of parallel programming. In sequential programming the flow is maintained by a stack and there is only a single flow. Parallel programming can implement many types of flow-control methodologies.
http://www.cyberdesignz.com/, you can get more feedback from this
April 9, 2009 3:31 AM PDT


Go-Gulf		 Analyst reports indicate that Cobol salaries are on the upswing. The language is easy to learn, there's a healthy demand for the skills, and offshore Cobol programmers are in short supply.
Seasoned Cobol programmers, in contrast, "should be in pretty good shape job-wise. If they have a position at an organization that intends to keep its legacy Cobol apps, then they are probably set for life.
July 20, 2009 4:39 AM PDT


Web Solutions		 Hi,

I would agree partially with Aaron Tersteeg, that sequential programming is yet alive and being used into many application. I would say that parallel programming is another innovative streamline but no body would deprive of the event driven programming and object oriented technologies.

- J.
August 12, 2009 1:14 AM PDT


Web Designing		 well it is not 100% true about the death of the sequential programming. it has been still practiced in a number of countries. although it is a fact that the worth has been reduced with the advent and innovation of new programming skills.
August 13, 2009 4:42 AM PDT


Memory Foam		 I think sequential programing is not dead , but some experts say that it is not as alive as to parallel programing.both of them have different methodology.thats why sequential programing is not taught in many institutes.
August 17, 2009 4:36 AM PDT


Web Designing		 I agree with but some extend the Sequential programming is only one type of parallel programming. In sequential programming the flow is maintained by a stack and there is only a single flow. Parallel programming can implement many types of flow-control methodologies.

Trackbacks (0)


Leave a comment  

To obtain technical support, please go to Software Support.
Name (required)*

Email (required; will not be displayed on this page)*

Your URL (optional)


Comment*