Preparing for Extreme Parallel Enviroments from a Student's Perspective

Every student (including me!) upon graduation hopes their University has prepared them with all the skills to beat out the applicant sitting next to them for a job. Before joining as an intern with the Intel Academic Community, my conception of parallelism was to the say the least hazy. It was something I had at some point heard in lecture and buried far in the back of my brain in case I had to define it on a test sometime in the future. Through my time as an Intern my relationship with parallelism has become, for lack of a better word, intimate. With more and more manufacturers shifting towards many-core platforms, parallelism is making a strong presence in today’s computer industry and, whether I liked it or not, it was a change I, and all computer science students, will have to reckon with.

So, as a student in the middle of this whirlwind of change, it would be nice to know what the revised list of skills is to beat out that applicant sitting next to me for a job. Luckily for me, this year's Supercomputing Conference had a panel session on just this issue. It was called ‘Preparing for Extreme Parallel Environments’ and it featured some of the greatest minds in academia and industry engaged in thought-provoking discussion on how industry and academia can prepare for a parallel world.

This year’s SC10 panel addressed three main questions everybody involved was itching to have answered.
1.) What does Industry expect from academia to prepare students for the workforce?
2.) What Changes must occur in curriculum to support High Performance Computing (HPC) and Programming models?
3.) What programming models are needed to support HPC and Parallel Programming?

The first panel breakout session was lead by Michael McCool (Intel) and Matt Wolf (Georgia Tech). This session sought to define what knowledge and skills Industry expected from graduating students seeking employment. Participants of this session acknowledged that HPC is a diverse field and there is a lot of information a student could potentially be expected to know. With that being said, a list of topics the ideal HPC grad would know as well as the ideal skill set of a recent college grad pursuing the position of HPC developer was generated.

Topics the ideal recent HPC grad would know:
• Computer Architectures (HW vs. SW)
• Domain Expertise(for example: physics – simulate vs. modeling vs. analyze)
• Engineering (design vs. manufacturing)
• Business (search vs. trading)

The ideal skill set for HPC Developer (College Grad Entry):

• Specific language skills (two languages even better!)
• Ability to reason about parallelism
• At least have written a shared-memory parallel program
• Know how to predict performance
• Experienced in writing parallel programs

Speaking from a student’s perspective both the above lists are quite daunting. It would be near impossible to cover that many topics in four years. Coincidently (not really), the panel’s discussion concluded with a look at whether six year would allow for enough time to cover all the above topics.

The second panel session was kicked off by Dan Ernst (University of Wisconsin Eau-Claire) and Wen-mei Hwu (University of Illinois). The panel discussed what changes must occur in the curriculums of computational science programs to support HPC and programming models. To start off they identified challenges academia would need to overcome in order for any of these curriculum changes to occur and then moved on to highlighting opportunities for change in the curriculum.

• Need to overcome current inertia and get ball rolling with changes
• Material too difficult to engage beginning students
• Currently available material(e.g. textbooks) are out of date
• Poor faculty readiness with material
• Application of HPC(real world examples) are not presented enough to students

Opportunities for change:
• Pull HPC down to minor degree level (to be understood by even non-CS majors)
• Degree Focus areas
• Cooperative Capstone Projects between CS major and non-CS major or CS major and CS minor.
• Encourage students to apply HPC to solve math, physics, and other department’s problems. Example: Link Math and CS through numerical apps and require problems to be solved using HPC

The panel discussion was concluded by proposing that CS degrees are isolated because of the lack of understanding students have in applying their knowledge. For example students may study how to build a library without knowing its use or how it will be applied. Too much focus is placed on the mathematics/details of the computation; whereas, more focus needs to be placed on assisting students in making the connection between the knowledge and actually applying the knowledge they have learned.

The last panel was directed by Ben Gaster (AMD) and Tim Mattson (Intel). The discussion here was centered on what programming models are needed to support HPC and Parallel Programming. The main take away from this panel was to urge developers to shift their attention from obsessing about which programming models should be created, and rather work on and perfect current software. Why? Because what people want are a small number of good standards that work and work well. The discussion participants listed OpenMP, OpenCL, and MPI as programming models that fit this standard. Further discussion revolved around how to actually make the models we have work better. Another point was that students are lacking understanding of the design patterns that parallel programming experts consider second nature and know like the back of their hand. It was recommended that class time needs to be taken to introduce and explain these design patterns in detail, because with that, understanding programming models will be easier.

I thought the SC10 panel on parallelism was a huge success in educating students, professors and academia. Personally, the panel provided me with insight on the future of the computer science curriculum and what employers might expect of me in the future. The panel produced mixed emotions from a student’s perspective, some combination of excitement and nervousness. Though, I am excited for the possibilities that result from shifting to many-core platforms, the magnitude of curriculum changes and shifting expectations from industry are rather overwhelming. Regardless of all the uncertainty surrounding parallelism, there is one certainty: a guarantee that the future of the computer industry will be action packed. So, at the end of the day, how can I not be excited to get to be a part of all the action?
For more complete information about compiler optimizations, see our Optimization Notice.


lilian.moraru's picture

In my country, Parallel programming is presented by University only for C#, and it's pretty different from what u see in a C/C++ implementation. After I saw an article in one of the Open courses of MIT, I decided to leave Java and C#, object-oriented languages in general for C/C++. In the example with "Matrix Multiplier": from Java to C, the implementation with some optimizations using C flags and Parallel programming the program achieved about x290 000 more performance...

yinong-chen's picture

Another direction of parallel computing is done through Service-Oriented Architecture (SOA). In this software architecture, components are services running different hosting servers. This paradigm shifts some wight to the servers. Cloud computing is an example of such shifting.
We need to think hard how to make the servers are well supported by many core computers. The reasons include:
- Cloud computing and Web-based applications are moving distributed desktop computing to the more centralized Web servers
- The server does multi-tasking in both kernel mode and does multi-tasking and multithreading in user mode, with potentially much higher load:
-- Many applications are deployed in a single Web server;
-- Each Web application and each Web service can be accessed by many clients at the same time;
-- Each access (session) resulting the creation of a new thread.
Notice that millions of user can access the same Web application at the same time.

Paul Steinberg (Intel)'s picture

Yes - Nice post Abi. Thanks.

I would be curious to hear from other students as to how parallel programming skills are (or, are not) introduced in their classes and if it is a skill they feel will bolster their employment goals after graduation.

Clay B.'s picture

Excellent summary, Abi! I was only able to sit in on one of the three breakouts, so it is nice to know more details about what was covered in the other two.

anonymous's picture

OpenCL does not fit this standard. It's lack of support for developers and a limited resource for learning makes it a bad choice for development time centric projects. I did not see any mention at all of CUDA! Is this because this is an Intel sponsored blog?

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.