<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated on Fri, 25 May 2012 09:01:38 -0700 -->
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <atom:link href="http://software.intel.com/en-us/articles/is-parallel-programming-hard-1/feed/" rel="self" type="application/rss+xml" />
    <title>Intel Software Network Comments Feed</title>
    <link>http://software.intel.com/en-us/articles/is-parallel-programming-hard-1</link>
    <description></description>
    <language>en-us</language>
    <item>
      <title>By thiamchunkoh</title>
      <description><![CDATA[ I believe that at the right level of abstraction, it is not and in fact can be as easy as sequential programming Pragmatically there are some problems today with taking advantage of the "easiness" of parallel programming. Parallel Thinking The first problem is that programmers have been trained for years in sequential thinking. Most of us have been taught over and over since our very earliest days of programming how to think about programming in a sequential way. This permeates just about everything we do in code development, from the abstract description and analysis of algorithms, to the coding of these algorithms, and from debugging and verifying code, to the way that we read code.Determinism The third problem is a lack of support for deterministic parallelism and overuse of non-deterministic parallelism. By deterministic parallelism I mean a parallel program for which the results and partial results are the same independent of how the computation is scheduled or the relative timing of instruction among processors. In non-deterministic parallelism, the results depend on such scheduling and timing. When we think of a summation in mathematics we often write it using the Sum symbol. In studying mathematics most of us do not think of it as a loop but rather as just the sum of elements---we implicitly understand that addition is associative and the order does not matter. But yet many programming courses spend weeks describing how to write loops to take sums of elements and similar tasks.
It seems that parallel programming today has some of the same issues as sequential programming had in the early days. In the interest of performance there has been significant emphasis on low-level details and often these low level details are not properly separated from the high-level ideas in parallelism. Various libraries and languages extensions require the user, for example, to specify stack sizes for forked tasks, implement their own schedulers or specify scheduling policies, or understand mapping of memory to processors. When presented with such level of detail it both makes programming seem much harder and also obscures the core ideas in the rest.Non-determinacy is not only a problem with testing and debugging programs, but in verifying their logic. one way to do get deterministic programs is to have a serial semantics to the program. In this approach the programmer reasons about the correctness with regards to a particular serial ordering of the program (typically a depth-first traversal of the fork-join structure). The language then makes a guarantee that if the program does not have race conditions, then any feasible interleaving of the execution will return the same result as this sequential ordering. In the case of Cilk++ race conditions can be checked using a race detector. This is perhaps not a perfect solution since the races can only be detected on the test cases, but is as good as any regression testing. Also it is much easier to verify that a program or program fragment does not have race conditions than to verify correctness in general. In the case of sequential vs. parallel thinking, I believe that at the right level of abstraction they are really not that much different. In the case of non-determinism, I also believe this is an issue of abstraction. My opinion is that as a programming language and runtime system Cilk++ does a excellent job at separating the wheat from the chaff. It introduces a small set of parallel constructs each with few if any options and presents a simple model for analyzing performance and correctness. Indeed I think that divide-and-conquer parallelism, parallel loops, analyzing performance in terms of work and span, and analyzing correctness using sequential semantics are perhaps the four most important concepts in deterministic parallel programming. In summary I believe that when parallel programming is brought to the right level of abstraction it is not inherently difficult. It requires parallel thinking, simple programming constructs, and limited use of non-determinism. ]]></description>
      <link>http://software.intel.com/en-us/articles/is-parallel-programming-hard-1/#comment-37671</link>
      <pubDate>Sun, 03 Jan 2010 23:54:09 -0800</pubDate>
      <guid isPermaLink="true">http://software.intel.com/en-us/articles/is-parallel-programming-hard-1/#comment-37671</guid>
    </item>
    <item>
      <title>By 
    Twitter Trackbacks for
     
    Is Parallel Programming Hard? - Intel® Software Network 
    [intel.com]
    on Topsy.com
  </title>
      <description><![CDATA[ n/a ]]></description>
      <link>http://software.intel.com/en-us/articles/is-parallel-programming-hard-1/#comment-50846</link>
      <pubDate>Mon, 25 Oct 2010 22:47:48 -0700</pubDate>
      <guid isPermaLink="true">http://software.intel.com/en-us/articles/is-parallel-programming-hard-1/#comment-50846</guid>
    </item>
    <item>
      <title>By Complexity as Enabler &amp;laquo; Gödel&amp;#8217;s Lost Letter and P=NP</title>
      <description><![CDATA[ n/a ]]></description>
      <link>http://software.intel.com/en-us/articles/is-parallel-programming-hard-1/#comment-64035</link>
      <pubDate>Thu, 08 Sep 2011 12:26:25 -0700</pubDate>
      <guid isPermaLink="true">http://software.intel.com/en-us/articles/is-parallel-programming-hard-1/#comment-64035</guid>
    </item>
  </channel></rss>
