Today at Intel Developer Forum (IDF) in Brazil, Intel announced that the Intel® Software Partner Program and four Intel® Software Network developer communities are launching in Portuguese. The Intel Software Network provides hundreds of technical documents and guidance on how to maximize software performance on Intel® architecture. The Intel® Software Partner Program helps companies develop and market commercial applications optimized for Intel® technologies.

Each community has a local community manager who will work with developers across the country to help build best-in-class solutions and end-user experiences.

• George Silva: Community manager for Ultrabook™, Consumer Client, and Android* Developer Communities
• Jomar Silva: Intel® vPro™ Developer Community
• Luciano Palma: Server Community and Parallel Programming Developer Community

Opportunities for Brazilian Software Companies to Partner with Intel
The Intel Software Partner Program will provide local marketing and sales support to drive campaigns with Brazil’s 300,000+ independent software vendors and 73,000+ software and services companies. Launching the program in Portuguese is an important step to connecting with Brazilian software companies for these new campaigns.

Future Investments in Brazil
Intel’s software programs are focused on building strong businesses through best-of-class products. In 2012, Intel will grow its in-country foot print as well as its corporate support teams to drive programs with universities and technology parks.

In a sequential program, it is not a big problem since the calculation order is exactly specified so the result is predictable and repeatable. The situation is not so clear in parallel programming.

To make the example parallel, I used the parallel_reduce template function from Intel® Threading Building Blocks (Intel® TBB):

std::vector<float> arr( N, 1.0f/(float)N );
float sum = tbb::parallel_reduce( tbb::blocked_range( arr.begin(), arr.end() ), 0.0f,
    []( const tbb::blocked_range& r, float sum ) {
        return std::accumulate( r.begin(), r.end(), sum );
    },
    std::plus<float>() );
std::cout << sum << std::endl;

As in the examples above, the code calculates the sum of N fractions, but it uses multiple processor cores if available. As it is well known, we face a disappointing fact of different results being possible for different orders of calculations. If we run it 10 times and N=1000 we will get something like this:

0.999991
1
0.999999
0.999996
0.999998
0.999998
0.999998
1
0.999997
0.999998

It’s worth mentioning that the result differs from run to run! In spite of the fact that the developer specifies the calculations – when it is calculated in parallel the order of calculation gets out of control.

On the other hand, it is not as bad as all that. Although the OS operates on threads and fills the application with indeterminism, it is still possible to manage the order of calculations. One of the new features of Intel TBB 4.0 is the parallel_deterministic_reduce template algorithm. The algorithm has the same interface as parallel_reduce except that it does not allow you to specify a partitioner. (For parallel_reduce it is possible to pass a partitioner as the last argument.) We will discuss why this restriction exists later. But for now, let’s replace the parallel_reduce with parallel_deterministic_reduce and look at how the result changes:

std::vector<float> arr( N, 1.0f/(float)N );
float sum = tbb::parallel_deterministic_reduce( tbb::blocked_range( arr.begin(), arr.end() ), 0.0f,
    []( const tbb::blocked_range& r, float sum ) {
        return std::accumulate( r.begin(), r.end(), sum );
    },
    std::plus<float>() );
std::cout << sum << std::endl;

Again run it 10 times:

1
1
1
1
1
1
1
1
1
1

The key point here is that the result is the same from run to run.

The sources of non-determinism in parallel_reduce derive from partitioning and body splitting. Let’s consider each of these subjects:

• Partitioning. The simple_partitioner determines exactly how many and which subranges are created. It splits the iteration range until each subrange is smaller than a given grain size. Thus the behavior only depends on the range size and grain size specified by the developer. However, other types of partitioning in Intel TBB are non-deterministic: to improve performance of the algorithms, range splitting provided by these partitioners depends on run-time stealing events, which we cannot predict.

• Body splitting. For performance reasons parallel_reduce minimizes body copies: it splits the body only when consecutive subranges are processed by different threads. Thus body splitting, like “advanced” partitioning, also depends on non-deterministic task stealing.

The example shows that parallel_reduce is really inapplicable for non-associative operations like floating point arithmetic. To achieve a repeatable result from a reduction with non-associative operations parallel_deterministic_reduce has been developed. From the considerations of partitioning (given above), it follows that only the simple_partitioner can be used for parallel_deterministic_reduce; and thus, no choice of an alternative partitioner is possible. Consequently, parallel_deterministic_reduce always challenges us with choosing an appropriate grain size. And smart body splitting has been disabled for the sake of deterministic behavior, so for each subrange a new body is created. This fact complicates the challenge of grain size selection even more: on the one hand, a small grain size increases the number of body copying and overall overhead, but on the other hand, a big grain size may lead to imbalance and underutilization. Fig. 1 shows the relative performance of parallel_deterministic_reduce (simple_partitioner with various grain sizes) in comparison with parallel_reduce (auto_partitioner with default grain size). An appropriate grain size provides the same performance of parallel_deterministic_reduce as parallel_reduce, - but an incorrectly chosen grain size may lead to significant performance degradation, as shown in Fig.1 at the extremes of the grain size axis.

Fig.1. Comparison of parallel_reduce (auto_partitioner) and parallel_deterministic_reduce (simple_partitioner) on Pi calculation example.

To demonstrate the split-join order behavior of parallel_deterministic_reduce, a small example is given with range [0, 20) and grain size = 5, similar to examples for parallel_reduce in the Intel TBB Reference manual:

A tree of subranges

For each right node a new body is created by the body split constructor. The slash marks (/) in the tree show where the body split is performed. Thus, for the current example the parallel_deterministic_reduce will always produce 4 subranges and 4 different bodies associated with them. Each of these subranges may be executed in parallel. When both children of a node finish, the corresponding bodies are merged: the right child body “added” to the left child body (in our examples via the std::plus<float>() binary function).

To conclude, parallel_deterministic_reduce provides a deterministic number and deterministic sizes of subranges, and it exactly defines which pairs of subranges are merged. It’s important to note that a repeatable result obtained with parallel_deterministic_reduce may still be different from that obtained via serial execution. Moreover, the results may be different for various grain sizes, since range splitting depends on the grain size. Also, the algorithm is not targeted to improve the accuracy of calculations. The exact result of 1 in the above example of fraction sum calculation has been obtained by chance. For other examples the algorithm can cause a decrease in accuracy. Overall, parallel_deterministic_reduce is not a replacement to parallel_reduce but an alternative solution for those who need repeatability.

-Mike

In support of the recent announcement of the 3^rd Generation Intel® Core™ Processors, Intel has released the Intel® SDK for OpenCL* Applications 2012. For the first time, OpenCL* developers using Intel® architecture can utilize compute resources across both Intel® Processors and Intel® HD Graphics Driver 4000/2500

From a person who, for the last couple of years has closely followed the emergence of the OpenCL standard, this announcement was something worth waiting for.  Less than a year ago, on this blog, I posted the news that the Intel® OpenCL SDK 1.1 gold  was released,  This was the first production OpenCL implementation from Intel targeting Intel® processors on Windows* OS. This current announcement is special, the Intel SDK for OpenCL Applications 2012 now supports not only the CPU but also the Intel HD Graphics 4000/2500 for Windows* 7 users.  We’ve come a long way in a year.

OpenCL on the 3^rd Generation Intel® Core Processor Family extends Intel’s line of tools and APIs on Intel platforms and adds interoperability with other graphics APIs like DirectX*, OpenGL* and Intel® Media SDK, directly on the Intel HD Graphics device.

So what else is new in this release?

• A Single OpenCL* platform enables shared context for OpenCL applications running on both the CPU and Intel HD Graphics 4000/2500. The OpenCL platform with both CPU and HD Graphics devices is available seamlessly on the Intel® HD Graphics Drivers.
• Interoperability with the Intel Media SDK with no memory copy overhead
• Improved performance for OpenCL applications running on Intel® Xeon® Processors and Intel® Core™ Processors. This CPU support is also available for Linux* OS developers.
• Intel® SDK for OpenCL* applications development tools includes an offline compiler and a step-by-step OpenCL Kernel debugger (for CPU) integrated in Microsoft Visual Studio* 2010 integrated development environment.
• 10 OpenCL code samples, three of them new, are now available for independent download.

The list above is just a sample of what is available with this new SDK. I recommend you read the product brief or watch the introduction video to get started with this new SDK.

Download the SDK for free at www.intel.com/software/opencl and begin optimizing your applications for the 3^rdGeneration Intel® Core™ Processors today.

Don’t forget to follow us on Twitter at @IntelOpenCL

What seemed most disturbing to me was the limit on the number of cores being self-inflicted. Surely we can't have reached the maximum number of cores that are possible to squeeze onto a chip? The whole "right turn" idea to add cores rather than try to cool processors reaching rocket engine temperatures was less than 10 years ago. I'm not sure where the physics starts to overshadow Moore's Law, but I thought I'd  heard that a few more generations of smaller wire sizes in processor dies were still possible. So why not push more and more cores into the same package?

It might be that the average server application (and, perhaps even more so, consumer applications) can't scale well beyond some fixed number of cores. How many cores does it take to type and post a tweet or update your Facebook status or to watch a streaming video? Would any of those tasks be faster or somehow enhanced if there were twice the number of cores available?

If we stop increasing the core counts in the next 5 years, how will new chips keep fulfilling the ever-growing hunger for more performance by consumers? Maybe it won't be about faster and faster application exeuction, but more about less energy consumption while maintaining a level of performance. I guess at some point we'll stop being concerned about Gigahertz or core counts because all processors will be able to do many of the same tasks in about the same amount of time.

I do know that power consumption is going to be a major driving design force as HPC moves closer toward Exascale platforms.  Thus, if the THX-1138 processor draws power twice as fast as the CFM602 processor, I would be more likely to build my system equipped with the former.

But what really caught my imagination was the first section of Chapter 7, "Programming Languages," that briefly described the Quantum Random Access Machine (QRAM) model of quantum computation. In addition to the few paragraphs that were devoted to this model, there was a picture that showed the relationship of a classic computer to a quantum computing device. Each part was simply a box with data/instructions passed from the classic to the quantum and data (results) passed from quantum to the classic side.

This setup looked familiar and it came to me during my cool down: this is how a system equipped with MIC would work. That is, your Intel Core processor does some initial computation to set up data, the data is passed over to the MIC (along with the computation instructions to be executed), and the results from the MIC can be returned to the Core side for use.

I know that MIC processors (and other GPU-like devices) don't have the same computational power as a quantum processor could have. However, the data-parallel and SIMD execution modes are similar to how a quantum device could take a superposition of all potential input data and execute a single computation step to arrive at a measurable result. This similarity got me thinking that MIC devices could be the first steps taken by the industry to better understand, prepare for and program effective quantum computations.

I don't know if we will ever see commodity quantum computation devices. I doubt they'll be developed within my lifetime, at least. Even so, I am nothing short of astounded when I look back at how far computer technology has come since I wrote my first COBOL program on an IBM mainframe. 

Knowing I should "never say never," how about on the day after I get my qPad(TM) quantum tablet device, I come back and comment on this blog post to say I was mistaken about how quickly quantum computation entered our lives? If it's anywhere in the cloud-o-sphere (and you know once these bits get pushed out, they never go away), I'll find it with the qSearch app, which will be based on the algorithm outlined in section 6.4 of Yanofsky's and Mannucci's book.

No, it was the fact that Gordon is the world's first supercomputer that uses flash memory instead of disk drives. This is the world’s largest thumb drive (300 Terabytes across 1024 Intel 710 series drives), as Allan Snavely, SDSC Associate Director, noted. There are quite a few advantages for using flash memory drives in place of spinning disks including lower power consumption, lower latency to access data, and fewer moving parts that can have mechanical failure. This just seems so logical that I'm surprised it hadn't happened sooner.

On a coincidental note, I saw a report earlier today that stated Intel was lowering Q4 revenue projections due to a drop in microprocessor demand from PC manufacturers. This is a direct result of not having enough disk drives available, which was caused by flooding in Thailand earlier this year.

I would think that the expected hard drive shortage will open the doors for wider adoption of SSD drives in the PC market and provide a hefty revenue stream for companies that can supply those drives. Big-scale projects like SCSD's Gordon computer just reinforce the efficacy of SSDs in desktops and laptops and even smaller form factors.

During the "SC11 season," a number of tool vendors announced they will be providing versions of their software tailored to supporting MIC architecture, starting with the Knights Corner co-processor.

Here are the ones I know about and can share (there are more who will make their own announcement in the future):

• IMSL Library, Rogue Wave
• NAG Libraries, NAG Ltd.
• Platform HPC, Platform LSF and Platform Cluster Manager, Platform Computing
• PBS Works, Altair
• Totalview debugger, Rogue Wave

[editor's note... additional announcements (post-SC11) include:

• CAPS directive-based HMPP compiler, CAPS

]

Of course... Intel tools as well, many of which we on display at the show in conjunction with Knights Ferry platforms.

There were also countless applications, many open source, that have been recompiled for MIC architecture and were being discussed around the show. Some I remember are NWChem, ENZO, ELK, MADNESS, MPI, GA, and Python... and I know I forget quite a few. Of course, Linux has been ported (and is running on both Knights Ferry and Knights Corner).

For additional tools announcements, please let me know, or post a comment!

NAG is one of a handful of companies that have been providing feedback on our Knights Ferry (prototype MIC architecture).

Mike told me: "We found porting existing routines from the NAG Library to the Intel Many Integrated Core Architecture (MIC) to be a relatively quick and painless process. The team was impressed at the way Intel has extended their existing software tools to support the MIC environment, allowing them to work in a familiar and productive environment."

I quizzed Mike on what it took to get it running on Knights Ferry, and he did share the one type of tuning they have to explore. Since they use OpenMP which has generally meant that the number of threads is more like 10-20 instead of the 120 threads they use on Knights Ferry.  I'll have to write more about that later - scaling and vectorization are keys as multicore and many-core grow. No mystery there. The good news is that their use of OpenMP made this a straightforward challenge they understood. It was not a mystery to them. They also make good use of MKL in their library as well, and of course we support that for MIC architecture.

It is great to know that NAG users will have the opportunity to continue using NAG software with Knights Corner.