The third myth is: "Dynamic analysis performed by tools like valgrind for C/C++ is much better than static code analysis".

The statement is rather strange. Dynamic and static analyses are just two different methodologies which supplement each other. Programmers seem to understand it, but I hear it again and again that dynamic analysis is better than static analysis.

Let me list advantages of static code analysis.

Diagnostics of all the branches in a program

Dynamic analysis in practice cannot cover all the branches of a program. After these words, fans of valgrind tell me that one should create appropriate tests. They are right in theory. But anyone who tried to create them understands how complicated and long it is. In practice, even good tests cover not more than 80% of program code.

It is especially noticeable in code fragments handling non-standard/emergency situations. If you take an old project and check it with a static analyzer, most errors will be detected in these very places. The reason is that even if the project is old, these fragments stay almost untested. Here is a brief example to show you what I mean (FCE Ultra project):

fp = fopen(name,"wb");
int x = 0;
if (!fp)
  int x = 1;

The 'x' flag will not be equal to one if the file wasn't opened. It is because of such errors that something goes wrong in programs: they crash or generate meaningless messages instead of adequate error messages.

Scalability

To be able to check large projects through dynamic methods regularly, you have to create a special infrastructure. You need special tests. You need to launch several instances of an application in parallel with different input data.

Static analysis is scaled several times easier. Usually you need only a multi-core computer to run a tool performing static analysis.

Analysis at a higher level

One of the advantages of dynamic analysis is that it knows what function and with what arguments is being called. Consequently, it can check if the call is correct. Static analysis can't know it and can't check arguments' values in most cases. This is a disadvantage of this method. But static analysis performs analysis at a higher level than dynamic analysis. This feature allows a static analyzer to detect issues which are correct from the viewpoint of dynamic analysis. Here is a simple example (ReactOS project):

void Mapdesc::identify( REAL dest[MAXCOORDS][MAXCOORDS] )
{
  memset( dest, 0, sizeof( dest ) );
  for( int i=0; i != hcoords; i++ )
    dest[i][i] = 1.0;
}

Everything is good here from the viewpoint of dynamic analysis, while static analysis gives the alarm because it is very suspicious that the number of bytes being cleared in an array coincides with the number of bytes the pointer consists of.

Here you are another example from the Clang project:

MapTy PerPtrTopDown;
MapTy PerPtrBottomUp;
void clearBottomUpPointers() {
  PerPtrTopDown.clear();
}
void clearTopDownPointers() {
  PerPtrTopDown.clear();
}

Is there anything here dynamic analysis may find suspicious? Nothing. But a static analyzer can suspect there is something wrong. The error is this: inside clearBottomUpPointers() there must be this code: "PerPtrBottomUp.clear();".

The fourth myth is: "A static analyzer must enable users to add user-made rules. Programmers want to add their own rules."

No, they don't. They actually want to solve some tasks of searching for particular language constructs. It is not the same thing as creating diagnostic rules.

I have always answered that implementation of own rules is not the thing programmers actually want. And I never saw any other alternative than implementing diagnostic rules by the analyzer's developers at the request of programmers (an article on the subject). I have had a fruitful conversation with Dmitry Petunin recently. He is the director of an Intel department of compiler testing and software verification tool development. He enlarged my understanding of this subject and voiced the idea I had been pondering over but failed to give the final formulation of.

Dmitry confirmed my belief that programmers wouldn't write diagnostic rules. The reason is very simple - it is very hard. Some static analysis tools enable users to extend the rule set. But it is done rather as a pure formality or for convenience of the tool's developers themselves. You need to know the subject very deeply to be able to develop new diagnostic rules. If an enthusiast without skill starts creating them, they will be of little use.

My understanding of the issue was over at this point. Dmitry, being more skilled than I, helped me learn more. In brief, this is how the situation looks.

Indeed, programmers want to be able to search for some particular patterns/errors in their code. They really need it. For example, someone needs to find all the explicit conversions of the int type to float. This task cannot be solved by such tools as grep, since it is unknown what type the FOO() function will return in a "float(P->FOO())" -like construct. At this moment the programmer comes to the idea that he/she can implement search of such constructs by adding his/her own check into the static analyzer.

This is where the key point lies. The programmer does not need to create his/her analysis rules. He/she needs to solve a particular issue. What he/she wants is a very small task from the viewpoint of static analysis mechanisms. It is like using a car to light cigarettes with its cigarette lighter.

That's why both Dmitry and I don't support the idea of providing users with API to handle the analyzer. It is an extremely difficult task from the viewpoint of development. Besides, people will hardly use more than 1% of it. So, it's irrational. It is easier and cheaper for a developer to implement users' wishes than create a complex API for add-ons or a special language of rule description.

The readers may say: "then make only 1% of the functionality in API available, and everyone will be happy". Yes, right. But look where the emphasis has moved: from developing own rules we have come to the idea that we just need a tool similar to grep but possessing some additional information about program code.

There is no such a tool yet. If you want to solve some task, write to me, and we will try to implement it in the PVS-Studio analyzer. For example, we have recently implemented several requests on searching for explicit type conversions: V2003, V2004, V2005. It is much easier for us to implement such wishes than create and maintain an open interface. It's also easier for users themselves.

By the way, such a tool might appear some time later within the scope of Intel C++. Dmitry Petunin said they had discussed a probability of creating a grep-like tool possessing knowledge about code structure and variable types. But it was discussed just in theory. I don't know whether or not they really intend to create this tool.

The fifth myth: "You can easily evaluate capabilities of a static analyzer on a small test code".

This is how this statement looks in discussions on forums (this is a collective image):

I've written a special program, its size is 100 code lines. But the analyzer doesn't generate anything although all the warning levels are enabled. This [tool of yours] / [static analysis] in general is just rubbish.

It is not the static analysis methodology which is rubbish, but this approach to evaluating the usability of a particular tool. The incorrectness of this kind of tool studying consists of two aspects:

1.

Programmers think they don't make simple mistakes. This phenomenon was discussed in Myth 2. So they try to feed an analyzer with a tricky sample and feel happy secretly when the analyzer can't find the error. This game is interesting yet senseless.

You should understand that most errors are simple as hell, and static analyzers detect them very well. The paradox is that it's much more difficult to invent a simple mistake than a complicated one. Here you are an example. Can you ever guess to write a sample like this?

int threadcounts[] = { 1, kNumThreads };
for (size_t i = 0;
     i < sizeof(threadcounts) / sizeof(threadcounts); i++) {

I doubt. I cannot imagine one can make such a silly mistake and write "sizeof(threadcounts) / sizeof(threadcounts)". So, such an example will never be created on purpose. By the way, this fragment is taken not from a student's lab work, but from the Chromium project. It is diagnosed by the PVS-Studio analyzer very easily, of course.

2.

Written samples are of random character, and they are few. So you may get very different results depending on chance. You may invent 5 errors that will be successfully found by one analyzer and not found by another analyzer. Or you may create a program with five errors, and two analyzers will give opposite results for it. The sampling for such an investigation is too small. To be able to compare and study tools with at least somewhat reliable results, you must write a program text with at least 500 different errors. An investigation based on 5-10 errors is not reliable.

Moreover, programmers expect to see diagnostic messages on errors of some particular type and forget about the rest. For example, almost all the programmers write one and the same sample with a memory release defect:

void Foo()
{
  int *a = (int *)malloc(X);
  int *b = (int *)malloc(Y);
  //...
  free(a);
}

Some analyzers detect this error, the others don't. For instance, PVS-Studio does not diagnose memory leaks currently. But it can find the following stuff:

static int rr_cmp(uchar *a,uchar *b)
{
  if (a[0] != b[0])
    return (int) a[0] - (int) b[0];
  if (a[1] != b[1])
    return (int) a[1] - (int) b[1];
  if (a[2] != b[2])
    return (int) a[2] - (int) b[2];
  if (a[3] != b[3])
    return (int) a[3] - (int) b[3];
  if (a[4] != b[4])
    return (int) a[4] - (int) b[4];
  if (a[5] != b[5])
    return (int) a[1] - (int) b[5];
  if (a[6] != b[6])
    return (int) a[6] - (int) b[6];
  return (int) a[7] - (int) b[7];
}

There must be "return (int) a[5] - (int) b[5];" instead of "return (int) a[1] - (int) b[5];".

Why does nobody write such examples? Note that PVS-Studio has found this error in the MySQL project.

The conclusion is, adequate investigation or comparison of tools can be carried out only with real projects. You take project A, test it with PC-Lint / Visual C++ / PVS-Studio / C++Test, study all the messages attentively, draw up a table of results (how many and which errors each analyzer has found). This is the only real investigation and comparison. For example: "Comparing the general static analysis in Visual Studio 2010 and PVS-Studio by examples of errors detected in five open source projects ".

In my blog “Transactional Synchronization in Haswell,” I describe new instructions (Intel TSX) that will improve the performance of coarse-grained locks.  Understanding coarse-grained locks and the concept of transactions are both key to understanding why Intel TSX matters.

Intel TSX may enhance performance of mutual exclusion other than simple coarse-grained locks, but I will focus on coarse-grained locking because it is common and Intel TSX allows highly concurrent accesses using only a simple locking mechanism.

An example

To motivate by illustration, let’s consider a simple hash table. Hash tables are used to map a key to a key and value pair in linear time. Two key operations are add (insert) and lookup (retrieve). Resizing and deletion are two additional operations of general interest also, but I will leave them for another time.

Designing a highly concurrent hash table is a non-trivial task, and there are many approaches to allow high levels of concurrency.  All these approach add complexity to the program, and often to the data structures themselves.

The simplest approach is a single lock approach. In such an approach, every operation on the hash table starts by obtaining the lock for the table and concludes by releasing the lock. While the lock is held for the operation, no other task on the system can obtain the lock and therefore no hash table operation is allowed to proceed.

Considering Figure 1, no concurrent operations are allowed, so each of the five operations shown would occur one at a time.

Figure 1: Five hash table operations requested

Solutions

A common solution is to break the hash table into smaller regions, and have locks that apply to regions. While this can reduce contention, it still can create needless delays and it definitely complicates the coding and the data structure.

Such an approach is a prime example of taking a coarse-grained lock (a single lock for the entire hash table) and working to make it a finer grained lock (multiple locks for smaller table sections). Coarse-grained locks are easier to use, easier to understand and easier to debug.  The only disadvantage is that they tend to impede performance in a multithreaded environment. Multicore processors are increasing the likelihood of this being a problem, and help motivate new hardware assistance so that programming has a chance to stay simple more often than without assistance.

Transactional Synchronization (Intel TSX) as a solution

What would be ideal, is to use the single lock (coarse-grained locking) because it is easy and not very error prone, but still have the performance of a fine-grained implementation. In our Figure 1 example, only one operation conflicts with another. This example does have more conflicts that would be expected in a real world example.

Considering this example, three of the operations have no collision with the other operation so the use of HLE (part of Intel TSX) on the single lock will completely elide the lock. In other words, the performance is very close to the performance of the code if no locking or unlocking code was present. The key however is that the operations are protected by the Intel TSX hardware, which has silently ensured that the protection intended by the lock is indeed assured.

The two operations that map to the same hash table entry will need to be staggered. This will occur even if we are unlucky enough to have them happen at the same time. In such a case, the Intel TSX will detect that the lock was indeed needed and some locking overhead will be incurred. What would actually happen in such a case, is that the colliding tasks will proceed into the protected code until the processor detects the conflict. As such a point, both updates will abort their protected code (also called the transaction). The most common solution then is to have each task proceed but actually enforce the lock on the second try. This means that one task will win, and delay the other, until the operation is complete. The precise decision on how to handle the collision is either up to the processor implementation with HLE, or the programmer with RTM. The processor implementation for HLE will also be fairly simple and conservative, in order to preserve the semantics of the original lock and hence compatibility with processors that lack Intel TSX.

Summary

For a hash map, Intel TSX allows for the right things to occur without losing the protection that the locks need to give. Intel TSX ensures the same results as the coarse-grained lock guarantees, but allows unrelated operations to proceed without delays that the coarse-grained locks would have caused. For more information on Transactional Synchronization, see my blog on Intel TSX.

Please check out the specification and stay tuned for information about supporting tools from Intel and others in the coming months.

In this blog, I’ll introduce Intel TSX and provide a little background. Please refer to The Transactional Synchronization Extensions Chapter (Chapter 8) in the manual for additional information. These new synchronization extensions (Intel TSX) are useful in shared-memory multithreaded applications that employ lock-based synchronization mechanisms.

In a nutshell, Intel TSX provides a set of instruction set extensions that allow programmers to specify regions of code for transactional synchronization. Programmers can use these extensions to achieve the performance of fine-grain locking while actually programming using coarse-grain locks. I have written a simple illustrative example in my blog “Coarse-grained locks and Transactional Synchronization explained.”

Locks are a low-level programming construct (close to the hardware), so any discussion of Intel TSX will be low level too. How Intel TSX might affect higher-level programming methods, or enable new programming models, is beyond the scope of my blog but I will briefly comment on it at the end of this blog.

Why is this useful?

With transactional synchronization, the hardware can determine dynamically whether threads need to serialize through lock-protected critical sections, and perform serialization only when required. This lets the processor expose and exploit concurrency that would otherwise be hidden due to dynamically unnecessary synchronization.

At the lowest level with Intel TSX, programmer-specified code regions (also referred to as transactional regions) are executed transactionally. If the transactional execution completes successfully, then all memory operations performed within the transactional region will appear to have occurred instantaneously when viewed from other logical processors. A processor makes architectural updates performed within the region visible to other logical processors only on a successful commit, a process referred to as an atomic commit.

These extensions can help achieve the performance of fine-grain locking while using coarser grain locks. These extensions can also allow locks around critical sections while avoiding unnecessary serializations. If multiple threads execute critical sections protected by the same lock but they do not perform any conflicting operations on each other’s data, then the threads can execute concurrently and without serialization. Even though the software uses lock acquisition operations on a common lock, the hardware is allowed to recognize this, elide the lock, and execute the critical sections on the two threads without requiring any communication through the lock if such communication was dynamically unnecessary.

Intel TSX Interfaces

Intel TSX provides two software interfaces. The first, called Hardware Lock Elision (HLE) is a legacy compatible instruction set extension (comprised of the XACQUIRE and XRELEASE prefixes) that are used to specify transactional regions. HLE is compatible with the conventional lock-based programming model. Software written using the HLE hints can run on both legacy hardware without TSX and new hardware with TSX. The second, called Restricted Transactional Memory (RTM) is a new instruction set interface (comprised of the XBEGIN, XEND, and XABORT instructions) that allows programmers to define transactional regions in a more flexible manner than is possible with HLE. Unlike the HLE extensions, but just like most new instruction set extensions, the RTM instructions will generate an undefined instruction exception (#UD) on older processors that do not support RTM. RTM also requires the programmer to provide an alternate code path for when the transactional execution is not successful.

In summary: “Intel Transactional Synchronization Extensions (Intel TSX) comes in two flavors: HLE and RTM. Hardware Lock Elision (HLE) is legacy compatible. Restricted Transactional Memory (RTM) offers flexibility but requires the programmer to provide an alternative code path for when transactional execution is not successful.”

The specification describes these extensions in detail and outlines various programming considerations to get the most out of them.

Intel TSX Applicability

Intel TSX targets a certain class of shared-memory multi-threaded applications; specifically multi-threaded applications that actively share data. Intel TSX is about allowing programs to achieve fine-grain lock performance without requiring the complexity of reasoning about fine-grain locking.

However, if there is high data contention the algorithm would need to change in order to have an opportunity for high scalability. There are no magic bullets that can solve the problem, since true high data contention implies that the algorithm is effectively serialized.

Transactional Programming?

How Intel TSX might affect higher-level programming methods, or enable new programming models, is beyond the scope of my blog. Several experimental compiler implementations, not related specifically to Intel TSX, are available including gcc 4.7 which will have an experimental implementation. We can expect languages standards committees will be reviewing proposals on how to add transactional models at a language level (Intel has supported the creation of the Draft Specification of Transaction Language Constructs for C++). Intel TSX may enable a more efficient implementation of some transactional models than without Intel TSX. Much work remains to focus on real-world examples of usages and applications to develop and refine future usage. Good luck to all involved!

While Intel TSX may enable efficient implementations of new programming models, it does not require a new programming model and does not propose a new programming model. Intel TSX provides hardware-supported transactional-execution extensions to ease the development and improve the performance of existing programming models.

Summary

Intel TSX provides extensions that allow programmers to specify regions of code for transactional execution. Programmers can use these extensions to achieve higher performance with lesser effort, for example achieve fine-grain locking performance while programming with coarser-grain locks. This is a big help and therefore big news for programmers.

Please check out the specification and stay tuned for information about supporting tools from Intel and others in the coming months.

What seemed most disturbing to me was the limit on the number of cores being self-inflicted. Surely we can't have reached the maximum number of cores that are possible to squeeze onto a chip? The whole "right turn" idea to add cores rather than try to cool processors reaching rocket engine temperatures was less than 10 years ago. I'm not sure where the physics starts to overshadow Moore's Law, but I thought I'd  heard that a few more generations of smaller wire sizes in processor dies were still possible. So why not push more and more cores into the same package?

It might be that the average server application (and, perhaps even more so, consumer applications) can't scale well beyond some fixed number of cores. How many cores does it take to type and post a tweet or update your Facebook status or to watch a streaming video? Would any of those tasks be faster or somehow enhanced if there were twice the number of cores available?

If we stop increasing the core counts in the next 5 years, how will new chips keep fulfilling the ever-growing hunger for more performance by consumers? Maybe it won't be about faster and faster application exeuction, but more about less energy consumption while maintaining a level of performance. I guess at some point we'll stop being concerned about Gigahertz or core counts because all processors will be able to do many of the same tasks in about the same amount of time.

I do know that power consumption is going to be a major driving design force as HPC moves closer toward Exascale platforms.  Thus, if the THX-1138 processor draws power twice as fast as the CFM602 processor, I would be more likely to build my system equipped with the former.

-------------
First, information about the webinar. I will be hosting this with my colleague Wendy Doerner on Feb 15th at 9AM PST. We will cover how to get started with vectorization, including examples and resources. To register or view the abstract, click this link:
https://www1.gotomeeting.com/register/761784545
If you attend the event live, you will also have the opportunity to request a followup from one of our vectorization experts!

-------------
For the blog series I will answer 3 questions:
What is Vectorization?
Who Can Use It?
What Are the Benefits?

Today I will start with the first question: What is Vectorization?

Vectorization is a method for achieving parallelism inside a single processor core. Vectorizing is done by using special instructions called SIMD (Single Instruction, Multiple Data) operations. SIMD instructions, and the hardware that goes along with them, have been present in Intel processors for over a decade. (Remember those commercials in the mid-90s with people dancing in bunny suits promoting MMX™ Technology? MMX was a set of SIMD instructions). The way that SIMD instructions work is that they operate on several pieces of data in parallel.
In the typical (non-vectorized) case, when you add together 2 variables, they will each be stored in their own CPU register. If you perform an operation on them, such as addition, the 2 register quantities are added and the result stored back into a register. Using a SIMD instruction, you can fill a register with multiple variables to be added, which is called "packing" the register. With the most recent SIMD instruction set, Intel® Advanced Vector Instructions (Intel® AVX), which are available on Intel® Microarchitecture Codename Sandy Bridge processors, you can pack up to 32 data elements into one register. The number of elements allowed depends on the size of the element - in Intel® AVX, for example, registers are 256 bytes wide, so each can hold 32 8-byte integers, or 8 32-byte floats, or 4 64-byte floats, etc. These data elements can all be combined with another packed register full of elements, allowing you to perform multiple operations on multiple pieces of data at once. For instance, adding 2 packed SIMD registers would produce multiple results, which would be stored into a packed register as well. Being able to do these operations at once rather than one right after the other can result in significant performance gains for the right type of code.

And addition is not the only operation possible on a packed register! Each set of SIMD instructions includes many different operations, with more being added in upcoming processor generations.

But that takes us to the next topic, Who can use vectorization, which we'll cover in the next blog. Feel free to ask questions in the comments of this blog series too, I might turn the questions into future entries. Thanks for reading!

The authors look to evaluate which of three possible processor core architectures might be best for parallel execution that minimizes energy consumption. The three model core arrangements are 1) multi-core (several large processing cores on a single chip), 2) manycore (lots and lots of simpler, more power efficient cores), and 3) a combination of a single large core with many simpler cores. The first is like the current dual-, quad-, and hexa-core processors, the second is akin to a GPU, and the third is a hybrid conglomeration of large core sitting on a GPU.

For the purposes of the model formulas, the maximum power consumption of a single large core is normalized to 1 and the power consumption of an idle processor is an added variable, k. For the first architecture, the new variable is used in the traditional Amdahl's Law formula as a multiplier to the serial percentage of time multiplied by the (n-1) idle cores. Some simple algebraic manipulation and the authors generate a formula for estimating the average power consumption, in watts (W), for a parallel application with n cores and the stated percentages of parallel and serial work. A similar derivation is done for the manycore model with the power consumption per simple core being 0.25 of the large core. With the hybrid model, the assumption used to derive a corresponding formula is for the large single core to handle the serial execution and the simpler cores do the parallel work.

Since a measure of the watts consumed is provided by the model, the authors then compute performance per watt (Perf/W) by computing the original Amdahl's formula divided by the formula to compute W. In order to compare the three model to each other, a power budget is imposed, which sets the number of cores available for each model.

The conclusions drawn from comparing the model with various numbers of cores and fractions of parallel execution of the overall execution time are probably the most interesting part of the article. For example, the first result reported is that to achieve the highest Perf/W value in the multi-core model, the parallelization must scale linearly. If the application doesn't scale linearly,the processor (model) must dissipate more energy than the serial version since the idle power of the extra cores scales linearly.

The ultimate result of the paper was that the hybrid model, one large core and many small cores, was the most power scalable. The manycore option does well with high amounts of parallelism and lower power budgets (fewer total cores), but as that budget increases, the number of simple cores increases and the effective serial execution performance does not. The hybrid model, with the single large core in place of several simpler cores, can more efficiently handle the serial portions of the execution (than one simple core out of the dozens sitting idle).

As I was reading the paper I could identify which was the model of current standard multi-core processors available in abundance today. The manycore model could easily be a GPU or MIC accelerator by itself. The hybrid model suggested the combination of a manycore accelerator and a dual-core processor (in the absence of  heterogeneous core chips). I wondered where vector hardware fits into the three models. Considering just the vector registers alone might suggest it would be an instance of the manycore model. However, these registers are part of a larger core, which makes me think of the hybrid model. Maybe they are a second level of parallel execution that isn't accounted for in the three models.

It's a good paper. But I have a couple of quibbles. First, the only way that parallel execution on the multi-core model underachieves the serial equivalent execution is if the sequential code is run on a single core system. A later comment in the paper makes me think that this is the assumption, but it's not too clear. This assumption is not valid in the real-world. For a true apples-apples comparison, the serial code needs to be run on a multi-core processor, too. If that were the case, I contend that the parallel execution consumes less energy.

For example, assume that we have an execution time of 10 time units (let's call them moops) . On a quad core processor running the serial code we would have one core running full speed for 10 moops and the other three cores generate an aggregate 30 moops of idle consumption. If the algorithm is 50% parallel, we would have 5 moops of  full power consumption in serial, 5 moops of full consumption in parallel across four cores, and 15 moops total of idle consumption. Even if the code is 10% parallel there would only be 27 moops of total idle consumption. Any level of (perfect) parallelism is going to prove to consume less energy than the serial equivalent on the same system. Am I missing something?

Note that I included '(perfect)' at the end of the previous paragraph. There will always be overhead in parallel computations and this will expand the execution time of the parallel portions and, consequently, the full consumption time of the execution (e.g., the 50% parallel portion above might require 5.4 moops of full consumption).

Second, Amdahl's Law is an estimate of speedup. Speedup is a dimensionless number. That is, I divide the execution time of the serial code with the time of the parallel execution to get a simple  number since the moops of the two quantities cancel each other out in that calculation. If I need 10 moops of serial time versus 6.35 moops of parallel time, I get a 1.57X speedup. 1.57 whats? ('X' is not a unit.) Speedup is a metric of relative performance, but it's not what I really think of when I think of performance.

To me "performance" is more absolute. Typically this is some countable quantity like number of transactions, floating-point operations, or feet traveled. It can be associated within a time unit measure, too, like transactions per moop, floating-point operations per second, or furlongs per fortnight. Thus, the metrics of transactions per watt or flops per watt or feet per watt make sense to me. Improvements that raise the performance value or lower the watt value show a trend in the right direction for achieving better energy efficient performance.

I'm still not able to quite wrap my head around the efficacy of speedup per watt (or even speedup per joule, which is also used in the Woo and Lee paper) as an absolute measure of energy efficient performance. It may be that I'm reading too much into this and the metrics are simply used to compare the three architectural models described (within the assumptions given). Perhaps it is simpy just a model after all.

Upon receiving the corks, Tom would demonstrate his "cork trick" and mystify everyone that had not seen it before. After going through it three or four times he would hand the corks back to our server and have them try to do it. They would go away, sometimes showing others, as they struggled to figure out the trick. If they actually tried to recreate the solution as Tom had been able to do, they always came back before we had paid the check and triumphantly demonstrated their dexterity.

At SC11, the Educational Alliance for a Parallel Future (EAPF) commissioned some corks with the organization's logo. Tom wandered around part of the conference urging attendees to try his cork trick. The tagline he used was that "Parallel Programming is easier than the cork trick." You can see a short video of his efforts to bring a little magic to the SC11 proceedings here.

If you meet Tom with some corks in his pocket and he brings them out to show you the cork trick, be aware that he will never show you the solution. (He says he really likes me, but I had to figure it out for myself.) Like most problems you encounter in life, very few are impossible to solve; it is just that you don't have a solution, yet.

Parallel programming is the same. It may seem difficult and impossible to figure out, but that only means you haven't discovered the key that will allow you to wrap your brain around the concepts.

• Intel® Many Integrated Core Architecture (MIC) MIC (pronounced "Mike") is a new architecture that uses many low-power, single-threaded Intel® processor cores working together to provide a high degree of parallelism. As more customers begin using the Knights Ferry development kit and the developer tool package we currently have available for MIC, I will start studying the architecture and conducting some experiments too. My first project will be to try to identify the most important performance aspects of MIC applications and how our tools can help developers measure them.

• Vectorization
Vectorization is parallelism within one CPU core, using special hardware that can work on more than one piece of data at once. First off I plan to write a series of blogs explaining what vectorization is, who should be interested in it, how to achieve it, and why it's important. Then I hope to work with some customers directly on this. One reason why vectorization will be of big interest to me is that it will definitely be a big help to performance on the MIC architecture.

• Intel® Cilk Plus
Fitting hand in hand with the 2 focus areas above is Intel® Cilk Plus, our open source parallelism model that also includes vectorization support. Although I have worked with and evangelized Cilk Plus a lot this past year, I had mainly been looking at the parallelism part. Next year, I will spend more time on the vectorization part of Cilk Plus. The other reason to focus here is that Cilk Plus code, like TBB code, will also run on the MIC architecture.

So get ready to hear more from me on the above topics! And Happy Holidays!

Motivation

We had vague requests from customers to implement a memory pool (Wikipedia calls it region) or some of its properties in the TBB scalable memory allocator. We summarized these requests and general information on memory pools from the Internet and got the following compilation of major properties and abilities:

• Memory pools basically do the same job as standard memory allocators but additionally group memory objects under umbrella of a specific pool instance which enables:
  • fast deallocation of all the memory at once on pool destruction or for sake of further reuse
  • less memory fragmentation and related synchronization between independent groups
• Memory pools allow more control over acquisition and release of memory resources, and may have user-specific sources of memory:
  • memory chunk/buffer of a fixed size
  • redirection to a specific memory provider, e.g. standard or custom implementation of malloc, big memory pages, memory tied to specific NUMA node, IPC shmem regions.

To squeeze more performance and to fight memory fragmentation, some specific implementations allocate objects of fixed size only (so called object pools, e.g. boost::pool, Wikipedia calls it memory pool) or are unable to deallocate individual object ("arena allocator"). In our implementation, we tried to provide more general functionality in thread-safe and scalable way. For that purpose, the implementation of the memory pools is based on TBB scalable memory allocator and so has similar speed and memory consumption properties. Later we may address more specific use cases, based on the feedback.

Usage

Our memory pools API consists of two classes for thread-safe memory management: tbb::fixed_pool and tbb::memory_pool. The first one is for the simple case when an already allocated memory block and is used for allocation of smaller objects. And the second one utilizes a user-specified memory provider to obtain big chunks of memory where smaller objects reside. As opposed to fixed_pool, memory_pool is able to grow on demand and relinquish unused chunks back to the provider.

Both classes provide familiar methods for allocation and deallocation:

void *ptr = my_pool.malloc( (size_t) 10 );  // allocate 10 bytes
ptr = my_pool.realloc( ptr, (size_t) 12 );  // extend the allocation to 12 bytes
my_pool.free( ptr );                        // deallocate it

Additionally, there is a method which deallocates all the memory at once, i.e. it is a faster equivalent to a series of calls to my_pool.free() for each pointer obtained in this pool by previous calls to my_pool.malloc():

my_pool.recycle();  // Frees all the memory in the pool for reuse

Please note, that it is not thread-safe to call it concurrently to other methods on the same instance (similarly to clear() method in containers).
We also provide an (almost, except absence of default constructor) STL-compliant allocator class to enable pools inside STL containers:

typedef tbb::memory_pool_allocator<int> pool_allocator_t;
std::list<int, pool_allocator_t> my_list( (pool_allocator_t( my_pool )) );

Now, the only thing that holds us back from the first experiment with this new feature of TBB is the question – how to create the ‘my_pool’.  First, we need to enable this feature and include the header:

#define TBB_PREVIEW_MEMORY_POOL 1
#include “tbb/memory_pool.h”

If you want to create a memory pool on top of your memory block, let’s specify its address and size in bytes to the constructor of tbb::fixed_pool class, as in following excerpt:

char buffer[1024*1024];
// The casts below are just to show the types of arguments.
tbb::fixed_pool my_pool( (void*)buffer, (size_t)1024*1024*sizeof(char) );

The maximal amount of memory which can be allocated from the pool declared above is limited by size of the buffer minus some space for control structures. And if you want to avoid this limitation, let’s use tbb::memory_pool template class specifying memory provider (which will be discussed later) as its template argument:

tbb::memory_pool< std::allocator<char> > my_pool(/*optionally: allocator instance*/);

You can specify any STL-compatible allocator as the memory provider (though this is a subject to change). It will provide (big) memory chunks for  my_pool when necessary. The destructor of the memory_pool class implies release of all the memory chunks back to the memory provider.

Let’s consolidate our knowledge in one artificial example:

// Link this with tbbmalloc library
#define TBB_PREVIEW_MEMORY_POOL 1
#include "tbb/memory_pool.h"
#include <list>
#include <stdio.h>

void main() {
    static char buf[1024*1024*4]; // buffer for interim data
    tbb::fixed_pool interim_pool(buf, sizeof(buf)); // pool for temporary objects
    tbb::memory_pool< std::allocator<char> > result_pool; // pool to store the results

    typedef tbb::memory_pool_allocator<int> result_allocator_t; // interface to STL containers
    std::list<int, result_allocator_t> result_list( (result_allocator_t( result_pool )) );

    for(int result = 0, i = 0; i < 100; i++, result = 0) {
        for(int j = 0; j < 1000000; j++) {
            int *p = (int*)interim_pool.malloc(4);
            if( p ) result++; // really dummy :)
        }
        // in real application, here can be some processing of allocated objects
        result_list.push_back(result); // no memory fragmentation here - separate pool
        interim_pool.recycle(); // free all the interim objects
        printf("%d\n", result); // should be the same number on each iteration
    }
} // all the memory is released back implicitly

The simple part is done, and I hope that you are interested enough to proceed with more complex questions, and tell us what you think about it.

Someone may want to know whether it is possible to construct a pool in a memory allocated form another pool. It is possible, but one should take care to destroy the inner pool prior to destruction of the outer pool or a call to recycle(). Do you know a good reason to enable such a nesting?

Memory provider interface

From an API designer perspective, the memory provider is the most questionable part of the scalable pools API. And since it is yet a community preview feature, you are welcome to influence its design. Curious readers might want to ask questions like the following:

• what are the requirements for the template argument?
• why is std::allocator used as a memory provider?
• why the type used with std::allocator in examples above is “char”?

The template argument of tbb::memory_pool accepts a memory provider class which satisfies minimal requirements of STL compatible allocator according to the last C++11 standard: allocate and deallocate methods, and a value_type definition.

Using std::allocator and compatible classes is perhaps the most straight-forward way to enable memory_pool anywhere. However from efficiency standpoint, it makes probably not much sense because such allocators are intended for rather small objects by design while memory provider should operate with megabytes. For users who don’t care what the memory provider is, we could better provide a default one instead which would map to system-default way for memory mapping.

And finally, TBB memory pools don’t really need the type of allocation (i.e. char in the declaration of tbb::memory_pool<std::allocator<char>>), but rather need to know the granularity of requests to the memory provider. And this is not only specification for type of arguments for allocate and deallocate, this information is used in our implementation to determine size of memory requests to memory provider. For example, consider big pages which can be mapped only by chunks of megabytes:

// A custom memory provider for memory_pool
class big_pages {
public:
    typedef char[2*1024*1024] value_type;
    void *allocate(size_t pages) {
        return mmap(0x0UL, pages*2*1024*1024, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, 0, 0);
    }
    // the pointer type requirement is also actually relaxed
    void deallocate(void *ptr, size_t pages) {
        munmap(ptr, pages*2*1024*1024);
    }
};
// usage:
tbb::memory_pool<big_pages> my_pool;

Some food for thoughts

The way granularity is specified in the line 4 in the above example is not straight-forward and can be viewed as confusing. This is the price of STL-compliant interface of the memory provider and we are not sure if it has more pros than cons:

• STL compatibility is supposed to reuse widely implemented memory allocators.
  • On the other hand, these allocators are usually purposed for small sizes of allocations but a pool will need memory chunks of at least hundreds of kilobytes.
• In theory, it allows easy nesting of memory pools using our memory_pool_allocator class.
  • But we studied that nesting of the pool in some other implementations does not mean reusing the memory allocated by parent pool but rather a hierarchy of pool objects.
  • And such a nesting is not yet supported anyway
• It is easier to remember the requirements based on well-known standard interface
• Granularity is a property of the memory provider and must be passed along with it

As an alternative interface, we consider to make the granularity explicitly specified but in a separate trait class which should be specialized only for the memory providers with granularity of allocations > 1. It is even possible to keep STL-compatibility using metaprogramming magic, e.g. define the granularity to sizeof(value_type) if value_type defined.

Another question is how to introduce alignment in the interface of memory pools. Basically, it can be either aligned_malloc() and aligned_realloc(), or an optional argument for malloc() and realloc() methods.

Also, are the suggested class names good, or do we need to find better names (for instance, "fixed_region" and "dynamic_region" to align with terms of Wikipedia)?

Feedback is very welcome

We are very eager to hear from you what do you think about above and how can it be used in your projects.

5. Intel® Cilk Plus open source port to GCC - Intel® Cilk Plus was announced in 2010, and an open source specification has been out since late 2010 as well. However this year we began, along with the open source community, to port Cilk Plus to GCC. Some of the first items ported were the parallelism keywords, which is significant to me because it makes our Cilk Plus parallelism model available to a greater audience.

4. Intel® VTune™ Amplifier XE and Intel® Inspector XE MPI Support - In the new Cluster Studio XE product, VTune Amplifier XE and Inspector XE are now MPI-enabled. This is important because we are beginning to see more hybrid programming in the HPC and cluster world - which means the applications use a combination of MPI and another threading model (such as OpenMP, Cilk Plus, or Intel® Threading Building Blocks). We have an existing product, Intel® Trace Analyzer and Collector, that analyzes MPI efficiency for a cluster app, but analyzing performance of an individual process running on an MPI rank was more difficult. Now we make it easier to use VTune Amplifier XE or Inspector XE to analyze the threading model used within a rank, which helps us support more cluster customers.

3. Intel® Threading Building Blocks Flow Graph - I was introduced to flow graph this year, when I worked with my colleague Victoria Gromova to create some TBB labs for Intel Developer Forum. Victoria wanted to highlight flow graph as one of the new features of TBB 4.0. Flow graph is a new construct that supports many more types of control algorithms, like dependency graphs, event-based models or reactive-based flows. In short, it opens up TBB to more customers while maintaining or improving the TBB performance we have come to expect.

2. VTune Amplifier XE attach to running process on Linux* - This is a great example of our development team responding to customer feedback. Being able to analyze a running process for a defined period of time (instead of launching it) has been requested by many of our clients. We first got this implemented on Windows*, then this September provided the feature for Linux* in Intel® Parallel Studio XE Service Pack 1. I have already been visiting some users who requested this and it is great to be able to share that the feature they have been asking for is here!

1. VTune Amplifier XE interface for Intel® Microarchitecture Codename Sandy Bridge - For readers of my blog this one should not be a surprise! I have created quite a bit of training material on these new Sandy Bridge features. We now provide an analysis type for Sandy Bridge that helps users easily identify the most common software performance issues at the microarchitectural level, and it includes pre-coded metrics, thresholds, and issue highlighting for usability. This is my favorite new feature because, even though I am not a developer, I got to help a little with making this interface by helping define some performance metrics and thresholds and validating them on workloads. It is very cool to see my contributions in the product.

There you have it! I hope you have a chance to try out some of our new product features now or in the coming year. Let us know your favorites, or your requests.

I want to thank Kathy Farrel for all her hard work in organizing and taking the lead on the last 40 shows. She had some fresh ideas during our collaboration and I enjoyed working with her. Kathy was taking on the role of Parallel Programming Community Manager and it seemed like she was asking me at least one question every day about the subject.  She started out a bit camera shy and tongue-tied. But she made steady improvement, started to explore parallel programming topics on her own, and soon got comfortable with the hosting duties. I am impressed with her drive and tenacity. She will do more great things in her new role.

Aaron Tersteeg deserves a big "Thank You" for developing Parallel Programming Talk back in 2008, first on blogtalkradio and then as an Intel Software Network web video show. When he first approached me about participating, he described it as "Car Talk" but focused around parallel programming topics. We tried to keep things both informal and informative. I was relegated to Aaron's monitor when we started the video shows (as he is in Oregon and I am in Illinois), which led to some light-hearted moments, some accidents, and some experimentation as we played with this restriction.

(My favorite anecdote from the early video days was when Aaron ran into a fan of the show at a Portland Trailblazers game. The fan knew he looked familiar and then asked if he was Clay Breshears.)

The production crew will always have my undying respect and appreciation. Jerry Makare and Josh Bancroft ushered the video era into existence and have always driven the technology and production values to deliver a higher quality product. They were always up for a challenge and conquered many of them during the show's run. I also appreciate all the work that the video production interns--Chris Davis and Anthony Lopez--did for the show. They worked tirelessly behind the scenes moving, setting up, and tearing down equipment, they did some of the editing chores, and they were always great fun during those rare times I was in town for a live shoot.

And finally, I want to thank the fans and casual viewers of the show. Thank you for all your support, questions, and comments. Without you we would have shut down soon after we started. During my tenure as co-host of Parallel Programming Talk, I got to meet some of the superstars of the field, got to hear more about cool parallel languages and approaches to parallel programming, and got to see some cool tools that are useful in making parallel programming, debugging and tuning much easier. I hope that you enjoyed hearing from the experts and finding out about new technology at least half as much as I did.

Keep writing parallel code and be good to each other.

Before I sign off here (on PPT – I am, still here in a new role and my blog will continue) I want to thank my cohost Clay Breshears for his support in this ride. His expertise, encouragement and exceptional sense of humor contributed heavily to the success of the show. Thanks to our outstanding guests for sharing the info that has drawn our audience. Special thanks to our crew: Jerry Makare, Technical Director, Intel Software Videos, Producer Josh Bancroft and Videographer Chris Davis,  for early morning shoots, patience through the technical challenges and for their continuing support (new phrase - for the love of Skype).

Extra thanks to Aaron Tersteeg, whose brainchild "Parallel Programming Talk" was and is. He is one of the most forward thinking people I know anywhere, whose wisdom and energy fuels a great deal of the goodness here at the Intel Software Network.  

Enjoy the video:

Direct Video Link: http://software.intel.com/en-us/blogs/2011/12/14/the-last-show-parallel-programming-talk-130-parallel-manifold-with-jim-dempsey/

But what really caught my imagination was the first section of Chapter 7, "Programming Languages," that briefly described the Quantum Random Access Machine (QRAM) model of quantum computation. In addition to the few paragraphs that were devoted to this model, there was a picture that showed the relationship of a classic computer to a quantum computing device. Each part was simply a box with data/instructions passed from the classic to the quantum and data (results) passed from quantum to the classic side.

This setup looked familiar and it came to me during my cool down: this is how a system equipped with MIC would work. That is, your Intel Core processor does some initial computation to set up data, the data is passed over to the MIC (along with the computation instructions to be executed), and the results from the MIC can be returned to the Core side for use.

I know that MIC processors (and other GPU-like devices) don't have the same computational power as a quantum processor could have. However, the data-parallel and SIMD execution modes are similar to how a quantum device could take a superposition of all potential input data and execute a single computation step to arrive at a measurable result. This similarity got me thinking that MIC devices could be the first steps taken by the industry to better understand, prepare for and program effective quantum computations.

I don't know if we will ever see commodity quantum computation devices. I doubt they'll be developed within my lifetime, at least. Even so, I am nothing short of astounded when I look back at how far computer technology has come since I wrote my first COBOL program on an IBM mainframe. 

Knowing I should "never say never," how about on the day after I get my qPad(TM) quantum tablet device, I come back and comment on this blog post to say I was mistaken about how quickly quantum computation entered our lives? If it's anywhere in the cloud-o-sphere (and you know once these bits get pushed out, they never go away), I'll find it with the qSearch app, which will be based on the algorithm outlined in section 6.4 of Yanofsky's and Mannucci's book.