Moving forward with the Windows 7 serial numbers, next I did a run with the recursive parallel technique I described last time:

I included the simple parallel run for comparison.  Looks like there’s more work to do.  My “fast” technique is still not measuring up against either the serial version or the buggy simple parallel version.  Time for another hot spot analysis:

Curious.  Where most of the work is done is in accAcc(),  yet most of time is being spent in rect_interact(), which just splits up rectangles.  Maybe I’m being too aggressive at subdividing.  Yup, it turns out this is another load balance problem.  And the test program comes equipped with a grain size control in the form of a parameter, fastgrain. You can experiment with various thresholds, but using a grain of 16 makes a substantial shift in the hot spots:

 Body_interact() is now taking less time than addAcc() (where it had been taking more than four times as much before).  And if I collect a set of runs with fastgrain 16, there’s a big change to the performance graph:

These runs start out like the previous parallel algorithms, but by 8 bodies, the curve has already fallen below that of the unsafe, simple parallel method, and by 64 bodies it’s beating the baseline serial numbers.  At 4096 bodies this algorithm is now running over 4x faster than the serial version.  Success!  Could we do even better with a larger grain size?  Maybe marginally, but note in the latter hot spot result that TBB internal code including the idle spinner are already taking more time than the code doing real work.  It’s possible that a higher threshold will just result in more idle time.  Something to look into another day.

This time I’ll turn the concept into some sample code, but first a reminder of that partitioning scheme.

Because the ranges of i and j are disjoint between triangles A and B, I can set two threads loose, one on each, and let them handle the interactions between bodies represented by points in the region.  I can’t touch any of the pairs represented in rectangle C while A and B are being worked, but I can wait until A and B are done, then start C.

So it looks like the algorithm will have two steps, with two different threads of recursion.  Each triangle will be split into two, smaller triangles and an adjacent rectangle; each rectangle will be subdivided into four smaller rectangles.  I can keep splitting until there’s only one point left, but that might prove to be overkill.

I’ll start with the triangle subdivision:

void
body_interact(int i, int j)
{
    int d = j - i;
    if (d > 1) {
        int k = d/2 + i;
        parallel_invoke([&]() {body_interact(i,k);}, [&](){body_interact(k,j);});
        rect_interact(i, k, k, j);
    }
}

Here’s a function called body_interact(), which takes a pair of indices representing the range of the incoming triangle.  The function finds the midpoint of the defining interval and uses that point, if it is distinct from the endpoints, to define the next pair of triangles and rectangle.  It schedules calls to itself for each of the triangles, and then calls rect_interact() to handle the adjacent rectangle.  Inherent in the behavior of parallel_invoke() is to wait for both (or all) calls to complete before returning, thus guaranteeing that interactions in the rectangle will not start until work is done in the triangles.

This triangle code is but a subset of what is needed for the rectangle recursion.  For it I need two midpoints and two parallel recursion calls:

void
rect_interact(int i0, int i1, int j0, int j1)
{
    int di = i1 - i0; int dj = j1 - j0;

    if (di > 1 && dj > 1) {
        int im = i0 + di/2;
        int jm = j0 + dj/2;
        parallel_invoke([&]() {rect_interact(i0, im, j0, jm);},
                        [&]() {rect_interact(im, i1, jm, j1);});
        parallel_invoke([&]() {rect_interact(i0, im, jm, j1);},
                        [&]() {rect_interact(im, i1, j0, jm);});
    }

    else {
        for (int i = i0; i < i1; ++i)
            for (int j = j0; j < j1; ++j)
                addAcc(i, j);
    }
}

Two balanced parallel operations are invoked when there is enough work to subdivide on both axes.  Otherwise, the else condition drops into the actual interaction function, addAcc().  (You might have noticed that body_interact() does not even call addAcc(). Because of the half-open interval notation I’m using, a triangle subdivision that reduces to a single interaction node is in fact pointing at a node representing the interaction of a body with itself, so we can ignore them.)

With this code, I can go back to the original addAcc(), which updated the acceleration vectors of both interacting bodies without concern about data races and without any locks.  There are still locks, but they are hidden in the library code that handles the synchronization of the rendezvousing threads within the parallel_invoke() call.  But will it run faster?  We’ll find out next time.

Sorry, too late to play 20 questions and WIN VALUABLE PRIZES!  Our Intel Parallel Studio expertise contest has concluded.

    // apply acceleration components using unit vectors
    {
        MyLockType::scoped_lock mylock(ijlocks[i][j]);

        for (int axis = 0; axis < 3; ++axis) {
            body[j].acc[axis] += aj*ud[axis];
            body[i].acc[axis] -= ai*ud[axis];
        }
    }

Rather than pursuing that right now, let me take a step back and try to look at the problem differently. I can represent all the interactions between pairs of bodies, i and j, in a simple graph:

The green triangle above covers the area where i ≤ j.  Skipping the points along the diagonal (we don’t need to consider interactions of bodies with themselves), all the other integral points within the triangle can represent unique interactions between pairs of bodies.  Dividing the outer loop between a set of threads is like splitting the triangle along horizontal lines as shown above.  Each sub-region can be handled by a separate thread.

One problem becomes immediately apparent by drawing this diagram: the sub-regions have different sizes.  That means the number of interaction pairs within each varies—the thread that gets the bottom sub-region has a lot more work to do than the one getting the topmost region.  The bottom thread will be working for a long time after the top thread finishes.  This is a great example of what is called a load balance problem.

A little more thought will net the realization that while each thread in the subdivision above has its own range of i values, they must share j values: any set of interactions of different i that have the same j are collisions that are either races or restricted by locks to the execution of a single thread.  What might help is if I could add some vertical partitions to this graph, and gain some separation between the threads like what happened with the i values.

It turns out there is a way to do this, using a combinatorial algorithm I learned from Matteo Frigo.  Considering the same interaction graph, this time find the midpoint of the diagonal and use it to draw a couple triangles like this:

The triangles A and B have a particular property of great interest: the ranges of i and j are completely disjoint.  A thread working in the range of A could pick any contained interaction pair and be guaranteed not to interfere with another thread doing the same thing within triangle B.  Two threads could simultaneously cover the interactions represented by the pair of triangles, each covering roughly the same number of interactions and thus load balanced.

That handles two threads.  What about more?  Well, following that great adage, “whatever is worth doing is worth overdoing,” I can repeat that partition on the new triangles:

Now in this triangle there are eight separate regions, each which could contain a separate thread operating without interfering with the adjacent ones.  However, more and more of our available interaction space is turning dark green, representing as yet unhandled pairs of interactions.  Fortunately these are easy to handle.  In fact, the triangle interactions are a subset of the rectangle interactions.  Consider the biggest rectangle:

I can split this rectangle into four sub-rectangles.  In this arrangement, I can turn two threads loose, each one targeting interactions in one of the two light green rectangles.  The disjoint property holds for those two regions.  When those two threads are both done, I can move one each to the two dark green regions because they also have the disjoint index range property.  And what I can do once I can also repeat here, subdividing these rectangles and generating as much balanced parallel work as I need to occupy all the available threads.

We’ll look at some sample code for doing this next time.

#ifdef ALIGN_BODIES
__declspec(align(128))
#endif
struct bodytype {
    double pos[3];  // body position in three axes
    double vel[3];  // body velocity
    double acc[3];  // body acceleration
    double mass;    // body mass
#ifdef BODY_LOCKS //{
    MyLockType lock; // local access lock
#ifdef ALIGN_BODIES //{
    unsigned char dummy[128-(10*sizeof(double)+sizeof(MyLockType))]; // 10 is the number of doubles as data
#endif //}
#else //}{
#ifdef ALIGN_BODIES //{
    unsigned char dummy[128-(10*sizeof(double))]; // 10 is the number of doubles as data
#endif //}
#endif //}
} body[BODYMAX];

I’ve added a new conditional, BODY_LOCKS, which when defined creates a lock object of MyLockType in each body object.  And I added the size of that object to the dummy field at the end so that I maintain cache alignment for each of the bodies.  But compiling it...

Oops. How many free bytes were there before adding the lock? Hmm.... (128 – 10* sizeof (double)) = (128 – 10* 8) = 48 bytes. So I guess the lock structure must take more than 48 bytes. And if I double the size of the body structure (using 256 instead of 128), that does fix the errors. Does it have any other impact? And for that matter, what is the cost of not padding the bodies into separate cache lines? Taking a few runs to test, we get numbers like these for the serial run (which have an insignificant impact in the overall graph):

These effects may be different when I  actually turn on multiple threads (and have them deal with bodies split across cache lines) so I’ll take another look then, but for now I’ll go to the 2-cache line body to flesh out the lock-per-body approach to thread safety. The code change to use these locks:

    // F = G*mi*mj/distsq, but F = ma, so ai = G*mj/distsq
    double Gdivd = GFORCE * ivdist * ivdist;
    double ai = Gdivd*body[j].mass;
    double aj = Gdivd*body[i].mass;

    // apply acceleration components using unit vectors
    {
        MyLockType::scoped_lock locki(body[i].lock);
        MyLockType::scoped_lock lockj(body[j].lock);

        for (int axis = 0; axis < 3; ++axis) {
            body[j].acc[axis] += aj*ud[axis];
            body[i].acc[axis] -= ai*ud[axis];
        }
    }

Updating accelerations now require two lock references, conveniently called locki and lockj. Also conveniently, the order in which I am acquiring new is and js, using a half triangle of the interaction plain (touching each i-j pair once, updating both) means that j is always larger than i.  This is a hierarchical ordering of the locks, which means that whichever thread is accessing a pair of bodies, it will always acquire the lock for the smaller-numbered body before the larger-numbered one (it would work equally well the other way, as long as all the threads do it the same way.

Nevertheless, though it is faster than the single, global lock version used before, it still isn’t as fast as doing it serially.

My use of locks doesn't seem to be getting me far enough.  Maybe next time I'll take a step back and see if there is a different approach.

    // F = G*mi*mj/distsq, but F = ma, so ai = G*mj/distsq
    double Gdivd = GFORCE * ivdist * ivdist;
    double ai = Gdivd*body[j].mass;
    double aj = Gdivd*body[i].mass;

    // apply acceleration components using unit vectors
    for (int axis = 0; axis < 3; ++axis) {
        body[j].acc[axis] += aj*ud[axis];
        body[i].acc[axis] -= ai*ud[axis];
    }

The lines revealed as problematic in my previous correctness testing were those comprising the last two calculations, adjusting the acceleration fields in each of the two bodies interacting in the addAcc() call. Since each HW thread must load the current acceleration value into a local register in order to perform the summing operation, two threads may read the same value before either has a chance to update it, leaving after both writes one update overwritten (and lost) by the other.

To avoid the update races, a mechanism is needed to assure that any thread can do both the read and the write before any other thread starts the same operation from another body (or rather, a thread working on the same j-body from a different i-body). The typical mechanism is to employ a lock, a location manipulated by special pieces of code that can guarantee only one thread can own the lock at a time. Using a lock, I can create a critical section (also called a monitor) a region of code where only one thread can operate at a time.

It's very easy to do this in Intel Threading Building Blocks, using scoped locks:

    typedef tbb::mutex MyLockType;
    MyLockType L;

        // F = G*mi*mj/distsq, but F = ma, so ai = G*mj/distsq
        double Gdivd = GFORCE * ivdist * ivdist;
        double ai = Gdivd*body[j].mass;
        double aj = Gdivd*body[i].mass;

        // apply acceleration components using unit vectors
        {
            MyLockType::scoped_lock lock(L);

            for (int axis = 0; axis < 3; ++axis) {
                body[j].acc[axis] += aj*ud[axis];
                body[i].acc[axis] -= ai*ud[axis];
            }
        }

Scoped locks are pretty cool because they take advantage of the basic language notion of object lifetimes. In the example above, I create a specific lock type (tbb::mutex) and create an instance of the lock, L. Then in the region where I want to use the lock, I create a separate scoped_lock object, lock, and tie it to object L in the constructor. This acquires the lock. The lifetime of this object is the region of the compound statement in which it is defined. When it passes out of scope (a thread leaves this region), then the secondary object goes away and the lock is released, while the lock object (L) itself sticks around for the next locking opportunity.

What’s cool is that I don’t need explicitly to release the lock, meaning simpler code.  Moreover, if there was something in my for-loop that caused an exception (an interrupt to normal processing), the semantics of the language guarantee that the scoped object will go away and the lock will be released automatically. That's very cool.

But that’s about as far as the coolness goes. Viewed from a different angle, this is pretty gross code. I've created a single lock to service however many bodies and threads that may exist when I run it. That means that even in cases where a pair of threads are operating on completely different pairs of bodies, they still must wait their turn. That's a lot of serialization, suggesting that the resulting code will be much slower, which it is:

Well, that's not going to win any performance awards. Representing that versus my previous graph:

I'll have to do better if I plan to show a beneficial parallelization. At least the code is thread-safe now:

Or, at least the reported races are not in the bodies code (mostly). Maybe I'll explore that, but first, what happens if the threads don't pile up on a single, global lock

In my last bumbling about, I tried defining the TBB_USE_THREADING_TOOLS macro as a stab to find the problem with my Intel® Parallel Inspector analysis of nbodies.  Didn’t seem to do much at the time, so I thought it might be interesting to find out what it really does.  It was easy to find examples of it in the open source.  spin_mutex.h contains has a scoped lock constructor:

 
        //! Construct and acquire lock on a mutex.
        scoped_lock( spin_mutex& m ) {
#if TBB_USE_THREADING_TOOLS||TBB_USE_ASSERT
            my_mutex=NULL;
            internal_acquire(m);
#else
            my_unlock_value = __TBB_LockByte(m.flag);
            my_mutex=&m;
#endif /* TBB_USE_THREADING_TOOLS||TBB_USE_ASSERT*/
        }

There is a bit of getting the cart before the horse to examine the details of a lock before even talking about races in my way-slower-than-expected narrative exploring the effort to parallelize some code, but it seems appropriate as a footnote.

So, what’s going on up there?  In the non-TBB_USE_THREADING_TOOLS case something called __TBB_LockByte is being called with a field of the spin mutex object (probably a byte?), which must be the lock part (a gate where only one thread gets by at a time).  Then the spin mutex object is stashed until later.  If multiple threads tried to do this __TBB_LockByte call at the same time, they might face some contention with each other, and some tool designed to detect those data races might flag this operation as suspect.

On the other hand, when TBB_USE_THREADING_TOOLS is asserted, it looks like local mutex pointer is set to a safe value and the mutex itself is passed to some other function, internal_acquire(), effectively hiding any lock funny business from our correctness inspection tool.  So that’s what it does.  Maybe after I introduce scoped locks, I’ll come back here and peel another layer, and we can look at the alternate implementations of the lock.

The rebuilt bodies program is just a touch slower (the 4K body serial average before was 283 sec vs. 278 sec for this parallel version).  In this case the serial number got hit harder so the parallel scaling number actually goes up slightly (from 1.02x to 1.03x) even though the numbers got worse!  Don’t trust parallel scaling numbers—understand the underlying data.

On to parallel correctness.  I know there’s a flaw in the parallel algorithm I’ve proposed, but how to demonstrate it?  If the n-bodies program actually had a means to display a projection of the bodies in motion I might be able to run that long enough to notice glitches in the body motion.  Maybe.  Or maybe not.  I’d like something a little more reliable and mechanical than exhaustive inspection, especially when there’s lots of data.  So let’s look at what we can learn from Intel Parallel Inspector.

First thing we need to do is change the command line to select a test for doing data collection.  I’ve been running these ramps using “select serial” and “select par” as the command lines.  These code paths have timing functions and run varying sizes of the problem to collect those times, and generally do way more work than I need to catch a race.  There is another option, the “single n” command that takes as an argument n, the number of bodies to use in the simulation.  I’ll change the debug command line to “single 128 par” to run the test with 128 bodies.  So here goes.

I’ll pick “ti3” the third level thread inspection, in order find out where my deadlocks or data races exist.  You can see from the estimates shown at the left that this might take a little longer than just running the test instance by itself.   So I hit the “run analysis” button at the bottom of the dialog and off it goes:

Assuming all goes well and data actually get collected from the run, I next see something like this:

And hitting the “Interpret Result” button takes me to this:

At the bottom are some of the observations collected during my program run.  The observations were interpreted when I hit the last button, generating the problem sets visible in the upper pane.  Looks like I have some data races (problematic places in memory where multiple threads may be writing and reading data in an undetermined and potentially harmful order).  I should be able to double-click on a problem set and find an error.  I’ll try P1:

Oops.  That’s not very helpful.  Oh, but the executable module is irml, not NBodies. I get the same thing with the  next two problem sets.  Moving on to P4, which does not mention irml (which is a part of TBB—Intel resource management layer?), I try again:

Hmmmmmmmm….  This doesn’t look like NBodies code either.  This is somewhere in the TBB parallel_for header file.  Parallel Inspector flags this as a Write-after-Write race but it doesn’t seem to give me much that might help me understand the problem.  Maybe this is a false positive, a case where it looks like there might be a race condition but there really isn’t.  The code that unwittingly commits a data race might not look that different from safe and legal mutex code.  Sometimes it’s only in how the code is used that can distinguish one case from the other. 

To tell the difference, Intel Parallel Studio has a library of function calls that can be used to declare safe operations that might otherwise be considered suspect.  To enable these TBB conditionally compiles under the TBB_USE_THREADING_TOOLS macro definition and will apply alternate code when enabled to hide questionable code.  But it can cost a little in performance so it does need to be turned on when you need it, which can be done inside Intel Parallel Studio:

Under Parallel Composer Select Build Components is the following dialog:

Oops. Not set.  That’s easy to fix.  Let me rebuild and recollect the ti3 data.

Well, that doesn’t look much better.  Now irml is a contributing module for all the data race problem sets.  Sure enough, drilling down to source on any of these problem sets is as unsatisfying as were any in the last collection.  So what am I doing wrong? And did that TBB_USE_THREADING_TOOLS do anything?

There is one other thing that I can try.  I’ve been using the Release configuration here, following on from the performance runs that started this post.  This has been a problem before when using analysis tools because of the aggressive function inlining that Parallel Composer normally applies.  I have an alternate configuration, Release-with-functions, which has the same settings as Release save the function inlining, which is turned down to /Ob1.  Switching to that configuration, rebuilding the program and collecting ti3 data one more time:

That doesn’t look much different than the last one.  However, when I double-click on P2 this time, I get this:

Well this looks better.  In fact, the highlighted lines are the known data-race.  My brute parallel code divides the index i among a collection of threads, which may simultaneously try to modify the same body j.  This display shows the two sides engaging in the race, which in this circumstance happens to be from different invocations (threads) of the same code.

Moreover, when I drilled down to all the other problem sets in this collection, none of them contained the “my_body( my_range );” function call I landed upon before.  Duh-oh!  Of course!  Now I recognize this function call as where parallel_for actually executes the kernel containing the racy code.  It looks like the aggressive inlining normally in play in the Release configuration left the relevant code stripped of symbols.  There had not been enough detail available for Parallel Inspector to navigate closer to the racy lines until I relaxed function inlining.  Backing off inlining may also affect performance, but hopefully not so severely that we’re “Heisenberg-ed” into observations that are substantially wrong.

If you haven’t gotten enough of this, we’re taking the Parallelism road show on the road again in a few weeks (middle of March, snow permitting) up the East coast with several stops from New Jersey to Boston.  Find out more details at this Programmer’s Paradise link.

Next time: parallel code: finding a fix for the leaky adds

But first, because I’ve made some serial code optimizations and now I’m using Intel® Parallel Composer update 4, I want to take more samples of the serial kernel with the improved addAcc() function.  Using the command, “select serial” I collected several runs of data and took their averages (first column is the number of bodies):

Then I did the same thing with the command, “select par” to get the basic parallel kernels for interactions and dynamics discussed previously.  To compare against the serial, first I color-coded the averages (I’m a sucker for Excel conditional formatting): parallel runs that are faster get the green—if they’re more than 10% slower, they see red.

OOOooooo…, that’s way more red than I expected.  Plotting a log-log graph of these data:

That’s not very impressive parallelism.  It takes around 2048 bodies interacting for my simple parallel kernel to beat the serial code.  It wasn’t like that six months ago when I first compared these kernels:

This is an image of the performance of these same kernels from a presentation I put together last summer.  Here the simple parallel kernel looks impressive, sweeping under the serial kernel albeit stuck on the same n-squared growth curve.  What happened?  Serial optimization.  I pulled the data for those serial runs and added it to my previous graph of the new results:

As you can see, the old serial code was quite a bit slower and the crossover point for the parallel runs was down around 64 bodies (the new parallel results are a little better than the old: the old 4096 bodies number was around 288 seconds, versus 277 here).  So, even as I was preaching about doing serial code optimization first, the serial code I was using for the previous graph lulled me into a false sense of accomplishment.  No wonder I was disappointed!

But it’s even worse, though this twist I was expecting.  This parallel kernel that is struggling to beat its serial cousin has a fatal flaw, which I’ll explore next time.

Next time: parallel code: first run’s fatal flaw

With this simple change, I notice something new in the compilation logs:

If I double-click on one of these lines in the Output panel, the system navigates to the corresponding source code lines:

Note the little marker in the left margin that indicates the lines referred by that vectorization report line. This is the ballistic step in the serial code implementation, suggesting that the little loop for setting the vector components of the bodies has been converted to a linear sequence of vector instructions. Some other time I’ll dig down into the assembly code to demonstrate that this code really has been vectorized (i.e., realized by emitting SIMD code to execute it), but for now let’s move forward and try to make this multi-thread parallel in addition to vector-parallel.

How do I “parallelize” that lower set of loops in the previous code sample? One simple way would be to add an OpenMP construct:

    #pragma omp parallel for
    for (i = 0; i < n; ++i) {
        for (int axis = 0; axis < 3; ++axis) {
            body[i].vel[axis] += body[i].acc[axis] * TIMESTEP;
            body[i].pos[axis] += body[i].vel[axis] * TIMESTEP;
            body[i].acc[axis] = 0.;
        }
    }

OpenMP has been around for a number of years and operates as a language extension for C, C++ and Fortran. Compilers enabled to recognize the constructs (such as the Intel® C++ and Fortran Compilers) can use them as hints to direct the compiler generation of parallel code. Non-complying compilers see these constructs as an unrecognized pragma (or a funny comment in Fortran) and ignore them. In this case the OpenMP line applies to the line that follows, directing the compiler to create code that divides the outer loop into some collection of chunks, each of which can be dispatched to a separate HW thread. Each thread processes the chunks assigned to it. As each thread finishes its work, it waits for the others in its team to complete their work. All these HW threads will land in a rendezvous or join point until all have arrived, because there’s an implied wait at the end of the parallel for-loop so that code that follows will not be executed until the preceding code has been completed, just to avoid any potential side effects. In this particular case, we’re also at the end of the parallel section so only one HW thread would proceed beyond the end of the for-loop, the rest returning to a thread pool to await more work.

With the advent of lambda constructs, described in the C++0x standard and implemented in the Intel C++ Compiler version 11, we can write nearly as compact a version of this parallel construct using Intel® Threading Building Blocks (before lambdas we’d need to use a full C++ function-object, which really breaks up the flow of the source code). As a lambda construct using TBB, the OpenMP code above would transform into something like this:

    parallel_for( blocked_range<int> (0,n),
      [] (const blocked_range<int> &r) {
        for (int i = r.begin(); i != r.end(); ++i)
            for (int axis = 0; axis <3; ++axis) {
                body[i].vel[axis] += body[i].acc[axis] * TIMESTEP;
                body[i].pos[axis] += body[i].vel[axis] * TIMESTEP;
                body[i].acc[axis] = 0.;
            }
      });

Not quite a compact as the OpenMP version so there must be some other reason to value this. Otherwise, why embrace the complexity? The key is flexibility. TBB offers a rich set of tools that can be used within the context of such a parallel function. The c++0x lambda-function expands that richness with a compactness of expression and flexibility that lets me use TBB with almost the same convenience of OpenMP. For example, that pair of square brackets leading off the lambda provides flexible control of what variables defined in the scope of the call will be available and in what form within the function (more on this later). The TBB parallel_for will divide the work of this inline body using as a helper class the TBB blocked_range, making work for as many HW threads as there are available.

Next time: parallel code: first runs

Then I ensured the vector component accumulations take advantage of the simplification:

To avoid changes to the body data structure that might affect the experiment, I redefined the acc field to mean accumulator instead of acceleration (cheap trick for a short hack ;-).

With addForce in hand, I needed to make some adjustments to the ballistic step to turn the forces back into accelerations:

Oops, adding three divides per body (one for each axis), which gives this result:

In my experiments, the add forces version came in slightly slower than my best add acceleration version. It’s easier to see in the numbers:   As this table shows, the times for the run accumulating force takes longer, as you would expect for a solution that requires more multiples (to include the extra mass term in the force equation and then to remove it to get to acceleration). But wait a minute! There’s something else going on here. What’s with those serial addAcc numbers? I remember lower numbers when I took my first serial run. Maybe there’s more variability in the results than I recall? That’s easy to check. I switched back to the bodies007 code and took several more runs.

That looks all pretty consistent across the range of n-values. Yet when I tried the same thing with bodies008:

Almost a second slower for 2K bodies, even though the supposed “code under test” didn’t change. Note: I’m not even running the addForce code!—just the conditional test in the RAMP test mode (see below). There were not many changes in going from bodies007.cpp to bodies008.cpp so it was pretty easy to isolate the code change that caused most of the slowdown. I was able to get these numbers...

…by the simple expedient of commenting out the following code:

//                           elapsed = (etime - stime).seconds();
//                           cout << "," << setw(20) << elapsed;
//                       }

This is one of several clauses in a for-loop that selects the values of n for the ramp; commenting out all the variant methods so that the only one remaining is the serial addAcc does even better, though the returns are diminishing:

So, given the ramp loop is somehow having an effect on the numbers, let me reverse the scenario and comment out all but the addForce variant:

Each of these last two sets are the averages of five runs each, and by my measure, adding accelerations still wins, though the answer is much more murky than I would hope. What are the gremlins that are plaguing these numbers? I have some hunches that involve optimization and inlining strategies but I can’t yet point my finger at specific problems. There are some tantalizing observations to be made, though.

For example, I could try doing a hot spot analysis to see if that would provide clues about the unexpected overhead. However, that means relaxing function inline optimization (the /Ob1 trick). But what does that do to performance?

Wow. Looks like function inlining is a big performance benefit for the serial acceleration accumulating code, in this case (one sample) having lost 14 seconds computing the interactions of 2K bodies.  Curiously, the serial code accumulating forces appears to take a much smaller hit from the loss of aggressive function inlining. Or, in glass-half-full parlance, it appears to take much less advantage of compiler optimizations.The same appears to be true if you continue relaxing optimizations, specifically looking at the performance of the Debug configuration with this same test (all these tests are using the optimizations available in Intel® Parallel Composer, so your mileage may vary depending on what compiler you use):

There’s a lot more to be discovered in this rich mine of anomalies, and perhaps when I have some more time, I will delve into it more deeply. For now though, I’ll continue to use the addAcc variant in the experiments going forward. After all, after over half a dozen posts in this series, I haven’t even gone parallel yet!

Expanding the view to show the function in detail is often called drilling down to source. In Intel® Parallel Amplifier I can do this by just double-clicking on the function, addAcc.

Parallel Amplifier lands on the hottest line in the function and provides easy navigation buttons (just below the Bottom-up button) to explore the other hot spots in order of time taken (max, step-up, step-down, min, respectively). Since I landed on the hottest hot spot, the "navigate-to-a-hotter-spot" buttons are grayed out.

Looks like addAcc has a problem with divides. Division is one of the more expensive arithmetic operations and while I can’t eliminate all of them, I certainly can reduce the number of them.

Computing the inverse distance once and then multiplying that seems to have had an effect: before the change, the times attributed to the nearby lines amounted to over 1.2 seconds while after, the total is down to 0.78 seconds. I accumulate the values from the adjacent lines because the reported event counts are at best approximate—the tool needs to deal with both the optimized code that may have scattered around the instructions that implement any particular line and phenomena that affect the actual process of determining the location of the instruction pointer, such as event skid (to be addressed in some other post). In fact, probably a significant portion of the 0.78 and 1.2 seconds is actually coming from the square root function that immediately precedes these lines. So I’ll run another ramp of n-bodies and see if my numbers are any better.

Yes, they are. And now that I have ivdist, it’s worth considering whether I can use it more efficiently to replace the divide by distsq into something like this:

    double Gdivd = GFORCE * ivdist * ivdist;

Sure enough, that change, though not as beneficial as the last one, still has an observable benefit:

Another hot spot run shows even less time being spent at the previously identified hot spots:

I’ll use this last version as my serial baseline, the benchmark against which I’ll compare to measure my progress in parallelization. It might change someday as I continue to evaluate alternatives like the persistent question about forces. This in fact is part of our recommended practice for migrating serial code into a parallel environment: I start by optimizing the serial version as much as I can so that the benefits I gain through parallel implementation are not just because multiple HW threads are just filling in the gaps left behind by reusing an inefficient, serial version.

Next time: parallel code, a first attempt

It installs right in Visual Studio as shown above. In order to collect hot spots on the serial algorithm, I switch the debug command to single 256 serial

I’ve also turned on symbols in my Release configuration (C/C++ >> General >> Debug Information Format set to and Linker >> Debugging >> Generate Debug Info set to on my latest build), then just click on the Profile button, and viola!

Huhhhhhh?! I see two seconds plus a quarter spent in main, but where are my functions? Do I get the same result if I try the Debug configuration?

Oh, there are my functions, runSerialBodies and addAcc, but the run takes over 5 seconds. I don’t want to spend time making Debug code run faster, so I want to tune the optimized Release code. However, something about that Release configuration is causing the functions to disappear. Experimenting a little with the configuration settings reveals that the Intel compiler is automatically inlining the functions into main. Unfortunately, apparently there’s no way to represent that inlining in the debug information so the functions just disappear. By relaxing the optimization a little, I can restore the function hierarchy for analysis at the cost of some extra function call instructions:

Now my hot spot analysis on the Release configuration looks much better:

Most of the time is being spent in the addAcc function, which is being called by runSerialBodies as can be seen in the function call hierarchy graph. Looks like addAcc will be one of my candidates for parallelization.

Next time: serial body drill-down

Oops, that’s right. bodies007.cpp relies on language extensions available in the Intel® Compiler version 11, some early arrivals from the C++0x standard. Fortunately, it’s pretty easy to switch compilers.

With the Intel C++ Compiler installed in Visual Studio from either of the regular distribution packages, the Compiler Professional Edition or Intel Parallel Composer, switching compilers is just a click away.

OK, two clicks. I did try to build already, so I’ll let the system clean up the project files.

Viola!!! The project now uses the Intel C++ compiler.

One more configuration setting to enable C++0x support:

And now it compiles!

Dropping the serial code into the test program I’ve prepared gives access to a simple command processor that allows me to select that kernel for one of several tests. A simple ramp, testing the algorithm with varying values of n, can be launched by putting this in the command line: select serial

Looks like the time it takes to complete the simulation (I’m running 1000 time steps for each body-count) is going up more than four times for every doubling of the number of bodies; in fact, if I plot it on a logarithmic scale, I can see the exponential growth:

Clearly this is an algorithm that has plenty of work to divide, if I can just figure out a way to do it.

Next time: serial body hot spots

i and j are indices selecting elements of the body array. First task is to compute the distance between them.

Pythagorean Theorem in three dimensions gets me the square of the hypotenuse, but before doing the square root, I’ll avoid the singularity:

That is, if the point masses get too close together, act like they’re not. But wait! Why do I even need the square root, if I’m working with gravitation, an inverse-squared law? Well, because acceleration is a vector so I need the next step.

Array ud represents the unit vector (length 1 direction vector) pointing from body j to body i. I need just one more thing, the magnitude of those accelerations.

All that’s left is to compute the acceleration vector components and apply them to the bodies.

Next time: serial bodies test run

Even better is if I can squeeze those inefficiencies out of the algorithm itself. Previously, before defining the interaction loops, I set up a data structure to represent each body, choosing mass, location, and velocity to represent the state, plus a place to accumulate accelerations. However, Jim Dempsey suggested it might be more efficient to accumulate forces rather than accelerations. Forces are the canonical result of the gravitational equation but divides are one of the most expensive floating point operations. Better to accumulate the influence of other bodies as forces first, then do one divide to compute the acceleration, right?

But where did that F come from?  We are actually dividing out a mass that was used to compute the original force. What if we never multiplied it in the first place?

The G-over-R-squared is a common factor that can be computed once. Two multiplies, which would be required anyway, give the accelerations directly, without any extra divides!   So we'll stick with the plan to add accelerations.

Next time: realizing addAcc(i,j)