Let's rename "for" to "serial_for"...

Proposal: rename for in C and C++ to serial_for
No more incumbent "for." (it was voted off the island)

(let's assume parallel_for == cilk_for in this discussion)

Consider:

serial_for (i=0; i < n; i++) { body }
vs.
parallel_for (int i=0; i < n; i++) { body }

  • serial_for allows the values of n and i to be modified inside body, parallel_for does not

  • serial_for allows body of one iteration to affect body of a future iteration, parallel_for does not

  • serial_for guarantees the exit value of i will be n, parallel_for does not


That's the highlights.

Which should we use?  serial_for or parallel_for?

What if I told you that every new computer today can reasonably be expected to work better with parallel_for than serial_for... and the difference will keep getting bigger in the future?

"Serial Traps" has been a topic between myself and two co-authors for an upcoming book on parallel programming.

I think we are coining phrase... and maybe this is the first coin off the presses.  Maybe someone else already coined it and we didn't know it?  It's hardly a new idea... but we constantly get reminder that it is not widely understood.

I am developing a list of "Serial Traps" which are use of a programming method, that has unintended consequences on parallel program performance.  for as a serialization method - is one such trap.

I say "unintended" because I do believe large numbers, if not most, for loops are not written to depend on the features of serial_for that are not supported by parallel_for.

My favorite short example:

for (int i=0;i<n;i++) { b[i] = a[i]; }

This is full of unintended consequences when for means serial_for.

written as:

parallel_for (int i=0;i<n;i++) { b[i] = a[i]; }
or (better yet)
b[:] = a[:];

does what the programmer probably wanted - without implying unintended consequences that kill performance that super-optimizing compiler try to get back but succeed only some of the time.

Perhaps I should have proposed "Make for == parallel_for, and create serial_for to be used only for legacy needs."  Would that win more friends?

 
For more complete information about compiler optimizations, see our Optimization Notice.

Comments

Here's a counterproposal: let the compiler figure it out. They're supposed to be smart, aren't they?

Here are the consequences:
1. "large numbers, if not most, for loops are not written to depend on the features of serial_for". A smart compiler can directly substitute parallel_for as the loop wouldn't notice.
2. A further group of for loops is trivially transformed, e.g. the common "for (i=0;i<N;++i) sum += a[i];".
3. A third group is for-loops that will be written with an eye towards such smart compilers, where the developer will take care.
4. Finally, some loops are best left serial.

The result: all the benefits, even for old code, no new keywords needed, and no gratuitious incompatibilities. Only downside: the compiler will need to figure out whether a dependency exists between two iterations, and if so, what the nature of that dependency is. But that is what modern compilers do for a living. They've got to figure such things out anyway, if only for register allocation.

If you need an example for your book, use C strings. Unlike C++ std::string, you don't know up front how many characters there will be, which forces a serial examination of each character in turn. You can't check pc[i+1] until you know that pc[i] != 0. Now _that_ is a serial trap. And since strings are so common, this hurts 99% of all C programs.