Thanks for the sympathy.

FWIW I am not bailing out on the project. I will (may have already) finish it and run it on the MTL system. If I am satisfied that it is working, I will ask the contest manager if he/she will make a run using the same test data. This run will be unofficial since I did not make the deadline. The purpose of the run is should the technique prove superior, then I will write-up a multi-part paper for the ISN. Outline the strategy and show how it is done.

My technique is a bit different. Let's see if I can diagram it.

The parallel pipeline is setup with three stages: Input (serial), Process (parallel), Output (serial)

The input and output pipes are not your standard: Input until EOF and Output while more data.

Instead the input pipe does the following:

Pair obj merge buffers (adjacent chunks, or select last remaining)
If source file remaining read into source input buffer

As with standard parallel pipeline input pipe as little work is done as possible

The output pipe does the following:

if processsource produces obj, send to merge pending pool
if merge A+B produces obj, send to merge pending pool
if merge A+B (or A alone) produces output, send to output file

The merge pending pool is Single Producer (Output Pipe), Single Consumer (Input Pipe). Therefor it can run without interlocked.

The input pipe does NOT shutdown on EOF. Instead it shut downs when the merge phase fails to produce further object code.

The source to obj process checks syntax and converts the expressions into in-line code with known expression/sub-component(s) of expression converted to values (double or text), or and expression with undefined components. The process pipe will determine if the (to be written) obj code represents chunk 0 or some other chunk.

chunk 0

When the expression yields a result, no obj code is generated, instead the symbol table is updated.
If the expression is not resolvable this is a error

chunk x not 0

When the expression yields a result the target variable is examined to see if it previously was loaded with a valid result (excepting for first time on buffer), and if it did, the output obj buffer at the location that stores the data is overwritten with a "skip this code". Then the target variable is updated, the current obj output position is recorded into the variable's property and object code is generated to set the variable to known value.
When the expression is unknown, the current obj output reflects the remaining computation to be performed at future merge state, then the obj code receives store some future value into variable xxx.

The process pipemerge phase works on prior obj code The A buffer is processed to squish out the "skip this code". Then the symbol table from B is merged with symbol table from A and the B buffer obj code is re-processed using the known values from the A buffer (A buffer immediately precedes B buffer). The result of this (generally) computes more values for expressions from former knowns in expressions thus condensing the obj code (or eliminating it). When the merge A buffer represents chunk 0 the output statements fill into the output buffer.

This process keeps running until no more merge data (or error is detected).

Jim

Close enough description to give you an idea. Will correct errors and omissions for the article.

Not given in the preceeding information is the layout of the symbol table, how it is indexed, what properties are in the variables. Symbol names are a-z, A-Z, 0-9 (62 characters) to this I added character '.' making 63 symbol characters plus a null character to indicate end of variable name. IOW 64 characrers (6-bits). The primary table has 4096 entries (two 6-bit characters). The secondary tables have 64 entries (for each successive symbol character).

Note, a number in an expression

a = 150;

generates a symbol name in 6-bit for "150". Resulting in an index of "15" in the primary table and "0" in the secondary table.Should the symbol entry for symbol "150" be defined then its value has previously been computed and is used witout a conversion from ascii to double. If the value is undefined, then the conversion from text takes place and the value is inserted into the symbol table.

There is a potential drawback here in that literals are (currently) never deleted.

BTW, "150" and "150.00" are two different symbols.

Also

free(symbol);

marks the variable a undefined, but leaves its entry in the symbol table for future use. Finding an undefined variable is quicker than deleting and adding an entry.

Jim Dempsey