P1:A1 Maze of Life, Apprentice Problem - Judging and Scoring Criteria & Methods for Selecting Our Winners

P1:A1 Maze of Life, Apprentice Problem - Judging and Scoring Criteria & Methods for Selecting Our Winners

Jeff Kataoka (Intel)'s picture

Threading Challenge 2011 -Maze of Life, Apprentice Problem 1: Judging & Scoring Criteria and Methods

As of June 21, 2011, we announced the winners forApprentice Problem 1, Maze of Life. Our group of Judges used the following judging and scoring criteria and methods for selecting the winners. In addition, you will find a link to the testing results and the scores further below.

Apprentice Level Problem Set 1 (P1A1) Maze of LifeKey Scoring Principles

Basic scoring principles used for the contest entries judging are described at the official rules page. Here is a short summary: each contest entry was scored according to the following criteria: 1) up to 100 points for solutions performance (speed); 2) a maximum of 25 bonus points for a contestants activity in the forum, calculated as 5 bonus points for each valid forum post/reply.

There may be multiple solutions to a given puzzle. A bonus of 5 points per puzzle awarded to those entries that find the shortest path. Sometimes bonus was awarded to multiple entries outputting the same length path, even if those entries discovered different solutions. Please find the detailed problem set description at the Apprentice Level contest page.

Input Data Sets Used for Performance Scoring

Ten different input data sets were used to compute the execution score for this problem. The simplest one is a 7x7 Maze of Life grid given as an example in the problem set description. The hardest is a large 300x300 grid. Full archive of input data sets can be downloaded here.

Points in Performance Scoring

Each input data set was judged individually based on a ranking scheme. The weights of data sets used in performance scoring were equivalent. The overall performance score was calculated as a sum of all ten input data sets individual performance and bonus points.

We allowed a total of 120 seconds execution (2 minutes) maximum for each input set; for those runs that took longer than 120 seconds or had runtime errors during execution, zero performance points were awarded. Some entries that could not be built on the MTL got zero points as well.

Successful contest entries that found smart cell path in less than 2 minutes were ranked based on their execution time and got performance points according a reciprocal rank scale. For example, the fastest solution[s] of a data set got 5 points, next solution got 2.5 points, then 1.67 points and so on and so forth.

On top of that all the solutions that found a shortest path in particular input data set got 5 bonus points. Thus a successful solution could get a maximum of 10 points per input data set if it finds the shortest path and demonstrates the best performance.

Execution Results and Point Spread

Weve received 28 contest entries in the Apprentice Level Maze of Life problem set.

Six entries successfully solved all 10 grids. One entry solved 9 out of 10. Two entries solved 8 grids, four solved 7 Unfortunately, nine entries were incomplete and therefore unable to solve any test data sets.

All the timings and performance points are available in the final Maze of Life scoring table below.

Bonus Points for shortest paths found

As for the results in terms of shortest paths, they vary. Only one participant provided a solution that always found a shortest path on the grid. Interestingly enough, none of the entries that solved all 10 grids were able to find more than 2 shortest paths. Therefore, they did not get significant bonus points. All the bonus points are available in the final Maze of Life scoring table below.

Forum Activity and Bonus Points

Additional bonus points were given for contestants forum posts made before the problem entries were closed. Five points per post (maximum 25 points possible) were awarded.

Entry points and penalties.

Each contest entry got 100 entry points. A penalty of 50 points was taken off in case the entry is not able to solve simple data grid given as an example in the problem set description.

Winners

The problem winners based on highest point total are:

1) VoVanx86

2) krivyakin

3) jmfernandez

These three contestants provided the solutions which resolved the maximum number of our test grids. They also had the fastest overall code execution and a fair amount of bonus points for the shortest paths.

Access Judging Score Card for Maze of Life, Apprentice Problem 1

6 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Congrats to all.

Quoting Jeff Kataoka (Intel)

Full archive of input data sets can be downloaded here.

This link is not working?

Jeff Kataoka (Intel)'s picture

Sorry, the hyperlink to the data sets got lost when we copied the criteria text to the forum. We fixed the link and it is now LIVE.

VoVanx86's picture

I try repeat datases 7 with 4 different algorithms and i can't get 16 steps... only 19. My be 16 step is a wrong result?
Can You print shotest path in dataset7?
And i got 55 steps in ~1h with dataset9. Very hard test (if you try to find only shortest path).

jmfernandez's picture

With a modification of my algorithm for a shortest path search, I also need 19 steps to find a solution for the seventh dataset. Anyway, I am not sure that my algorithm is 100% correct.

By the way, congrats ;o)

I also found for 16 steps. Here is a path:DownRight,Down,UpLeft,Stand,DownRight,Down,DownRight,UpRight,Stand,Stand,UpRight,DownLeft,Stand,DownRight

Login to leave a comment.