Scoring Criteria
Three test cases were used to judge execution speed. The first had 100,000 keys to search through and 20 keys to search for. In the second file 10 of these were contained in the data file and 10 were not. The second and third test cases had a search space of 100,000,000 keys; the number of keys to be searched was 100,000 in both cases. One search file had 20 good keys repeated 5000 times and the other had 100K keys that were not contained in the search space.
A maximum of five minutes overall was set for each test case. The maximum time recorded for those entries that encountered runtime errors or other execution problems was 60 seconds (since the time was only for searching and printing the results. A penalty of 15 points per test case was levied for any results that had incorrect results (finding keys not within the data, not finding keys within the data, etc.). The sum of the three test case runtimes was compared to the minimum overall time and execution points were assigned linearly based on the difference.
There were 24 submissions of code solutions, 6 on Linux and 16 on Windows. There was one entry written in Java, one in C#, one in Erlang, and the rest in C/C++. Thirteen entries had runtime issues, especially in the last two test cases.
Point spread:
100
99 99 99 99 99 99 98 93
75
69
33 33 33 33 33 33
18 18
0 0 0 0 0 0
The write-up portion of each entry was read and scored by two judges. Each judge used the 10-30-10 breakdown of points for serial algorithm description, parallel algorithm description, and performance, respectively. One important component to the judging was to determine how close the submission was for publication on ISN. The assigned score was the average of the two judges scores.
Point spread:
44 43
37 36 33 32
29 28 24 24
19 17 16 14 14 12 11 11 10
7 4
0 0 0 0
Bonus points were given for contestant’s forum posts made before the problem entries were closed. Five points per post (maximum 25 points possible) were awarded.
The overall winner was denghui0815, who also had the fastest code execution. The highest scoring write-up was submitted by Dmitriy Vyukov.
| |
Re: Scoring Criteria
Three test cases were used to judge execution speed. The first had 100,000 keys to search through and 20 keys to search for. In the second file 10 of these were contained in the data file and 10 were not. The second and third test cases had a search space of 100,000,000 keys; the number of keys to be searched was 100,000 in both cases. One search file had 20 good keys repeated 5000 times and the other had 100K keys that were not contained in the search space.
A maximum of five minutes overall was set for each test case. The maximum time recorded for those entries that encountered runtime errors or other execution problems was 60 seconds (since the time was only for searching and printing the results. A penalty of 15 points per test case was levied for any results that had incorrect results (finding keys not within the data, not finding keys within the data, etc.). The sum of the three test case runtimes was compared to the minimum overall time and execution points were assigned linearly based on the difference.
There were 24 submissions of code solutions, 6 on Linux and 16 on Windows. There was one entry written in Java, one in C#, one in Erlang, and the rest in C/C++. Thirteen entries had runtime issues, especially in the last two test cases.
Point spread:
100
99 99 99 99 99 99 98 93
75
69
33 33 33 33 33 33
18 18
0 0 0 0 0 0
The write-up portion of each entry was read and scored by two judges. Each judge used the 10-30-10 breakdown of points for serial algorithm description, parallel algorithm description, and performance, respectively. One important component to the judging was to determine how close the submission was for publication on ISN. The assigned score was the average of the two judges scores.
Point spread:
44 43
37 36 33 32
29 28 24 24
19 17 16 14 14 12 11 11 10
7 4
0 0 0 0
Bonus points were given for contestant’s forum posts made before the problem entries were closed. Five points per post (maximum 25 points possible) were awarded.
The overall winner was denghui0815, who also had the fastest code execution. The highest scoring write-up was submitted by Dmitriy Vyukov.
Would still love to have a way to find out my score so I could assess what I was doing right/wrong.
| |
Re: Scoring Criteria
You sorted out the statements。 thanks！
写字楼里写字间,写字间里程序员
程序人员写程序,又拿程序换酒钱
酒醒只在网上坐,酒醉还来网下眠
酒醉酒醒日复日,网上网下年复年 | |
Re: Scoring Criteria
Three test cases were used to judge execution speed. The first had 100,000 keys to search through and 20 keys to search for. In the second file 10 of these were contained in the data file and 10 were not. The second and third test cases had a search space of 100,000,000 keys; the number of keys to be searched was 100,000 in both cases. One search file had 20 good keys repeated 5000 times and the other had 100K keys that were not contained in the search space.
Clay, please attach input files used for scoring.
In his blog denghui0815 write that his execution time for 10Mx10M input is 6.81s, while my is 0.61s. There is some difference in our machines and we use different input files, but I want to investigate it anyway.
---------------------------------------------
All about lock-free algorithms, multicore, scalability, parallel computing and related topics:
www.1024cores.net | |
Re: Scoring Criteria
You sorted out the statements。 thanks！
:D... well someone had to do it...
The organizers don't seem very interested in the task, so I thought why not me. I had to find out where I stand... ;)
| |
Re: Scoring Criteria
Would still love to have a way to find out my score so I could assess what I was doing right/wrong.
Hi,
Sorry. Who can tell me why did not get any point for this testing? I just suspect whether my solution is really evaluated by anybody? Thanks.
Best regards,
HouZhe
| |
Re: Scoring Criteria
Would still love to have a way to find out my score so I could assess what I was doing right/wrong.
Hi,
Sorry. Who can tell me why I did not get any point for this testing? I just suspect whether my solution is really evaluated by anybody? Thanks.
Best regards,
HouZhe
| |
Re: Scoring Criteria
Would still love to have a way to find out my score so I could assess what I was doing right/wrong.
Hi,
Sorry. Who can tell me why I did not get any point for this testing? I just suspect whether my solution is really evaluated by anybody? Thanks.
Best regards,
HouZhe
| |
Re: Scoring Criteria
Three test cases were used to judge execution speed. The first had 100,000 keys to search through and 20 keys to search for. In the second file 10 of these were contained in the data file and 10 were not. The second and third test cases had a search space of 100,000,000 keys; the number of keys to be searched was 100,000 in both cases. One search file had 20 good keys repeated 5000 times and the other had 100K keys that were not contained in the search space.
A maximum of five minutes overall was set for each test case. The maximum time recorded for those entries that encountered runtime errors or other execution problems was 60 seconds (since the time was only for searching and printing the results. A penalty of 15 points per test case was levied for any results that had incorrect results (finding keys not within the data, not finding keys within the data, etc.). The sum of the three test case runtimes was compared to the minimum overall time and execution points were assigned linearly based on the difference.
There were 24 submissions of code solutions, 6 on Linux and 16 on Windows. There was one entry written in Java, one in C#, one in Erlang, and the rest in C/C++. Thirteen entries had runtime issues, especially in the last two test cases.
Point spread:
100
99 99 99 99 99 99 98 93
75
69
33 33 33 33 33 33
18 18
0 0 0 0 0 0
The write-up portion of each entry was read and scored by two judges. Each judge used the 10-30-10 breakdown of points for serial algorithm description, parallel algorithm description, and performance, respectively. One important component to the judging was to determine how close the submission was for publication on ISN. The assigned score was the average of the two judges scores.
Point spread:
44 43
37 36 33 32
29 28 24 24
19 17 16 14 14 12 11 11 10
7 4
0 0 0 0
Bonus points were given for contestant’s forum posts made before the problem entries were closed. Five points per post (maximum 25 points possible) were awarded.
The overall winner was denghui0815, who also had the fastest code execution. The highest scoring write-up was submitted by Dmitriy Vyukov.
I have a similar question. Is it possible to find out about the score, where I did wrong and also whether anybody looked at the files that I uploaded into the folder with the Problem Name. Or maybe I had to specify which files should be used when I submitted the solution. Any feedback will be very helpful.
| | |
