Svyatoslav,

First of all, looking at your data one can conclude that your cluster seem to have some problems - note, for 64 processes the times differ by a factor of 3!

Second, if the problem size is rather small and is fixed for all number of processes, the computation time will increase when you increase the number of nodes - this is caused by the size of data sent from one process to another decreasing, thus increasing the latencies.

In order to utilize the full computing power of your cluster you need to challenge it with big enough transform size. In general, the best performance (in terms of gigaflops) is achieved for transforms which utilize all the memory available on each node. However, please keep in mind that due to additional buffers being allocated the local part of the data being transformed has to occupy about 25% of the local memory.

Best regards,
Vladimir