Comparing FFT Performance MKL11 with 1 thread and 4 threads

Comparing FFT Performance MKL11 with 1 thread and 4 threads

Portrait de Marian L.

Hi All,

I'm evaluating the performance (this time not MKL6 vs MKL11) of MKL11 with 1 thread versus 4 threads.

The 4 thread version seems to be slower. Furthermore, the 4 thread implementation has a huge number of outliners. Does anyone have any explanations, why?

Below the source (float and double are similar), I shortened it for better overview.

Main function:

int _tmain(int argc, _TCHAR* argv[])
{
   int threads = 4; //or 1
   mkl_set_num_threads(threads);

   SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS ); // Set a process priority to 'High'

   TEST FUNCTION HERE

   SetPriorityClass( GetCurrentProcess(), NORMAL_PRIORITY_CLASS ); // Restore the process priority to 'Norma'l
}

TEST FUNCTION

  DFTI_DESCRIPTOR_HANDLE hand;
   cxdTimeLoops.alloc(loops);

   //    FLOAT
   k=0;
   for (exp=exp_start;exp<=exp_stop;exp++)
   {
      Nfft = (unsigned int) pow(2.0,exp);
     
      myRndNumber = 1; //seed
     
      for (i=0;i<Nfft;i++) //get pseudo random signal
      {
         myRndNumber    = NextRand32(myRndNumber);
         cxfTimesig[i]  = ((float) myRndNumber / UINT_MAX)*2-1;
         cxfTimeaxis[i] = ((float) i + 1.0) / fs;
      }
      hand   = 0;
      status = DftiCreateDescriptor(&hand, DFTI_SINGLE, DFTI_REAL, 1, Nfft);
      status = DftiSetValue(hand, DFTI_PLACEMENT, DFTI_NOT_INPLACE);
      status = DftiCommitDescriptor(hand);
      
      for (i=0;i<loops;i++)
      {
         hpfcTimer.Start(); //start timer for single execution
         status = DftiComputeForward(hand, cxfTimesig.ptr(), cxfFreqsig.ptr());
         cxdTimeLoops[i] = hpfcTimer.Time();
      }
      DftiFreeDescriptor(&hand);
      
      dTimeMax = 0;
      dTimeMin = cxdTimeLoops[0];
      dTimeAvg = 0;
      for (i=0;i<loops;i++)
      {
         dTimeAvg += cxdTimeLoops[i];
         dTimeMax = max(cxdTimeLoops[i],dTimeMax);
         dTimeMin = min(cxdTimeLoops[i],dTimeMin);
      }
      dTimeAvg /= (double) loops;
      k++;
   }

   //    DOUBLE
   k=0;
   for (exp=exp_start;exp<=exp_stop;exp++)
   {
      Nfft = (unsigned int) pow(2.0,exp);
      cxdFreqsig.alloc(Nfft);
      cxdTimesig.alloc(Nfft);
      cxdTimeaxis.alloc(Nfft);
      
      myRndNumber = 1; //seed
      for (i=0;i<Nfft;i++)      //get pseudo random signal
      {
         myRndNumber    = NextRand32(myRndNumber);
         cxdTimesig[i]  = ((double) myRndNumber / UINT_MAX)*2-1;
         cxdTimeaxis[i] = ((double) i + 1.0) / fs;
      }

      hand   = 0;
      status = DftiCreateDescriptor(&hand, DFTI_DOUBLE, DFTI_REAL, 1, Nfft);
      status = DftiSetValue(hand, DFTI_PLACEMENT, DFTI_NOT_INPLACE);
      status = DftiCommitDescriptor(hand);
      
      for (i=0;i<loops;i++)
      {
         hpfcTimer.Start(); //start timer for single execution
         status = DftiComputeForward(hand, cxdTimesig.ptr(), cxdFreqsig.ptr());
         cxdTimeLoops[i] = hpfcTimer.Time();
      }
      DftiFreeDescriptor(&hand);
         
      dTimeMax = 0;
      dTimeMin = cxdTimeLoops[0];
      dTimeAvg = 0;
      for (i=0;i<loops;i++)
      {
         dTimeAvg += cxdTimeLoops[i];
         dTimeMax = max(cxdTimeLoops[i],dTimeMax);
         dTimeMin = min(cxdTimeLoops[i],dTimeMin);
      }
      dTimeAvg /= (double) loops;
      k++;
   }
}

dTimeAvg is plottet versus Nfft for float and double. I'm attaching the individual plots with min/max for visualizing the outliners.

Thanks, Marian

25 posts / 0 nouveau(x)
Dernière contribution
Reportez-vous à notre Notice d'optimisation pour plus d'informations sur les choix et l'optimisation des performances dans les produits logiciels Intel.
Portrait de Sridevi (Intel)

Marian,

can you please give me your machine/processor specifications? Do you observe this in every machine?

- Sridevi

Sridevi Allam Technical consulting engineer - Intel MKL
Portrait de Sergey Kostrov

>>...
>>int _tmain(int argc, _TCHAR* argv[])
>>{
>> int threads = 4; //or 1
>> mkl_set_num_threads( threads );
>>
>> SetPriorityClass( GetCurrentProcess(), HIGH_PRIORITY_CLASS );
>>...
>>The 4 thread version seems to be slower.

You've changed a priority of the main thread and I don't think it will raise priorities of OpenMP threads. Then, when MKL creates OpenMP threads they could have Normal priorities and because of this all these four threads could be preempted more often by the thread with the higher priority, that is your main application thread. Also, I'm not sure that this is the only problem in your test case. If Nfft is too big than cache related issues could negatively affect performance.

Portrait de iliyapolak

Does MKL library have a function which raises OpenMP thread's priority?

Portrait de Sergey Kostrov

>>...Does MKL library have a function which raises OpenMP thread's priority?

I don't know if MKL has it but I know that OpenMP specification 3.1 ( July 2011 ) does not have any functions to change priorities of OpenMP threads.

Portrait de Marian L.

@Sridevi: It's an Inten Core i5-2500@3.30 GHz, 8 GB RAM. I've only checked this on this computer.

@Sergey: Would you wrap the priority raising around the for loop like this:

//Set to high priority

for (i=0;i<loops;i++)
      {
         hpfcTimer.Start(); //start timer for single execution
         status = DftiComputeForward(hand, cxfTimesig.ptr(), cxfFreqsig.ptr());
         cxdTimeLoops[i] = hpfcTimer.Time();
      }

//Set to normal priority

Portrait de Victor Pasko (Intel)

Hi,

You measure FFT performance on powers of two sizes in a range [exp_start, exp_stop]  for real-to-complex.

What they are? You know, small sizes have always sequential implementation.

To check that MKL implementation is threaded please set environment KMP_AFFINITY=compact,verbose and be sure you linked with MKL threaded libraries.

 

Thanks, -- Victor
Portrait de iliyapolak

>>> I know that OpenMP specification 3.1 ( July 2011 ) does not have any functions to change priorities of OpenMP threads.>>>

So the priorities of OpenMP thread is hardcoded to be Normal.

Portrait de Sergey Kostrov

Marian,

>>SetPriorityClass( GetCurrentProcess(), HIGH_PRIORITY_CLASS );

Did you try to comment that?

Portrait de Victor Pasko (Intel)

Hi,

Setting high-priority may not help and used just for getting performance stability on overloaded machine.

However, you can initialize OMP-threads in your program easily with required priority before using MKL functions:

#pragma omp parallel

{

   SetThreadPriority( GetCurrentThread(),THREAD_PRIORITY_HIGHEST);

}

so that MKL will use these theads doing parallel FFTs.

BTW, how many real CPUs are on your machine. And what about HT (hyper-threading)?

Thanks, -- Victor
Portrait de Marian L.

@Victor:

What they are? You know, small sizes have always sequential implementation.

The FFT length is written in the absissa, so the exponent runs from 10 to 20.

BTW, how many real CPUs are on your machine. And what about HT (hyper-threading)?

Intel Core i5-2500@3.30 GHz, according to this it has no hyper threading

SetThreadPriority( GetCurrentThread(),THREAD_PRIORITY_HIGHEST);

Sadly, that does not change the performance significantly.

@Sergey:

>>SetPriorityClass( GetCurrentProcess(), HIGH_PRIORITY_CLASS );

Did you try to comment that?

Yes, but it sets the priority of the whole process, not threads. So the placement should be irrelevant, correct?

Portrait de Sergey Kostrov

>>...Yes, but it sets the priority of the whole process, not threads. So the placement should be irrelevant,
>>correct?

Yes, and this is what I've seen in my tests in 2012.

Marian, I'll try to investigate it again and then post my new results ( I can't tell when it will be done ). Also, I actually have that task on my list for a long time and it's a right time to look into that.

Portrait de Marian L.

Sergey, thanks for your help. If you need some of my testing code or gnuplot export functions, let me know.

Portrait de iliyapolak

@marian

Before running your tests can you measure overall sytem load?I would suggest to do it with xperf tool.There is possibility that your threads are preempted by code which is running at high IRQL(like driver's routines).As it was suggested in other post by Sergey it is recommended to disable some of the unneded windows services and even disable some of the hardware loke network cards.

Portrait de Sergey Kostrov

>>... If you need some of my testing code or gnuplot export functions, let me know...

Yes, it would be nice to look at it. Thanks, Marian.

Portrait de Marian L.

@ iliyapolak: Thanks for your comment. If the performance is not good with my system, it is very likely the same on a customer's computer. I agree it could be some driver or something else. But I can't go into details of the root cause here, because I don't know what is on other systems, that I cannot control.

@Sergey, I'll send you a pm.

Portrait de Sergey Kostrov

>>...There is possibility that your threads are preempted by code which is running at high IRQL (like driver's routines)....

Many software developers simply do not care what IRQL, or a driver routing, etc, are in essence. If a software developer has a problem he / she could ask a question on a forum and a solution / proposal / hands-on-R&D, test, etc is / are expected ( ideally ). I see that in Marian's case a real practical investigation is really needed what is going on with MKL v11. It is clear that performance has degraded compared to older versions of MKL.

Best regards,
Sergey

Portrait de iliyapolak

>>>I see that in Marian's case a real practical investigation is really needed what is going on with MKL v11. It is clear that performance has degraded compared to older versions of MKL.>>>

As far as it concerns performance of some program/software at the beginning of the investigation you cannot exclude anything.I completely agree with you that any software developer should not care about the IRQL's and driver's routines,but in the case of software performance problems everything must be taken into account even system load.

Portrait de iliyapolak

@Marian

I was not talking about the other system.Testing your code on another machine is important to understand the root cause.Regarding the problem I believe that sometimes you software performance can degrade because of interference from the OS (it's services) and the other code maybe more priviledged one.

Portrait de Evgueni Petrov aka espetrov (Intel)

Marian,

The best time observed by the benchmark scales (decreases with the number of threads) but this best time is dominated by instability of measurement.

Here are some tips to stabilize measurements.

  1. Pin threads to CPU cores using the KMP_AFFINITY environment varibale or the Windows API for thread affinity
  2. Ensure the benchmark single-threaded; if your use-case is multi-threadedm you may want to look through http://software.intel.com/en-us/articles/different-parallelization-techniques-and-intel-mkl-fft 
  3. Prevent the cache warm-up time from dominating your performance measurement -- either increase the value of the loops variable in your code, or exclude from measurement the first call to DftiComputeForward for each Nfft.

Please let us know if the above tips help.

Thanks,

Evgueni.

Portrait de Evgueni Petrov aka espetrov (Intel)

Marian,

The best time observed by the benchmark scales (decreases with the number of threads) but this best time is dominated by instability of measurement.

Here are some tips to stabilize measurements.

  1. Pin threads to CPU cores using the KMP_AFFINITY environment varibale or the Windows API for thread affinity
  2. Ensure the benchmark single-threaded; if your use-case is multi-threadedm you may want to look through http://software.intel.com/en-us/articles/different-parallelization-techniques-and-intel-mkl-fft 
  3. Prevent the cache warm-up time from dominating your performance measurement -- either increase the value of the loops variable in your code, or exclude from measurement the first call to DftiComputeForward for each Nfft.

Please let us know if the above tips help.

Thanks,

Evgueni.

Portrait de Sergey Kostrov

Hi Marian,

If this is a real problem for your project please try to contact Intel Premier Support. I don't think something could be done on our sides since this is an internal issue with the latest version of MKL. I hope that Intel software engineers will look into it.

Best regards,
Sergey

Portrait de Evgueni Petrov aka espetrov (Intel)

Marian,

The best time observed by the benchmark scales (decreases with the number of threads), but the average time is dominated by instability of measurement.

Here are some tips to stabilize measurements.

  1. Pin threads to CPU cores using the KMP_AFFINITY environment varibale or the Windows API for thread affinity
  2. Ensure the benchmark is single-threaded; if your use-case is multi-threaded, you may want to look through http://software.intel.com/en-us/articles/different-parallelization-techniques-and-intel-mkl-fft 
  3. Prevent the cache warm-up time from dominating your performance measurement -- either increase the value of the loops variable in your code, or exclude from measurement the first call to DftiComputeForward for each Nfft.

Please let us know if the above tips help.

Thanks,

Evgueni.

Portrait de Sergey Kostrov

Marian, Could you try to compile the test case with /Qopenmp-report{0|1|2} option? It will be nice to see these reports.

Thanks in advance.

Portrait de Marian L.

Thank you all for the comments. I will try to do it before the evaluation period runs out.

Connectez-vous pour laisser un commentaire.