cpu_time can't be compared against QueryPerformance, as the latter doesn't separate process time.
I've not understood explanations on why Windows system_clock could not approach the performance of QueryPerformance or omp_get_wtime. This situation requires us to write applications to switch timers between linux and Windows.

Windows updates the "time of day" clock every 10ms - this is what SYSTEM_CLOCK uses. Yes, Windows has higher precision timers but they have drawbacks for general use.

You keep saying "best", "better" etc, but doesn't that depend on what you are trying to do?

What are you trying to do?

Are you timing your program, or timestamping data, or...?

      SUBROUTINE ELAPSE_SECOND (ELAPSE)

      use ifwin, only: T_LARGE_INTEGER,QueryPerformanceCounter, QueryPerformanceFrequency

!

!     Returns the total elapsed time in seconds

!     based on QueryPerformanceCounter

!     This is the fastest and most accurate timing routine

!

      real*8,   intent (out) :: elapse

!

!      STDCALL   QUERYPERFORMANCECOUNTER 'QueryPerformanceCounter' (REF):LOGICAL*4

!      STDCALL   QUERYPERFORMANCEFREQUENCY 'QueryPerformanceFrequency' (REF):LOGICAL*4

!

      real*8    :: freq  = 1

      logical*4 :: first = .true.

      integer*8 :: start = 0

      integer*8 :: num

      logical*4 :: ll

      type(T_LARGE_INTEGER) :: arg

!      integer*4 :: lute

!

!   Calibrate this time using QueryPerformanceFrequency

      if (first) then

         num   = 0

         ll    = QueryPerformanceFrequency (arg)

         num = transfer(arg,num)

         freq  = 1.0d0 / dble (num)

         start = 0

         ll    = QueryPerformanceCounter (arg)

         start = transfer(arg,start)

         first = .false.

!         call get_echo_unit (lute)

!         WRITE (lute,*) 'Elapsed time counter :',num,' ticks per second'

      end if

!

      num    = 0

      ll     = QueryPerformanceCounter (arg)

      num = transfer(arg,num)

      elapse = dble (num-start) * freq

      return

      end 
      program MAIN__

      real*8 elapse

      call elapse_second(elapse)

      write(*,*) elapse

      end program MAIN__

T_LARGE_INTEGER is in ifwinty - in my IVFv11 version, I doubt it has been moved.

use ifwin, only: QueryPerformanceCounter, QueryPerformanceFrequency

use ifwinty, only: T_LARGE_INTEGER

works

Les

T_LARGE_INTEGER is ifortspeak for the LARGE_INTEGER union that QueryPerfomanceFrequency and QueryPerformanceCounter want as a reference argument. Except for big-endian systems where the companion C processor doesn't have a 64-bit integer type, it's the same as INTEGER*8.

I was surprised to see that your machine/OS is one of those that doesn't use RDTSC as the basis for QueryPerformanceCounter. You can use RDTSC directly from ifort if you wish:

module setup

   use ifwin

   implicit none

   private

   public initialize, rdtsc, cp_rdtsc

   integer, parameter :: code32(1) = [-1866256113]

   integer, parameter :: code64(3) = [-1052233457,155721954,-1869560880]

   interface

      function rdtsc()

         integer(8) rdtsc

      end function rdtsc

   end interface

   pointer (cp_rdtsc,rdtsc)

   contains

      subroutine initialize

         integer code(*)

         pointer (ap,code)

         integer, parameter :: nbits = bit_size(ap)
         ap = VirtualAlloc(NULL,12_HANDLE,MEM_COMMIT,PAGE_EXECUTE_READWRITE)

         if(nbits == 32) then

            code(1:size(code32)) = code32

         else

            code(1:size(code64)) = code64

         end if

         cp_rdtsc = ap

      end subroutine initialize

end module setup
program main

   use setup

   implicit none

   integer i
   call initialize

   do i = 1, 10

      write(*,*) rdtsc()

   end do

end program main

My understanding is that the per thread CPU time counters are only updated at the tick rate that the scheduler uses (which is what's behind the ~60 Hz frequency that you are seeing for anything that has some sort of dependence on the scheduler). So as far as I know - no.

Your elapsed time approach has the issue that on a desktop system it will be influenced by things such as the user moving the mouse (etc.) or other background operating system activities.

When you described the problem you were trying to solve my initial though was that there must be some easier way of getting a measure of the computation effort required by a particular step than timing it - an iteration counter or similar.

When I am tuning code to see what is the fastest, the units of time have little meaning to me because I just compare the number of clock cycles and from that determine whether the changes I have made were actually an improvement and if so, whether the improvement is worth the risk and effort of incorporating the changes into the working code. Always do a loop with a few measurements so that you can see the signal and the noise. Even so, sometimes the OS throttles the processor and messes up your measurement.

I have only a minimal amount of experience timing multithreaded code and what I did in that case was RDTSC before starting the threads and again after all threads were done. Timing the progress of individual threads seems like it could get noisy as other processes and threads kick each other out of cache as they move from core to core.

To find out about GetTickCount, for example, I would google <b>GetTickCount msdn</b> and the first hit gives me useful documentation including the C prototype and the fact that you link to it via Kernel32.lib.  If you then open up %INCLUDE%\kernel32.f90 with a text editor you can search for GetTickCount and see how ifort writes its interface, nonstandard because ifort doesn't provide a STDCALL companion processor for f2003 interoperability.

Sort of a hacker's method I suppose, but it works well for me.

Zitat:

Repeat Offender wrote:

To find out about GetTickCount, for example, I would google <b>GetTickCount msdn</b> and the first hit gives me useful documentation including the C prototype and the fact that you link to it via Kernel32.lib.  If you then open up %INCLUDE%\kernel32.f90 with a text editor you can search for GetTickCount and see how ifort writes its interface, nonstandard because ifort doesn't provide a STDCALL companion processor for f2003 interoperability.

Sort of a hacker's method I suppose, but it works well for me.

If you are interested in exact machine code implementation of GetTickCount function I would advise to use IDA Pro disassembler.I suppose that this function might indirectly(when acting as a wrapper) access RTC clock.

>>>Your elapsed time approach has the issue that on a desktop system it will be influenced by things such as the user moving the mouse (etc.) or other background operating system activities.>>>

The problem lies in unpredictable and pseudorandom(from the programmer's point of view) behaviour of the scheduler.

>>>QueryPerformanceCounter, and QueryPerformanceFrequency ( 2.6 million cycles per second ) works well rdtsc  has a cycle rate = processor cycle rate (2.67 billion cycles per second ) works very well, but might have some problems. These are both elapsed time counters>>>

You can use RDTSC or QueryPerformanceCounter/QueryPerformanceFrequency timing functions(instruction) they are very accurate.RDTSC problem is the high latency and CPUID serialization which can add even more latency so for very short loops it is not recommended to use RDTSC .Afaik QueryPerformanceCounter uses HPET timer if you are interested here is an article about the HPET drawbacks : http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commit;h=54ff7e595d763d894104d421b103a89f7becf47c

Hi John

Thanks for your answer.Unfortunately I do not know fortran so I won't be able to offer you a helping hand in everything which is related to the programming in fortran. I will try to help you in everything which is related to time measurement on Windows platform.

We've pointed out several times that the OpenMP (omp_get_wtime) and MPI timers are much better than the Windows Fortran intrinsics, while system_clock (with integer(8) arguments) is satisfactory in the linux implementations of many compilers, including ifort.

This situation presents a dilemma with the implication that the timer intrinsics (other than possiblty OpenMP timers) can't be made portable between Windows and other operating systems.

On occasion, the Intel compiler team has met such challenges, as when the Microsoft __rdtsc intrinsic was added to Intel C and C++ for linux as well as Windows..

>>>Thanks for your comment. What I have been trying to highlight is the poor performance of the fortran intrinsic timers, both elapsed and CPU>>>

QueryPerformanceCouter function which could probably access as user mode wrapper HEPT timer should be able to achieve 3 ms time precision on Windows 7 platform.HEPT timer supports periodic and aperiodic time measurement which is signalled by the interuppt firing.

>>>I had trouble understanding the problems you have identified with RDTSC>>>

Sorry for not explaining it more clearly.RDTSC itself is not more recommended to use on multithreaded and frequency-throttled CPU.Main reason is that your code can be scheduled to run on different CPU and operating system(ACPI) can lower the frequency of the CPU when the load is not significiant so the RDTSC  as beign derived from the QPI clock is enough accurate for the time measurement.Microsoft recommends HEPT timer which is not dependend on the CPU.Moreover there is also an issue when you are measuring very short code blocks for example a few assembly instruction in such a situiation the measured code is shadowed by the longer latency of the RDTSC and CPUID which is used for serialization.So you need run your code thousand or hundred of times in order to remove RDTSC and CPUID latency. 

Here is a good article about the drawbacks of RDTSC and usage of this instruction to measure performance of the few instructions.

http://software.intel.com/en-us/forums/topic/306222

>>>I don’t agree with your restrictions on the windows implementation of SYSTEM_CLOCK, because it would be different to Linux. There are many differences between the two O/S>>>

I suppose that hardware timers are the same on both OS's and timer's registers as seen by the both OS's are also the same.

"results from SYSTEM_CLOCK showed that the elapsed time for the routine I tested was zero"

As others pointed out, the actual tick rate (not the count_rate) for SYSTEM_CLOCK on Windows is in the range 0.01 to 1/64 Hz so you may measure 0 time for smaller intervals.  On linux, SYSTEM_CLOCK will resolve intervals as small as microseconds when used correctly.   It's non-portable only in this sense of poor resolution on Windows.  I don't make the rules, and I agree entirety about the relative inconvenience of timing on Windows.

There are long-established benchmarks such as Livermore Fortran Kernels which perform an analysis to find out how many repetitions are needed to get satisfactory timing accuracy.  That benchmark may take half an hour to run with a timer of 1/64 Hfz resolution or as little as 3 seconds by rdtsc.  rdtsc of course exhibits degraded synchronization among CPUs in spite of the best efforts of the OS.

Benchmarks like lmbench which want to measure cache timing or other events which may take only a few CPU clock cycles must use more inconvenient techniques, with total lack of synchronization between CPUs.

Because your routine ran too fast to be precisly measured by CPU_TIME.Even Win raw thread can run for shorter period than quantum which is based on clock interrupt.

Completely agree with you.By the way if SYSTEM_CLOCK is really based on GetTickCount so the time measurement can be biased by sleep and hiberantion state.

Because of two timing providers on Win platform where one of them is so called interrupt time and the other is called system time.It is not clear which one use Fortran timing routines.For short performance measurement of code execution more accurate system time should be applied.System time functionality  is reperesented(accessed) by user mode QueryPerformanceCounter/QueryPerformanceFrequency pair of functions.

>>...Even Win raw thread can run for shorter period than quantum which is based on clock interrupt...

A test case with C/C++ sources, please! I really would like to see your test case that proves it.

And how I can predict that my thread will run shorter than quantum period when the thread is executed in unpredictable environment.If I set the lowest priority how I can know and ensure that when my thread is running before quantum tick expires other more priviledged thread will be scheduled to run.There is option to create another thread more priviledged and can someone  be sure that this thread will preempt the first thread even before the quantum expires.How you can be sure that higher interrupt will not run and preempt all those threads.Maybe you can shed some light on it.

Calling sleep(0) on currently executing thread will stop thread's execution before its quantum expires.

In pseudocode

int main(){

Handle currThHndl;

currThHandl = GetCurrentThread();

if(currThHandle == Null){

printf("Error obtaining current thread handle 0x%lx \n",GetLastError());

printf("Current thread pseudo handle is 0x%lx \n",currThHndl);

ExitProcess(0);

}

else

 printf("GetCurrentThread successfuly called current thread pseudo handle is 0x%lx\n",currThHndl);

//calling sleep function with zero argument if successful thread will relinquish its quantum time and highest priority next  ready thread wiil run.

sleep(0);

return 0;

}

Btw. the sentence quoted by you is taken from Windows Internals book in its 6 edition.And you can agree that this book was written by real experts on windows kernel.

@Sergey

What makes you think that some internal(kernel mode) OS mechanism(behaviour) can be exactly measured or estimated by user mode client code?

>>...the sentence quoted by you is taken from Windows Internals book in its 6 edition....

Hold on, please. You're always taking something from some books and not providing C/C++ sources inplemented by you which are proving, or disproving, what you've said. Sorry, but I don't see any evidence that you're doing serious programming and that is why you're quoting someone esle statements. Once again, my question is Could You prove it?

... raw thread can run for shorter period than quantum which is based on clock interrupt...

And, What is a Raw Thread? Or, What is Not a Raw thread? Could you explain by yourself? I'd like to see two C/C++ examples of Raw thread and Not Raw thread.

Of course, many IDZ users ( including me ) also quote MSDN, some articles and another docs. However, we do practical things and we can not be too theoretical all the time because many IDZ users have practical issues or problems and they need practical solutions.

>>...What makes you think that some internal(kernel mode) OS mechanism(behaviour) can be exactly measured or
>>estimated by user mode client code?

I have Not started that discussion and please don't ask me with another question until you've answered my initial question. There has to be a dialog when some technical issues are discussed.

You forgot that in some another thread I've provided a completed test-case to measure differences in values returned by RDTSC intrinsic instruction executed from several threads with accuracy to several nano-seconds ( of course this is Not absolutely accurate, however it will satisfy many performance measurement requirements ) in non-deterministic and non-realtime environment, like Windows XP or Windows 7. Actually, there are already two threads related to that subject:

Forum Topic: Synchronizing Time Stamp Counter
Web-link: software.intel.com/en-us/forums/topic/332570

Note: A test case is attached to my post dated on Tue, 11/06/2012 - 06:49

and

Forum Topic: TSC Synchronization Across Cores
Web-link: software.intel.com/en-us/forums/topic/388964

Once again, my question is Could You prove it?

Is that book   not enough for you.Do I need to prove implemetation of kernel scheduler.May you ask the authors of that book to prove thread quantum question.So according to your logic every technical sentence I will need to prove in order to satisfy you.

Raw thread it is windows thread  not raw thread could be java thread which run on Win platform.

Some code examples not related to this discussion.

Examples of Not raw threads in this case Java threads.

>>Do I need to prove implemetation of kernel scheduler.

No.

>>May you ask the authors of that book to prove thread quantum question.

I simply want to understand how it could be possible, if it is possible at all, regarding your statement about an execution of a Win32 thread. I'd like to bring clarity in your stetement and nothing else (!).

Sorry for creating a confusion.

I think that sleep() function called with arg == 0 can simulate such a behaviour when thread is stopped before quantum expires.One interesting question arises which is related to exact moment of quantum interval when the execution is postponed and how to control it programmatically.If  new thread is created from within the main function thread and that new thread has priority rised to high and it is scheduled to run immediatly after creation so how (inside thread's function) or better when sleep(0) call will be executed in order for example to stop the execution after 1/2 of quantum expires.

>>Routine Ticks per CPU cycles Notes
>> second per call
>>RDTSC 88514093 30 ticks at processor rate, accuracy limited by call rate
>>QueryPerformanceCounter 2594669 47 possibly more robust than RDTSC
>>GetTickCount 64 14 fast, but poor precision
>>system_clock 64 325
>>GetProcessTimes 64 386 poor precision but best identified for CPU
>>CPU_Time 64 387

Thanks, John! These numbers are really interesting. In 99% GetTickCount satisfies requirements I have.

Did you take into account that your test was executed in Non-Deterministic environment of some Windows operationg system? I simply wanted to say that in order to make these measurements as accurate as possible you need to boost a priority of your process to High or Realtime. In that case your process will preempt threads with lower priority currently executed on a system and they won't affect accuracy of measurements.

Also, Patrick Fay ( Intel ) recommends to do such tests on a different CPU instead of the 1st one ( it is named as CPU 0 in the Task Manager ).

Thank you John great job.

@Sergey returning to your question I have simple multithreaded Win32 threads program which uses Sleep() function to terminate its currently running thread so such a action can simulate what I wrote in one of my previous post.So far I was unable and I do not know if it is possible to relinquish the cpu time at some point during the quantum interval.

>>>While GetTickCount is faster to run ( only 14 processor cycles)>>>

Do you mean total time needed to execute this instruction from user mode stub through the switching to kernel mode?

[ Iliya wrote ]

>>...So far I was unable and I do not know if it is possible to relinquish the cpu time at some point during the quantum interval...

That's not a problem and the Negative Result is also Result because it proves or disproves something. Thanks for the update.