runSerialBodies()
{
...
// Run the simulation over a fixed range of time steps
for (double s = 0.; s < STEPLIMIT; s += TIMESTEP)
{

  // Compute the accelerations of the bodies
  for (i = 0; i < n - 1; ++i)
  {
   for (j = i + 1; j < n; ++j)
   {

     // compute the distance between them
     double dx = body[i].pos[0]-body[j].pos[0];
     double dy = body[i].pos[1]-body[j].pos[1];
     double dz = body[i].pos[2]-body[j].pos[2];

     double distsq = dx*dx + dy*dy + dz*dz;

     if (distsq < MINDIST)distsq = MINDIST;

     double dist = sqrt(distsq);

     // compute the unit vector from j to i
     double ud[3];
     ud[0] = dx / dist;
     ud[1] = dy / dist;
     ud[2] = dz / dist;

     // F = G*mi*mj/distsq, but F = ma, so ai = G*mj/distsq
     double Gdivd = GFORCE/distsq;
     double ai = Gdivd*body[j].mass;
     double aj = Gdivd*body[i].mass;
     // apply acceleration components using unit vectors
     for (int k = 0; k < 3; ++k)
     {
        body[j].acc[k] += aj*ud[k];
        body[i].acc[k] -= ai*ud[k];
      }
    }
  }

  // Apply acceleration and advance bodies
  for (i = 0; i < n; ++i)
  {
    for (j = 0; j < 3; ++j)
    {     
       body[i].vel[j] += body[i].acc[j] * TIMESTEP;
       body[i].pos[j] += body[i].vel[j] * TIMESTEP;
       body[i].acc[j] = 0.;
     }
  }

}
...
}

// compute the unit vector from j to i
double ud[3];
ud[0] = dx / dist;
ud[1] = dy / dist;
ud[2] = dz / dist;

// compute the unit vector from j to i
double ud[3];
double dd = 1.0 / dist;
ud[0] = dx * dd;
ud[1] = dy * dd;
ud[2] = dz * dd; 

In this article, you will find out how hardware based Event Based Sampling (EBS) technology can help developers estimate the floating point operations per second executed by their applications. FLOPS will refer to 32 bit and 64 bit floating point operations and the operations will be either addition or multiplication (computational).

The Intel® VTune^TM Amplifier XE is a performance analysis tool, which can help the software developers analyze their application to identify algorithmic and microarchitectural performance issues. The VTune^TM Amplifier XE uses the processor's Performance Monitoring Unit (PMU) to sample processor events and some of these processor events can be used to statically sample the number of computational floating point operations at execution.

Figure 1: Scalar processing vs. SIMD (Single Instruction Multiple Data) processing

Figure 2: Intel® Architecture integer, floating point, MMX and SSE (Streaming SIMD Extensions) registers.

Note: The figure doesn't show the latest AVX extension and registers.

As Figure 1 and 2 demonstrate, floating point operations can be performed on legacy x87 registers or on SSE registers, depending on how the compiler generates the code. If the floating point instructions are executed on SSE registers, then they can be either scalar or packed operations. Table 1 (below) gives the PMU event names which can be used to statistically estimate the computational floating point operations executed by the hardware. It is a good idea to keep in mind that not all the executed instructions, hence counted by these events, are retired due to speculative nature of the architecture. Therefore, it is possible to experience overcounting of these events.

Table 1: PMU events are used to count the computational floating point operations at execution.

Note: This table does not include the event names used to sample AVX FP operations

The VTune^TM Amplifier XE can use any of the events or all of them at the same time to estimate the flops achieved by an application. In order to measure the elapsed time, the CPU_CLK_UNHALTED (a.k.a clockticks) event can be used. If the processor frequency is constant during the measuring period, you can use the clockticks event to calculate the elapsed wall clock time. Please keep in mind that the CPU_CLK_UNHALTED event name might vary by the processors architecture.

Alternatively, CPU_CLK_UNHALTED.REF, which counts the number of reference cycles and is not affected by thread frequency changes, can be used. The difference between the reference clocktick event and clocktick event is that even if a thread enters the halt state (by running the HLT instruction), the reference clocktick event continues to count as if the thread is continuously running at the maximum frequency.

Estimating FLOPS

The FLOPS formula can be given as follows:

FLOPS = ((number of FP ops / clock) * number of total computational FP ops) / Elapsed Time

Elapsed Time = CPU_CLK_UNHALTED / Processor-Frequency / Number-of-Cores.
Note: The cores with non zero CPU_CLK_UNHALTED event count needs to be considered for this formula.

To demonstrate how EBS technology can be used to estimate the FLOPS, a simple multi-threaded matrix multiplication will be used. Each thread in the thread pool executes the following code.

double a[NUM][NUM];

double b[NUM][NUM];

double c[NUM][NUM];

...

slice = (unsigned int) tid;

from  = (slice * NUM) / NUM_THREADS;

to    = ((slice + 1) * NUM) / NUM_THREADS;

 

for(i = from; i < to; i++) {

for(j = 0; j< NUM; j++) {

for(k = 0; k < NUM; k++) {

// 2 fp ops / iteration: 1 add, 1 multiply
c[i][j] += a[i][k] * b[k][j];

}

}

}

...

The application also reports the flops measured by dividing the total FP operations ( 2 / iteration * NUM * NUM * NUM) with the elapsed time. The elapsed time only includes matrix multiplication part and doesn't include the initialization and thread creation overhead.

In order to collect samples for the relevant code section __itt_pause() (pauses the collection) and __itt_resume() (resumes the collection) APIs are used. Please refer to VTune^TM Amplifier XE documentation on how to use the user APIs.

VTune^TM Amplifier XE can be configured as follows on Intel® Core^TM i7 (x980) based system (3.33GHz, 6 core + Hyper Threading enabled):

Using x87 Registers

The sample application is compiled in released mode (optimization level set to 0x) on a Windows* system using Visual Studio

The application reports the following when analyzed under VTune^TM Amplifier XE.

The results below give us insight on how the compiler generated the code.  In this run, we can clearly see that we only collected samples on FP operations using x87.

If we plug the numbers into the formula:

MFLOPS Formula = FP_COMP_OPS_EXE.FP / 1x10⁶ / Elapsed Time

Elapsed time = CPU_CLK_UNHALTED.THREAD / Processor-Frequency / Number-of-Cores

Elapsed Time = 607,652,000,000.00 / 3.33 x 10⁹ / 12 = 15.206 secs

MFLOP = 18,470,000,000.00 / 1x10⁶/ 15.206 secs = 1,214.652 MFLOPS

Using SSE registers

Now, let's look at the same application when SSE registers are used.  If we compile the application using Intel® compiler version 12.0, we see the following results under the VTune^TM Amplifier XE.

One thing you will notice right away in the new result displayed is the difference in the function names where the samples are happening.  In the earlier example, we were getting the samples in matrixMultiply function, but now we see the samples in threadPool function.  This is due to inlining (for more information: http://en.wikipedia.org/wiki/Inline_expansion). Drilling down into the threadPool makes this clear.

We multiply FP_COMP_OPS_EXE.SSE_DOUBLE_PRECISION event by 2 because two packed double precision floating operations can be performed on 128 bit XMM registers in every clock. For single precision floating point operations, the total count for packed single precision floating operations needs to be multiplied by 4.

MFLOPS Formula = 2 * FP_COMP_OPS_EXE.SSE_DOUBLE_PRECISION / 1x10⁶ / Elapsed Time

Elapsed time = CPU_CLK_UNHALTED.THREAD / Processor-Frequency / Number-of-Cores

Elapsed time = (66,178,000,000 / 3.33 x10⁹ / 12 ) =  1.656 secs

MFLOPS = 2 * FP_COMP_OPS_EXE.SSE_DOUBLE_PRECISION / 1 x 10⁶ / 1.656 secs =  11,053.140 MFLOPS

Windows*

1. Right-click on "My Computer", then click "Properties"
2. Click on "Advance" tab
3. Under "Startup and Recovery", click "Settings"
4. Under "Write debugging information", select "Small memory dump (64KB)"
5. Default directory "C\Windows\Minidump" for "Small dump directory:"
6. Click "OK" button.
7. Use regedit.exe to verify item named "CrashDumpEnabled" value is 0x3 under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl
8. Restart the machine
9. If software crash happens, Minixxxx-01.dmp will be generated

Note: usually developer uses WinDbg to know what and why caused this failure.

Linux*

1.   Use "ulimit -c" to check the switch of generating core file. If result is zero, the switch is disabled, will not generate core file

2. Use "ulimit -c filesize" to enable this switch (filesize is KB), or use "ulimit -c unlimited". Using filesize might cause file truncated, gdb will report error.
3. If the user met software crash, core dump file (like as core.xxxx) will be generated
4. Disable "core dump" function by using "ulimit -c 0"

Note:  the developer will use "gdb -c corefile problematical-executable", "bt" command in gdb to know what happened, with call stack info.

When the user ran Call graph to statically instrument EXE or some DLLs, met error message such as "Error in module ###.dll - Module is an instrumented module"

Cause:

Call graph copies (saves) original user's EXE/DLLs to Call graph's cache directory first, then put instrumented EXE/ DLLs in their original directories respectively. After doing data collection, copies original modules from cache directory to original directories, and keeps instrumented modules to Call graph's cache directory. Next time to run Call graph again, user's modules will not be instrumented (assume that source code are not changed, and don't rebuild modules), get them from Call graph's cache to save time.

Usually this problem occurred if Call graph was aborted abnormally by user, original modules might be corrupted, or can't be restored into original directories. This interferes next time Call graph's running.
 

Solution:

1. Empty Call graph cache directories first.
2. Secondary rebuild all influenced modules

[DISCLAIMER: The information on this web site is intended for hardware system manufacturers and software developers. Intel does not warrant the accuracy, completeness or utility of any information on this site. Intel may make changes to the information or the site at any time without notice. Intel makes no commitment to update the information at this site. ALL INFORMATION PROVIDED ON THIS WEBSITE IS PROVIDED "as is" without any express, implied, or statutory warranty of any kind including but not limited to warranties of merchantability, non-infringement of intellectual property, or fitness for any particular purpose. Independent companies manufacture the third-party products that are mentioned on this site. Intel is not responsible for the quality or performance of third-party products and makes no representation or warranty regarding such products. The third-party supplier remains solely responsible for the design, manufacture, sale and functionality of its products. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. *Other names and brands may be claimed as the property of others.]

However there is other way to do this better - that is, ActivityController can provide more powerful functions, such as Stop/Cancel/Pause/Resume Activity. In this way, the user doesn't need to click buttons on VTune(TM)Analyzer's graphic user interface! 

Please note if the user has no activity running, ActivityController will report "You are not currently running any Activities!"

ActivityController supports any situation of data collection, only if the user has started activity running.

Here are two examples of running VTune(TM) Analyzer under graphic user interface and command line. Please see their outputs after using ActivityController.

1) VTune(TM) Analyzer starts with paused mode

2) VTL command works as long run sampling session

[DISCLAIMER: The information on this web site is intended for hardware system manufacturers and software developers. Intel does not warrant the accuracy, completeness or utility of any information on this site. Intel may make changes to the information or the site at any time without notice. Intel makes no commitment to update the information at this site. ALL INFORMATION PROVIDED ON THIS WEBSITE IS PROVIDED "as is" without any express, implied, or statutory warranty of any kind including but not limited to warranties of merchantability, non-infringement of intellectual property, or fitness for any particular purpose. Independent companies manufacture the third-party products that are mentioned on this site. Intel is not responsible for the quality or performance of third-party products and makes no representation or warranty regarding such products. The third-party supplier remains solely responsible for the design, manufacture, sale and functionality of its products. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. *Other names and brands may be claimed as the property of others.]

Application Crashes or No Data Collected When Attempting Call Graph Profiling

• Gray: Modules you added to the project from the Application/Module Profile Configuration dialog box, or from the Call Graph Configuration Wizard.
• Blue: Modules added during call graph instrumentation as dependencies of the selected modules.
• White: Modules you added via Add button, or added during run-time instrumentation.