Reducing the amount of data collected by vtsarun

Log in to post comments

Sun, 01/11/2009 - 05:17

> Obviously I could change the sampling interval

This is a good catch

> It might be possible to manually set collection off using the -ec parameter, but specifying that for each event other than the one I'm interested in is unpleasant.

I am not sure I understand it. The -ec option is for specifying the events you want to collect and not for turning off these you do not need.

Here are some samples to illustrate my words:

1) default sampling

> vtsarunexp -s -d 5
===== RUN 1 =====

Duration (sec): 5
Events configuration:

"CPU_CLK_UNHALTED.CORE"
"INST_RETIRED.ANY"

2) custom sampling with one event

> vtsarunexp-clk-only -s -d 5 -ec CPU_CLK_UNHALTED.CORE
===== RUN 1 =====

Duration (sec): 5
Events configuration:

"CPU_CLK_UNHALTED.CORE"

Log in to post comments

edwh

Mon, 01/12/2009 - 07:43

Quoting - Konstantin Lupach (Intel)
I am not sure I understand it. The -ec option is for specifying the events you want to collect and not for turning off these you do not need.

Thanks, I'd misinterpreted the usage message.

That reduces the data, but not enough for my needs. So I'll need to write a wrapper script to do shorter runs and then record the last n files of data, or something like that.

Log in to post comments

Konstantin Lupach (Intel)

Tue, 01/13/2009 - 03:55

Quoting - edwh
Thanks, I'd misinterpreted the usage message.

That reduces the data, but not enough for my needs. So I'll need to write a wrapper script to do shorter runs and then record the last n files of data, or something like that.

Can you please describe your needs and your performance tuning use cases. If we find them enough common and beneficial we will probably provide somesupport in PTU or future products.

Generally, the best way to reduce the ammount of data is to specify exactly the events you need and specify some high enough SAV. Thisshould also allow to filter out many insignificant hotspots. However, as I understood it does not serve you well for some reason.

Log in to post comments

edwh

Tue, 02/03/2009 - 02:59

Quoting - Konstantin Lupach (Intel)
Can you please describe your needs and your performance tuning use cases. If we find them enough common and beneficial we will probably provide somesupport in PTU or future products.

Generally, the best way to reduce the ammount of data is to specify exactly the events you need and specify some high enough SAV. Thisshould also allow to filter out many insignificant hotspots. However, as I understood it does not serve you well for some reason.

Is SAV the sampling frequency?

I'm trying to track down an intermittent performance problem which causes brief spikes of activity. I therefore need both to run for a long time (because it's intermittent) but to have fine granularity of sampling (because the spike is brief).

Log in to post comments

Konstantin Lupach (Intel)

Tue, 02/03/2009 - 03:15

Quoting - edwh
Is SAV the sampling frequency?

Right. SAV = sample after value.

Quoting - edwh
I'm trying to track down an intermittent performance problem which causes brief spikes of activity. I therefore need both to run for a long time (because it's intermittent) but to have fine granularity of sampling (because the spike is brief).

It looks like you need a kind of sampling overtime view that is currently missed in PTU. I would suggest you to complement your analysis with the Intel VTune Performance Analyzer that has this ability.

To avoid any compatibility and co-existence issues install the following versions of products:

- VTune analyzer 9.1 Update 1

- PTU 3.2 Update 1

Log in to post comments

edwh

Tue, 02/03/2009 - 05:31

Quoting - Konstantin Lupach (Intel)
Right. SAV = sample after value.

Quoting - edwh
I'm trying to track down an intermittent performance problem which causes brief spikes of activity. I therefore need both to run for a long time (because it's intermittent) but to have fine granularity of sampling (because the spike is brief).

It looks like you need a kind of sampling overtime view that is currently missed in PTU. I would suggest you to complement your analysis with the Intel VTune Performance Analyzer that has this ability.

To avoid any compatibility and co-existence issues install the following versions of products:

- VTune analyzer 9.1 Update 1

- PTU 3.2 Update 1

Actually I'm trying just now to open up .tb5 files collected by vtsarun within the VTune GUI. I get a "Summary information can not be obtained" popup, and in the output I see:

Tue Feb 03 13:26:27 2009 An error occurred during drill-down.
Cause of error cannot be determined.

This is after running vtsaview --convert on the .tb5 file I collected using vtsarun.

I'm running VTune 9.1 (but not Update 1) and PTU 3.1 Update 3, so I'll try upgrading. I can find the PTU update, but haven't yet found the VTune one.

Log in to post comments

edwh

Tue, 02/03/2009 - 07:26

Quoting - edwh
Actually I'm trying just now to open up .tb5 files collected by vtsarun within the VTune GUI. I get a "Summary information can not be obtained" popup, and in the output I see:

Tue Feb 03 13:26:27 2009 An error occurred during drill-down.
Cause of error cannot be determined.

This is after running vtsaview --convert on the .tb5 file I collected using vtsarun.

I'm running VTune 9.1 (but not Update 1) and PTU 3.1 Update 3, so I'll try upgrading. I can find the PTU update, but haven't yet found the VTune one.

I've installed the new PTU, and Update 1 (which is probably what I had installed already - Help doesn't show Update 1 in this version either, just says Build 187). I'm still seeing the same error when trying to load results. Any suggestions?

Log in to post comments

Konstantin Lupach (Intel)

Wed, 02/04/2009 - 03:31

Quoting - edwh
I've installed the new PTU, and Update 1 (which is probably what I had installed already - Help doesn't show Update 1 in this version either, just says Build 187). I'm still seeing the same error when trying to load results. Any suggestions?

This is the right VTune build to use.

Log in to post comments

edwh

Wed, 02/04/2009 - 03:43

Quoting - Konstantin Lupach (Intel)
This is the right VTune build to use.

Thanks for your reply. So what am I doing wrong when I try to load information collected using vtsarun?

I'll clarify my situation a bit more. I have a long-running stress test, which lasts for many hours. During this there might be several intervals of a second or two where something happens which I need to investigate.

Running VTune using the RDC for the duration of the whole test would produce more data than it's practical to manipulate using the GUI, I think, and more than I can store on disk.

So what I've done is write a script which
- runs vtsarun to collect information for five minutes
- do a check on external events to see if that was an interval I need to investigate
- if so, save off those diags
- loop back

That way I have much less data to maintain. However I can't load the vtsarun data into VTune, which means it's much harder to analyze (for example, the over time view is crucial).

Log in to post comments

Konstantin Lupach (Intel)

Wed, 02/04/2009 - 03:53

I installed on my machine VTune 9.1 U1 and PTU 3.2 U1.
Collected data with PTU 3.2 U1 (some small application) and successfully opened this tb5 in VTune.
I was able to see overtime for processes, modules, and threads.

We need to figure our what is wrong with your file.
To make it happen I would suggest the following:
- go to to ptu/bin
- run install_sep (this is to make sure you use a proper driver)
- create a file using vtsarun and back it up before opening by any tool (1-st opening in both VTune and PTU modifies it)
- try to open a copy in VTune
- if it fails, provide the original tb5 file
- ifVTune fails only for the files that were opened at least once by vtsaview or PTU GUI then it is a bug but you have a workaround with original file usage

Log in to post comments

edwh

Wed, 02/04/2009 - 04:05

Quoting - Konstantin Lupach (Intel) I installed on my machine VTune 9.1 U1 and PTU 3.2 U1.
Collected data with PTU 3.2 U1 (some small application) and successfully opened this tb5 in VTune.
I was able to see overtime for processes, modules, and threads.
We need to figure our what is wrong with your file.
To make it happen I would suggest the following:
- go to to ptu/bin
- run install_sep (this is to make sure you use a proper driver)
- create a file using vtsarun and back it up before opening by any tool (1-st opening in both VTune and PTU modifies it)
- try to open a copy in VTune
- if it fails, provide the original tb5 file
- ifVTune fails only for the files that were opened at least once by vtsaview or PTU GUI then it is a bug but you have a workaround with original file usage

Thanks, will do that and report back. If I get a problematic tb5 file, do you want me to attach it here or email it? If email, to which address?

Log in to post comments

edwh

Wed, 02/04/2009 - 05:07

Quoting - edwh
Thanks, will do that and report back. If I get a problematic tb5 file, do you want me to attach it here or email it? If email, to which address?

Attached are two files, the original tb5 file and the version created by vtsaview --convert.

Log in to post comments

Konstantin Lupach (Intel)

Wed, 02/04/2009 - 06:52

Quoting - edwh
Attached are two files, the original tb5 file and the version created by vtsaview --convert.

It looks like you did not finalize this not straight forward procedure.
You should:
- create a folder
- add there your file(s)
-click an addedfolder if you are on the folders level
-click an addedfile
- check add as attachment or add to editor

I also did not manage to do it without help:)

Log in to post comments

Konstantin Lupach (Intel)

Wed, 02/04/2009 - 06:56

Quoting - edwh
Attached are two files, the original tb5 file and the version created by vtsaview --convert.

I fact I am able to open both the original file I collected with vtsarun and the file after vtsaview -c.
Need to check what is wrong with yours files.

Log in to post comments

edwh

Wed, 02/04/2009 - 08:01

Quoting - Konstantin Lupach (Intel)
I fact I am able to open both the original file I collected with vtsarun and the file after vtsaview -c.
Need to check what is wrong with yours files.

Ok, thanks - let me know if you need more diags.

Log in to post comments

Konstantin Lupach (Intel)

Wed, 02/04/2009 - 11:15

Quoting - edwh
Ok, thanks - let me know if you need more diags.

I told that I was able to open the files collected on my machine, however,I did not have an opportunity to try your files. Please attach them properly (as I described above). Thank you in advance.

Log in to post comments

edwh

Thu, 02/05/2009 - 02:28

Quoting - Konstantin Lupach (Intel)
I told that I was able to open the files collected on my machine, however,I did not have an opportunity to try your files. Please attach them properly (as I described above). Thank you in advance.

Trying again to attach.

Attachments:

Attachment	Size
Download orig.tb5	3.57 MB
Download converted.tb5	6.49 MB

Log in to post comments

Konstantin Lupach (Intel)

Thu, 02/05/2009 - 03:26

Quoting - edwh
Trying again to attach.

Downloading them now. Will check today.
Next time when attaching tb5 files please zip them in advance.
Will keep you updated on the progress

Log in to post comments

Konstantin Lupach (Intel)

Thu, 02/05/2009 - 04:18

Quoting - edwh
Trying again to attach.

I managed to reproduce the issue with your data.
The nature of the issue is in the following behaviour.
When VTune reads an activity result (tb5 file) that contains only only event and this event is clockticks-like (CPU_CLK_UNHALTED.CORE in your case) it tries to open forcibly the Summary View. It happens only in case when VTune failed to resolve any functions for the modules that got samples during the activity/experiment.

You anyway needbinaries and functions resolved from themto doperformance analysis so just make available for VTune your application binaries. You can use search directories or point needed binaries explicitly when VTune asks for them.

In case binaries for some reason are not available you can do the hack I did:
- open your tb5 file in a binary editor
- find CPU_CLK_UNHALTED.CORE
- change any symbol so VTune does not recognize it as clockticks anymore (e.g. CPU_CLK_UNHALTED.CORR)

Using one of these ways you will be able to open the Overtime View from hotspots by modules/processes/threads.
To see the functions or address hotspots you should:
- selectthe time range on the overtime picture
- press zoom-in (+ button under the view)
- press sigma button (Display Regular Sampling View for the Selected Time Range)

Note that leads to creation an activity result subnode in the tuning browser Window. Working with the overtime picture of this subnode you can not use zoom-in and sigma again

Log in to post comments

edwh

Thu, 02/05/2009 - 10:21

Quoting - Konstantin Lupach (Intel)
I managed to reproduce the issue with your data.
The nature of the issue is in the following behaviour.
When VTune reads an activity result (tb5 file) that contains only only event and this event is clockticks-like (CPU_CLK_UNHALTED.CORE in your case) it tries to open forcibly the Summary View. It happens only in case when VTune failed to resolve any functions for the modules that got samples during the activity/experiment.

You anyway needbinaries and functions resolved from themto doperformance analysis so just make available for VTune your application binaries. You can use search directories or point needed binaries explicitly when VTune asks for them.

In case binaries for some reason are not available you can do the hack I did:
- open your tb5 file in a binary editor
- find CPU_CLK_UNHALTED.CORE
- change any symbol so VTune does not recognize it as clockticks anymore (e.g. CPU_CLK_UNHALTED.CORR)

Using one of these ways you will be able to open the Overtime View from hotspots by modules/processes/threads.
To see the functions or address hotspots you should:
- selectthe time range on the overtime picture
- press zoom-in (+ button under the view)
- press sigma button (Display Regular Sampling View for the Selected Time Range)

Note that leads to creation an activity result subnode in the tuning browser Window. Working with the overtime picture of this subnode you can not use zoom-in and sigma again

That's very helpful indeed, thankyou. I'll try your hack or making the binaries available on the analysing machine.

Log in to post comments

Konstantin Lupach (Intel)

Tue, 02/10/2009 - 03:52

Quoting - edwh
That's very helpful indeed, thankyou. I'll try your hack or making the binaries available on the analysing machine.

You can install VTune analyzer 9.1 U1 for Linux on the same machine to simplify analysis with two products.

Log in to post comments

edwh

Wed, 02/11/2009 - 09:38

Thanks, editing the stat name in the binary file enabled me to process the .tb5 file fine. Good tip, and thanks for the help.

Intel® Developer Zone

Reducing the amount of data collected by vtsarun

Reducing the amount of data collected by vtsarun

Attachments:

Login to leave a comment.