Q. for John Readey

Q. for John Readey

Hi John,

I think I'm close to my goal of being able to use the nice 3D viewing capabilities in the astsrojet.h5 example within my Fortran code, but I've run into a snag. I hope you'll be able to tell me whether what I'm trying to do is achievable by someone with my rudimentary understanding of AV.

At this stage, I can create an xml file with data for an NxNxN array, and most important all the clever javascript and html to view it (ala astrojet.h5), open it with avOpen() and view it with avNewViewer(). Now what I want to do is change the data in the array and view the changes. I thought I could use avStartWatch(), but this creates a new dataset. I then looked into getting the object handle, and using the avWriteElement() method, but the linker says avWriteElement is an unrecognized symbol, as is avReadElement.

Is there a way to open an AV file containing a dataset and graphs (like astrojet.h5), view a graphs then write to the dataset that is being displayed? This not quite the same as the update2 sample, in which graphs are created after calling avStartWatch(). I feel that I should be able to do something with the object handle that I've retrieved.

Edit: I have now found the AvObjMod module, and have learned that I can indeed use avWriteElement() to modify the dataset that I am viewing. This is great! My only question, then, is whether this is the best way to do it. It would be great to be able to write directly to the dataset memory, but I get the impression that to do this requires code written in C++.

Any suggestions will be much appreciated
Thanks
Gib

15 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

>>I thought I could use avStartWatch(), but this creates a new dataset.

Recode to use avStartWatch to create/update your only dataset.

Jim

Quoting - jimdempseyatthecove

>>I thought I could use avStartWatch(), but this creates a new dataset.

Recode to use avStartWatch to create/update your only dataset.

Jim

I tried several coding approaches, maybe you can show me the way. I'll explain why I gave up on avStartWatch.

I have an xml file, template.xml, that contains a considerable amount of html and javascript to create the various graphs that I want to use. I pinched this code from astrojet.h5 (after converting it to xml). I modified the astrojet file slightly and replaced numbers relating to the array dimensions with strings that I can later detect and replace.

This is the procedure that works. In my Fortran program I read template.xml and write test.xml with my dataset (dset) and appropriately modified html and javascript to create the graphs. Now I can open test.xml with avOpen and view the data with the 3D graphs, and use avGetObject and avWriteElement to change the array one entry at a time and view the changes while live.

When I try using avStartWatch after the avOpen, asking to watch dset, it creates a second dataset called dset(2). If I use avStartWatch without having opened test.xml then dset is the only dataset, but I do not have the graphs that I need. How can I get access to the graph code in test.xml without loading the dset dataset? If I open test.xml with AV, delete dset and save the file as test_null.xml, this new file with just the graph code, no dataset, cannot be opened by AV (not surprisingly).

Am I missing something obvious?

I see, you are using the external file to contain your charts, graphs, and plots (as opposed to creating them within your application). And you are finding a performance problem using avWriteElement and would prefer to use A(i) = newValue

What may work is to use the application memory as a cache (think of it as if it were)

Open your test.xml file

Use avStartWatch on a shadow array (or arrays) named differently than that in your test.xml file.

Using avGetObject and avReadElement, copy test.xml data into differently named arrays in the application

Now modify the properties in charts, graphs, and plots to replace the path to the data in the test.xml to point to the new copy. Now your plots will use the copies from the shared memory (assuming you avUpdate when necessary).

When done running, if required, use avWriteElement to copy the "cached" data back to the test.xml data.

Then delete the database objects maping your "cached" data.

Then close the test.xml

Jim Dempsey

Quoting - jimdempseyatthecove

I see, you are using the external file to contain your charts, graphs, and plots (as opposed to creating them within your application). And you are finding a performance problem using avWriteElement and would prefer to use A(i) = newValue

What may work is to use the application memory as a cache (think of it as if it were)

Open your test.xml file

Use avStartWatch on a shadow array (or arrays) named differently than that in your test.xml file.

Using avGetObject and avReadElement, copy test.xml data into differently named arrays in the application

Now modify the properties in charts, graphs, and plots to replace the path to the data in the test.xml to point to the new copy. Now your plots will use the copies from the shared memory (assuming you avUpdate when necessary).

When done running, if required, use avWriteElement to copy the "cached" data back to the test.xml data.

Then delete the database objects maping your "cached" data.

Then close the test.xml

Jim Dempsey

That's right, Jim, it's all about using the existing excellent graphing code (presumably the work of John Readey). My approach works, but making a subroutine call to update each array entry chews up a lot of CPU, and if I were to update the display on a continuous basis it would slow my program down too much. It would be very nice to be able to write directly to the dataset memory, which (in principle) is really just a matter of getting the address of that block. Your suggestion sounds promising. I didn't think of modifying the graph (etc) properties as a way of getting them to use a different dataset.
I'll report back when I've tried this...

... Looking at the graph/plot code in the xml file, I see that references to the dataset name ("dset") occur in the javascript. This code handles the sliders that move slices around, the very nice feature that I'm particularly interested in having. It looks as if the javascript would need to be modified "on the fly" in order to make the plots use a different dataset. This seems to be a bridge too far...

Gib

Gib,

Then the Java script can be easily change to read a stored variable in the xml file that contains the name 'dset'. You change the name to 'yourset' with your application then change it back.

Don't be too quick to give up. You are almost there.

Jim

Quoting - jimdempseyatthecove

Gib,

Then the Java script can be easily change to read a stored variable in the xml file that contains the name 'dset'. You change the name to 'yourset' with your application then change it back.

Don't be too quick to give up. You are almost there.

Jim

You make it sound so easy, Jim. I'm not saying you are wrong, but have you looked at the code in astrojet.h5? I'm not sure that it is as straightforward as you indicate.

I have tried some variations on what you suggest, handicapped of course by my primitive level of understanding of how it all works. The first observation is that the name of the dataset (currently "dset") crops up in several places. The dataset definition contains definitions for a number of sections, "slice1", "slice2", "slice3", "sec1", "sec2", "sec3" that are used to construct the graphs.

If I manually edit the file and change the dataset name to "dsetold", leaving all the other references to "dset" as they are, then avOpen() the file and avStartWatch() a new dataset called "dset", then view the result, I see two datasets as expected, "dsetold" and "dset". "dsetold" still has the various sections associated with it, "dset" has no sections associated. The graphs are all blank - it seems that they have not been linked to the new dataset "dset". If I click on a page with a slider a runtime error is generated. (I think this could be related to the fact that although "dsetold" is a 0-based array, the array "dset" that was created by avStartWatch() is 1-based. Without expecting it to have any effect, I changed my Fortran to make dset 0-based - it did nothing.) EDIT: Correction - this error is not related to the array basing, since it occurs when viewing the file before the avStartWatch() call. Clearly it is an effect of changing the name of the dataset, but it isn't fixed by adding back a dataset called "dset".

I'd be very grateful if you could give more explicit instructions as to how I can do this.

Thanks
Gib

I've realized that the path I'm on is taking me to the avLoadTemplate() routine, which in principle does exactly what I want. I have therefore made a template file with the dataset dset removed. I can load this then create a dataset called dset with avStartWatch(). So far so good. I found that by manually creating the required dset sections in AV (i.e. slice1, slice2, etc.) I can get some of the graph functionality. so set about creating these using library calls. My first attempt to use avCreateNewSection() is not successful. Here is the code (minus a couple of declarations):

character(*), parameter :: dsetpath = '/dset'
integer(int_ptr_kind()) :: object, item
integer :: N, viewerId, status, indx(3), lbnd(3), dim3d(3)
logical :: load_template = .true.

N = 50
allocate(dset(0:N-1,0:N-1,0:N-1))
dim3d = shape(dset)
lbnd = 0

if (load_template) then
call avLoadTemplate('template_nodset.xml','',status)
call avStartWatch(loc(dset),3,dim3d, AV_REAL4,"dset",status)
call avLBound(loc(dset),lbnd,status)
call make_test_array(N,dset)
call avNewViewer(viewerId)
call avVisible(viewerId, AV_TRUE, status)
object = avGetObject(dsetpath)
write(*,*) 'dset object: ',object
call avCreateNewSection(object, "slice1", item)
write(*,*) 'slice1 item: ',item
write(*,*) 'Hit enter to end ...'
read(*,*)
stop
endif

This doesn't indicate any errors, the value returned for object looks reasonable (17707532) but the value returned for item is 0, which doesn't look right, and AV does not show that a section has been created for "dset". Can someone tell me how to use avCreateNewSection() to create a section?

Gib

Gib,

For diagnostic purposes, insert conditional code to open AV and make it visible early on in your probram (i.e. showing an empty AV). Then as you step through the program you can follow the progress. I use FPP

    ! Create the display objects here
    hRoot = avGetObject("/")
!#define _TRACE_INIT_AVFRT
#ifdef _TRACE_INIT_AVFRT
    ! ============ for now show viewer for debugging
    call avNewViewer(viewerId)
    IavStatus = 0   ! some AVFRT do not set IavStatus to 0
    call avVisible(viewerId, 1, IavStatus)
    call avStatusCheck(IavStatus)
    ! ============ for now show viewer for debugging
#endif

Uncomment the !#define _TRACE_INIT_AVFRT for trace mode

Later on, where you would normaly make the viewer visible, reverse the conditionals

#ifndef _TRACE_INIT_AVFRT
    ! Create the viewer
    call avNewViewer(viewerId)
    ! display the graph in the viewer
    call avSetViewerPath(viewerId, "graph:/graphXYZwithCharts", IOstatus)
    ! Make it visible
    call avVisible(viewerId, 1, IavStatus)
    call avStatusCheck(IavStatus)
    call avSetViewerWindowText(viewerId, ControlPanelCommentLine1, IavStatus)
    call avStatusCheck(IavStatus)
    call avSetViewerStatusBarText(viewerId, ControlPanelCommentLine2, IavStatus)
    call avStatusCheck(IavStatus)
#endif

As you step through your AV initialization you can click on the AV window and browse through the tree pane and use Right-Click Properties to see just what happened (after each avFunctionNameHere).

I noticed in your code sample that you did not set the avSetViewerPath.

If you can also open up a seperate AV to display the original test.xml file you can check side by side for differences as you run through your code. The code is a bit touchy to debug.

Jim

Thanks Jim, this debug stepping and viewing might be useful.

I have found that I can make things almost work by adding the required sections manually in AV after my program fires it up, so it is natural to try adding sections using library calls. It seems to me a rather fundamental problem that I can't get this to work. The documentation is a bit ambiguous, in that it doesn't say what 'object' points to:

Fortran syntax
subroutine avCreateNewSection(object, name, item)
integer(int_ptr_kind()), intent(IN) :: object
character(*), intent(IN) :: name
integer(int_Ptr_kind()), intent(OUT) :: item
end subroutine avCreateNewSection

I assumed that it would point to the dataset, i.e. object = avGetObject('/dset'), but that does not work - avCreateNewSection(object,"slice1",item) returns item = 0.

Can someone tell me how to add a section using the Fortran library routines?

avGetObject('/dset')

Would imply dset is in the root. It also implies /dset exists. Have you created dset at this point?

I haven't done what you are doing but I do muck around with graphs.

For avCreatGraph3DObj I supply as 1st argument avGraphs(hRoot), where hRoot is the result from avGetObject("/").

I am guessing that you will need something like avDatasets(hRoot) to get a handle to the data sets

If I were to venture a guess

avCreateNewSection(avGraphs(avGetObject("/")), "dset/slice1", item)
or
avCreateNewSection(
(avGetObject("data:/dset")), "slice1", item)

Where "data:" might be different (?dataset:?)

It is a sad fact that the documentation lags behind this marvolus work. My guess the documentation was one of these auto-generated pieces of work. The doc is full of properties and methods but is void of context. The lack of context is a difficult barrier to overcome when trying to learn how to program with the API.

Jim Dempsey

Quoting - jimdempseyatthecove

avGetObject('/dset')

Would imply dset is in the root. It also implies /dset exists. Have you created dset at this point?

I haven't done what you are doing but I do muck around with graphs.

For avCreatGraph3DObj I supply as 1st argument avGraphs(hRoot), where hRoot is the result from avGetObject("/").

I am guessing that you will need something like avDatasets(hRoot) to get a handle to the data sets

If I were to venture a guess

avCreateNewSection(avGraphs(avGetObject("/")), "dset/slice1", item)
or
avCreateNewSection(
(avGetObject("data:/dset")), "slice1", item)

Where "data:" might be different (?dataset:?)

It is a sad fact that the documentation lags behind this marvolus work. My guess the documentation was one of these auto-generated pieces of work. The doc is full of properties and methods but is void of context. The lack of context is a difficult barrier to overcome when trying to learn how to program with the API.

Jim Dempsey

Hi Jim, I appreciate your efforts. Yes, I have created dset (with avStartWatch) when trying to add a section. I can see dset in AV, and can add sections manually there.

hRoot = avGetObject("/") returns 16788532 (good)
hData = avGetObject("/dset") returns 16790060 (good)
avGetObject("data:.dset") and avGetObject("dataset:/dset") both return 0 (bad)

It looks as if hData must be the right object handle for dset. But
call avCreateNewSection(hData,"slice1",item)
gives item = 0, as does
call avCreateNewSection(hRoot,"dset/slice1",item)
and in both cases AV shows that nothing was created.

I'm starting to wonder if it really is possible to do this with Fortran. Maybe this is one of the things that was not completed. It is a great shame about the documentation, but not surprising - who likes to write the docs? I was hoping that John Readey might show up. Perhaps it wasn't diplomatic, or even considerate, to address my post to him personally. :-(

Gib

Maybe you can write the "create" function in C++ or other language.

Note, on the Fortran side, I think you canuse the debugger tostep into avGetObject("/") to get the internal static object variable name (Fortran creates a single AV object), then roll your own function to return a void* to the internal Fortran AV object. Such as avGetObjectPointerForGib("/") . This void pointer can be use into your call out to your C++ (or other) code to perform the missing IVF / AV functionality.

The first time you step into the AV code you may have to browse for the sources (assuming you unpacked the sources to the Fortran interface files).

Jim Dempsey

Quoting - jimdempseyatthecove

Maybe you can write the "create" function in C++ or other language.

Note, on the Fortran side, I think you canuse the debugger tostep into avGetObject("/") to get the internal static object variable name (Fortran creates a single AV object), then roll your own function to return a void* to the internal Fortran AV object. Such as avGetObjectPointerForGib("/") . This void pointer can be use into your call out to your C++ (or other) code to perform the missing IVF / AV functionality.

The first time you step into the AV code you may have to browse for the sources (assuming you unpacked the sources to the Fortran interface files).

Jim Dempsey

This is fun (if you are a bit twisted and have lots of time - time is the constraint for me). Thanks for reminding me that there is source code. Looking in AvObjMod.f90 I see that the first argument to avCreateNewSection() is actually a handle to the Sections collection, which is obtained by avSections(hData). I added
hSections = avSections(hData)
and put a checkpoint on this statement. To my surprise, when I debug and Step Into the code at this point, I find myself at a line of code in the middle of the function AVSETLBOUND() in AvObjMod.cpp. This makes no sense to me, and doesn't throw any light on the failure of avSections() to return a handle (it returns 0).

Your idea of trying to do this with a C++ function sounds sensible, but I'm not sure if I'm up to the task, I mean within a reasonable amount of time.

I stubbornly persisted a bit longer, and discovered that the debugger was putting me in the wrong function (only about 8 lines from the expected place) because the library was out of synch with the source file AvModObj.cpp, in fact the source had been last saved about 5 min after the library was created, in 2006. Who knows what that signifies, but when I rebuilt the library (after minor surgery on the make file) the debugger started behaving in a more understandable way. Poking around the code I realized that the sequence of handle fetches required to make a new section and give it some properties is this:

hData = avGetObject("/root/dset")
hDataspace = avDataspace(hData)
hSections = avSections(hDataspace)
call avCreateNewSection(hSections, sectionname, hSection)
call avSetStart(hSection, dim, start, status)
! etc.

It works! I can update the data by writing directly to memory, as required. I'm not totally in the clear yet, because there are some funny things happening with the colours that I don't understand, but I'm very happy that it runs without generating errors. I must be almost out of the woods.

And today's my birthday - this is my best present :-)

Colours fixed, all working now. Whew!

Gib

Happy Birthday!

I had to fix the make file for myself too. The make files didn't work out of the box on Windows.

In the programming I do, I create a very large number of elements (~2000)

In the file AVLIST.H, at the bottom you will find

MAX_AVLIST_ELEMENTS=100

This number is way too small for my purposes. I use

MAX_AVLIST_ELEMENTS=5000

This is a static array of pointers. 5000 may be overboard but this is only 20KB or 40KB of space.

Jim Dempsey

Leave a Comment

Please sign in to add a comment. Not a member? Join today