Workaround code

Workaround code

eos pengwern's picture

I found Array Visualizer to be the biggest productivity booster of any software package I've used, but now I'm usingParallel StudioXE on 64-bit Windows 7 I have lost all hope of ever being able to use it again; an inexplicable omission on Intel's part.

As a workaround, I have written myself a couple of Fortran routines that will save the contents of a rank-two array to a file, either as a .csv file (for seeing the values in text form) or as a .bmp file (for a graphical representation with high values being shown in red and low values in blue). All one has todo is to call SaveAsCSV or SaveAsBitmap with an array and a filename, and the files will bewritten to the current working directory.

I have found these so useful that, as a service to the community, I am publishing them here: with permission for them to be used and modified freely.Caveat utilitor.

    subroutine SaveAsCSV(array, filename)
    
        real(kind(1d0)), intent(in), dimension(:,:) :: array
        character(len=*), intent(in) :: filename             ! Must end in '.csv'
    
        character(len=24) :: fmtstr
        integer :: j, n, m
        
        n = size(array,1)
        m = size(array,2)
        
        ! Prepare the format       
        write(fmtstr, '(''('',i4,''(E13.6,'''',''''), E13.6)'')')  n-1 
                              ! Increase the 'i4' if more than 9999 elements in a row
        
        ! Open the file:                 
        open(unit=12, file=filename, status='REPLACE')
        
        ! Write the data
        do j=1,m
            write (12, fmtstr) array(:,j)
        end do
        
        ! Close the file
        close(12)    
        
    end subroutine SaveAsCSV  
    
  ! --------------------------------------------------------------------------------------------------------------------

    subroutine SaveAsBitmap(array, filename)
           
        real(kind(1d0)), intent(in), dimension(:,:) :: array
        character(len=*), intent(in) :: filename             ! Must end in '.bmp'
        
        integer :: width, height, filesize, row_length, padded_length, i, j, k
        character, allocatable, dimension(:,:) :: BGR_data
        character, allocatable, dimension(:) :: file_data
        character(len=12) :: fmtstr

        ! Convert the incoming data into BGR format
        width = size(array, 1)
        height = size(array, 2)
        row_length = width * 3
        padded_length = 4 * (row_length/4)
        if (padded_length.lt.row_length) then
            padded_length = padded_length + 4         ! The width, rounded up to the nearest multiple of four
        end if        
        allocate (BGR_data(padded_length, height))
        filesize = 54 + padded_length * height        ! Total file size = header + data
        allocate (file_data(filesize))
        
        call Create2DMap(array, BGR_data)
        
        ! Header 1 (file header; bytes 1..14)
        file_data(1) = 'B'                          ! Declare that this is a BMP file
        file_data(2) = 'M'
        file_data(3:6) = four_byte_integer(filesize)
        file_data(7:10) = char(0)                      ! Reserved bytes
        file_data(11:14) = four_byte_integer(54)       ! The total header length

        ! Header 2 (bitmap header; bytes 13..54)
        file_data(15:18) = four_byte_integer(40)       ! Length of the bitmap header
        file_data(19:22) = four_byte_integer(width)
        file_data(23:26) = four_byte_integer(height)
        file_data(27:28) = two_byte_integer(1)         ! Number of planes
        file_data(29:30) = two_byte_integer(24)        ! Colour depth in bits
        file_data(31:54) = char(0)                     ! Fields which aren't applicable in this case   
        
        ! We add the image data line-by-line, in upside-down order:
        k = 55
        do j=height,1,-1
            do i=1,padded_length
                file_data(k) = BGR_data(i,j)
                k = k+1
            end do
        end do

        ! Open the file:                 
        open(unit=12, file=filename, status='REPLACE')

        ! Write formatted data
        write(fmtstr,'(''('',i8.8,''a)'')') filesize
        write(12, fmtstr) file_data
        
        ! Close the file
        close(12)
        
    contains
   
        function four_byte_integer(i)

            integer, intent(in) :: i
            character, dimension(4) :: four_byte_integer
            
            integer :: itmp1, itmp2
            
            itmp1 = i
            itmp2 = itmp1 / 256**3
            four_byte_integer(4) = char(itmp2)
            itmp1 = -itmp2 * 256**3 + itmp1
            itmp2 =  itmp1 / 256**2
            four_byte_integer(3) = char(itmp2)
            itmp1 = -itmp2 * 256**2 + itmp1
            itmp2 =  itmp1 / 256
            four_byte_integer(2) = char(itmp2)
            itmp1 =-itmp2 * 256    + itmp1
            four_byte_integer(1) = char(itmp1)
            
        end function four_byte_integer

        function two_byte_integer(i)

            integer, intent(in) :: i
            character, dimension(2) :: two_byte_integer
            
            integer itmp1, itmp2
            
            itmp1 = i
            itmp2 = itmp1 / 256
            two_byte_integer(2) = char(itmp2)
            itmp1 = -itmp2 * 256 + itmp1
            two_byte_integer(1) = char(itmp1)
            
        end function two_byte_integer

    end subroutine SaveAsBitmap
    
  ! -------------------------------------------------------------------------------------------------------------------
  
    subroutine Create2DMap(values, BGR_image)
    
        ! Create a 2D map from a set of double-precision values, with high values represented as red
        ! and low values represented as blue; the output data is presented in row-padded BGR format, 
        ! suitable for output via a .BMP file. 
        
        real(kind(1d0)), intent(in), dimension(:,:) :: values
        character, intent(inout), dimension(:,:) :: BGR_image     
        
        integer :: i, j, offset, rowlength
        real(kind(1d0)) :: minvalue, maxvalue, denom, notional_wavelength, red, green, blue


        minvalue = minval(values)
        maxvalue = maxval(values)
        denom = 1d0/(maxvalue - minvalue)
        rowlength = size(BGR_image,1)
        
        ! Now construct the image
        do j=1,size(values,2)
            do i=1,size(values,1)
            
                notional_wavelength = 380d0 + (values(i,j)-minvalue) * (640d0 - 380d0) * denom
                call RGB_from_Wavelength(notional_wavelength, red, green, blue)
            
                ! Populate the BGR array:
                offset = (i-1)*3
                BGR_image(1 + offset, j) = char(int(255 * blue,  4))
                BGR_image(2 + offset, j) = char(int(255 * green, 4))
                BGR_image(3 + offset, j) = char(int(255 * red,   4))
        
            end do
            if (3+offset.lt.rowlength) then
                BGR_image(4+offset:rowlength,j) = char(0)
            end if
        end do
    
    end subroutine Create2DMap
    
  ! --------------------------------------------------------------------------------------------------------------------
      
    subroutine RGB_from_Wavelength(wavelength, red, green, blue)
    
        ! Provides the appropriate ratio of red, green and blue to represent the colour of light with a given 
        ! wavelength; based on the 'Mandelbrot' example provided in the main Qt documentation.
        
        real(kind(1d0)), intent(in) :: wavelength
        real(kind(1d0)), intent(out) :: red, green, blue
        
        real(kind(1d0)) :: s
        
        red=0d0
        green=0d0
        blue=0d0
        
        if ((wavelength.ge.380d0).and.(wavelength.lt.440d0)) then
            blue = 1d0
            red = (440d0 - wavelength) / (440d0 - 380d0)
        elseif ((wavelength.ge.400d0).and.(wavelength.lt.490d0)) then
            blue = 1d0
            green = (wavelength - 440d0) / (490d0 - 440d0);
        elseif ((wavelength.ge.490d0).and.(wavelength.lt.510d0)) then
            green = 1d0
            blue = (510d0 - wavelength) / (510d0 - 490d0)
        elseif ((wavelength.ge.510d0).and.(wavelength.lt.580d0)) then
            green = 1d0
            red = (wavelength - 510d0) / (580d0 - 510d0);
        elseif ((wavelength.ge.580d0).and.(wavelength.lt.645d0)) then
            red = 1d0
            green = (645d0 - wavelength) / (645d0 - 580d0)
        elseif ((wavelength.ge.645d0).and.(wavelength.lt.780d0)) then
            red = 1d0
        end if

        s = 1d0
        if (wavelength.gt.700d0) then
            s = 0.3d0 + 0.7d0 * (780d0 - wavelength) / (780d0 - 700d0)
        elseif (wavelength.lt.420d0) then
            s = 0.3d0 + 0.7d0 * (wavelength - 380d0) / (420d0 - 380d0)
        endif
        
        red = exp(0.8d0 * log(red * s)) 
        green = exp(0.8d0 * log(green * s))
        blue = exp(0.8d0 * log(blue * s))
        
    end subroutine RGB_from_Wavelength   
9 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.
dwodwyer@tcd.ie's picture

Many thanks! I've been using this Fortran compiler, or its forerunners, for nearly twenty years and the development of the array visualiser was a godsend. Particularly as a debugging tool for anyone working on 2D arrays. Given the use of Fortran for finite element programs it is hard to understand how Intel could just let this feature go. Unfortunately it suggests a lack of commitment.

I've just bought a new copy of the latest compiler and the full ISML libraries but before I upgrade again I will check that the program hasn't been scaled back before I part with the cash.

Dwod

Fons Brosens's picture

In our research group "Theory of Quantum and Complex Systems" I volunteered to test the option for replacing CVF by IVF for all members of the group. My advice was clearly NOT to invest in that replacement because of the lack of an Array Visualizer. I also distributed this (negative) advice among the scientists and research groups with whom we have a collaboration. An Array Visualizer comparable in quality with the one in CVF is absolutely required. Fortunately, I am grateful that this workaround code overcomes the difficulty partially, but I still do not understand why Intel did not anticipate on such an elementary need.

Sergey Kostrov's picture
Quoting Fons Brosens ...I also distributed this (negative) advice among the scientists and research groups with whom we have a collaboration...

Let me be absolutely honest. Idon't likea constant flow of differentproblems and bugsin software
developed during last10 years. But, since I really love my jobI'm ready toforgive it. I'm ready to work
hard in order to find a solution when a problem "hits"a project, instead of spreading a negative feedback for
the company, like Intel or Microsoft, onthe web, or among softwaredevelopers and scientists.

I consider the statement:

"...distributed this ( negative ) advice among the scientists and research groups..."

as a very unprofessional and disrespectful.

Best regards,
Sergey

jimdempseyatthecove's picture

Sergey,

>>as a very unprofessional and disrespectful.

This cuts both ways.

Removing a valuable feature (to some) causes very large problems. These users are hurt, not just emotionally, but in a manner it which it costs them dearly. Many users used AV as a debugging tool. This may have saved them substantial time in debugging applications, and in some cases, may have been the only way to uncover the bug.

For me, and I assume many others, I did not use AV for debugging, rather it was used for monitoring simulation progress and producing simulation results suitable for presentation. This includes 3D simulation playback (with minor modification to code to periodicallylog arrays to be viewed).

Trying to do this using OpenGL is problematic (I discussed this in earlier post some time ago). Trying to duplicate most of the features places the user in the position of expending a man-year of coding (could be much more) or not upgrading compiler version. Upgrade is almost a requirement as new features are added (e.g. AVX, coarrays, ...). For these users, and upgrade becomes a $50,000, $100,000, $200,000... decision, or the lack of productivity in running the code without the older features.

These users are experiencing real damage.

Jim Dempsey

www.quickthreadprogramming.com
jimdempseyatthecove's picture

I wish to add something constructive for the current users of AV (graphs and charts integrated into application).

In the simulations I run, the program periodically logs the state of the simulation to a log file. This file can be used to backtrack errors (simulation crashes) and to restart simulation at a specific point. This is not unusual to have in long running programs.

What I added as a feature was a companion program that could be run concurrent with the simulation that would restore the state sequentially through the log file. I.g. advancement in coarse steps without computation of simulation. The AV code would then display the state changes.

Now then, consider using an older PC in the lab, running an older O/S observing the log file over the network. The monitoring/display presentation program could be compiled andmaintained using the version of the compiler that supported AV.

While this isn't a perfect solution, it may be a workable solution, and this would reduce the programming cost of any alternative by a factor of 1/10 or so.

Jim Dempsey

www.quickthreadprogramming.com
Fons Brosens's picture

Apparently Intel is totally disregarding the needs of the users for a good AV. Note that their older version which they "recommend" but do not support, can absolutely not compete with the AV of CVF. Fortunately we have the work around code of the initiator of this forum. And now it is a great relief that Jim Dempsey proposes this fantastic idea, which I now use systematically. Fantastic! Many thanks! But I maintain that it is not normal how Intel takes a step back and disables earlier accomplisments in CVF. Therefore we will live with CVF as long as possible, and limit the purchases of Intel Fortran to the strict minimum. Thanks Jim for your understanding and your constructive idea. Fons Brosens

jimdempseyatthecove's picture

Fons, and others.

I am using Array Visualizer (API during runtime, not AV watch while debugging) on Windows 7 x64, Intel Parallel Studio 2011 w/ IVF 2011.9.300. There were a few issues in getting it to work. First off the installation directory and paths in VS were not coordinated (issues mainly between what was available on VS2003/2005 when AV shipped and use on XP, and VS2010). A second issue was I had to edit and recompile the IVF AV interface code (to the AV library). The edits were required to increase the number of objects that could be handled. The AV library code does not ship as part of the download and I am unable to fix some remaining problems. I am surprised (happy) it still works.

Jim Dempsey

www.quickthreadprogramming.com
NsK's picture

Dear Jim,

If you could describe in details the different steps one has to follow in order to achieve this, that would be just awesome.
Thank you so much in advance.

Nick

Nick

Login to leave a comment.