rounding inconsistency

rounding inconsistency

Will someone confirm below is a problem with CVF6.6B ?

program cvf_rounding_inconsistency
implicit none
real,parameter :: b = 0.002
integer :: a, n

do a = 15320499,15320499
n = a * b
write (*,*) n ! = 30640 truncates
end do

a = 15320499
n = a * b
write (*,*) n ! = 30641 rounds
end program

This problem surfaced when testing a CLF poster's
rounding case complaint for his unnamed compiler.

If I compile with /nooptimize the output truncates
BOTH outputs above to 30640 ??

6 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

There's no inconsistency in how CVF rounds, irrespective of optimization.

For floats, the default rounding mode is round to nearest. Rounding mode doesn't apply to integers. Use NINT. NINT(a*b) will give 30641 all the time.

Gerry T.

Actually, "round to even" is used. See The Perils of Real Numbers, Part 3 for more info.


Retired 12/31/2016

Yes, I meant to say that my default is to round to nearest so YMMV.

Gerry T.

Actually, 'round to even' is a loose way of saying take the result whose LSB is 'off' in the event of a tie in 'round to nearest'. From the Feb/2003 IEEE 754 standard:

4.1. Round to Nearest
An implementation of this standard shall provide round to nearest as the default rounding mode. In this mode the representable value nearest to the infinitely precise result shall be delivered; if the two nearest representable values are equally near, the one with its least significant bit zero shall be

The CVF docs correctly use the term 'round to nearest' as does F2K.

Gerry T.

n = a * b ! = 36040
n = a * b ! = 36041

obviously in the 2nd evaluation above
CVF didnt follow the same rule it followed in the 1st
Hence my subject line: rounding inconsistency

Leave a Comment

Please sign in to add a comment. Not a member? Join today