Doctor Fortran in "It Takes All KINDs"

So this is retirement? As I noted earlier, I may no longer be working for Intel, but I do intend to stay active in the Fortran and Intel development communities. While I am back in the forum answering questions, it is liberating knowing that I am not responsible for making sure every question (and bug report) gets answered. I recently learned a wonderfully appropriate Polish saying "Nie mój cyrk, nie moje małpy", which translates literally to "Not my circus, not my monkey", or more colloquially, "not my problem". I plan to apply this a lot.

As has often been the case in Doctor Fortran posts, I get the ideas for these by seeing what sorts of problems and questions often arise about Fortran. Recently there was an extended thread in the comp.lang.fortran newsgroup about Fortran KIND values, started by someone learning Fortran who had a basic misunderstanding of how they worked. Most of the advice in the thread was good, but there were some folk who, while very knowledgeable and whose expertise I admire, offered a suggestion I thought flat-out wrong. Thus begat this post. But first, some history is needed.

In the beginning, there was FORTRAN - FORmula TRANslator for the IBM 704. The original FORTRAN, as well as its successor FORTRAN II, was almost entirely lacking in the ability to specify what kind of numbers you were calculating with. The 704 had "fixed point" registers, which we know as integers, and "floating point" registers (reals). Just one size of each. The early FORTRAN language didn't even have a way to explicitly declare whether a variable (or in the case of FORTRAN II, a function) was  real or integer - that was indicated by the choice of first letter of the variable's name. Yes, this is where the implicit typing rule for "letters I through N being integer, everything else being real" came from. (FORTRAN II also applied a meaning to the last letter of a name being F (made it a function) or the first letter being X (made the function be integer).)

Next came FORTRAN IV, also known as FORTRAN 66. F66 introduced the notion of explicit typing through the REAL and INTEGER declaration statements. (F66 also introduced COMPLEX and LOGICAL, concepts that had not been in the earlier languages.) While there was still only one kind of integer (and logical or complex), there were two kinds of reals, reflecting newer computer architecture. The larger one was called DOUBLE PRECISION. In addition to being able to declare variables as either REAL or DOUBLE PRECISION, you could indicate which kind a real constant was by the use of an E exponent letter for REAL or a D exponent letter for DOUBLE PRECISION - for example, 3.1415926535897D0.

What the language didn't specify, and really still doesn't, was the precision, range and in-memory format of integer and real values. Different computer architectures had different designs for these. Register sizes could be 8, 12, 16, 30, 32, 36, 48, 60 or 64 bits, some CDC machines had ones-complement integers (allowing minus zero for an integer!), the PDP-11 line had several different floating point formats depending on the series, the IBM 360/370 had "hex normalization" for its floating point, etc. As many computers offered a choice of integer and real sizes, programmers naturally wanted to be able to specify these in their programs. Thus came about the common extension of adding *n to the INTEGER or REAL keyword where n was the size in bytes of the value, for example, REAL*8 or INTEGER*2. So popular were these extensions that many programmers thought (and even today many think) that this syntax is standard Fortran; it isn't!

Next we come to FORTRAN 77, but nothing really changed here in the way that variables were typed. F77 did add CHARACTER (with the *n syntax), but that's it for typing.  This was the era of the shift of the mainstream computer architectures to 32-bit (from smaller or larger sizes). Since FORTRAN (still in upper case) still assumed only one, unspecified size of integer (and two unspecified sizes of real), and the default sizes changed as architectures did, the use of syntax such as INTEGER*2, as well as compiler options such as -i8) became even more entrenched, making code portability problematic. 

Fortran 90 (mixed-case!) finally addressed the problem of different kinds of integer and real (also logical, complex and character, but I'm not going to spend much time on those.) Recognizing that the widely-used *n syntax served only to specify size in bytes and not any other properties, the language adopted the notion of "kind numbers" that identified a particular variation of a type but did not specify it directly. Kind numbers were to be positive integers, but other than that an implementation was free to choose whatever set of kind numbers it wanted. The syntax for using these in declarations was the "kind type parameter" specified by the optional keyword KIND, for example: INTEGER(KIND=3) or REAL(17). For literals, the kind type followed an underscore, and the kind could be an integer literal or a named (PARAMETER) constant, for example: 486_4 or 3.1414926535897_DP.

Now that Fortran admitted that there could be multiple kinds for a type, it introduced the concept of "default" kinds (default integer, default real, etc.). The default kinds were implementation dependent, though up through Fortran 2008 a compiler was required to support only one integer kind and two real kinds. (That's still true in Fortran 2015, but there's an added requirement that at least one integer kind support 18 decimal digits.) If you write a constant literal without a kind specifier, you got the default kind. Since FORTRAN 77 didin't have this concept, many compilers gave you "free" extra precision or range if you typed a constant with more digits than the default. For example, one might see:

DOUBLE PRECISION PI = 3.1415926535897

and the compiler would evaluate the constant as if it were double precision. Fortran 90 put a stop to that, though - the number here is default real (typically "single precision") and you might end up with a much less accurate value. If the programmer had added D0 at the end, that would have been ok, but the Fortran 90 way is to use kind specifiers.

While most vendors decided to map the old *n syntax to kind numbers, so that real kinds of 4 and 8 were common, not all did. One popular implementation chose sequential kind numbers starting at 1 (1,2,3, etc.). Obviously, using explicit kind numbers this way was still not portable. The language provided two intrinsic functions, SELECTED_INT_KIND and SELECTED_REAL_KIND, to allow you to, finally, specify what characteristics you wanted from your numeric type. Instead of saying "I want a 32-bit integer", you would instead ask for an integer kind that can represent values of n significant decimal digits, or a real with a particular minimum precision or range. The recommended way of using these is to define, perhaps in a module, named constants for the various kinds you want to use, and then use these constants in declarations and constants. For example:

integer, parameter :: k_small_int = SELECTED_INT_KIND(5) ! 5 decimal digits
integer, parameter :: k_big_int = SELECTED_INT_KIND(12)
integer, parameter :: k_extra_real = SELECTED_REAL_KIND (10,200) ! 10 digits, 10**200 range
integer(k_small_int) :: tiny_int
real(k_extra_real) :: lotsa_digits = 2.780486193E98_k_extra_real

Doing it this way makes you completely independent of the compiler's scheme for assigning kind numbers and makes sure that you get the precision and range your application needs, in a portable fashion. Notice there's no mention of bits or bytes here, but there is also no mention of the underlying capabilities of the type. What if you specifically wanted an IEEE floating type? There are architectures where both IEEE and non-IEEE floating types are available, such as the HP (formerly Compaq formerly DEC) Alpha. In this case you can use IEEE_SELECTED_REAL_KIND from intrinsic module IEEE_ARITHMETIC to get an IEEE floating kind. And what if there is no supported kind that meets the requirements? In that case the intrinsics return a negative number which will (usually, depending on context) trigger a compile-time error.

If you want to ask what the kind of some entity is, the KIND intrinsic returns it. So if you want the kind of double precision, say KIND(0.0D0).

There is one interesting aspect of the new kind numbers that trips up some programmers, and that relates to the COMPLEX type. If you wanted double precision complex (note that DOUBLE COMPLEX was never standard), you might write COMPLEX*16 to indicate a COMPLEX that occupies 16 bytes. But in a Fortran 90 (or later) compiler where, just for illustration's sake, the kind of double precision was 8, you would use COMPLEX(KIND=8) instead. (Or really, some PARAMETER constant with the proper double precision kind value.)

A brief mention of the C interoperability features: intrinsic module ISO_C_BINDING declares constants for Fortran types that are interoperable with C types, for example C_FLOAT and C_INT. Use these if you're declaring variables and interfaces interoperable with C.

And now we get to the point of contention I mentioned at the start of this post...  Fortran 2008 extended intrinsic module ISO_FORTRAN_ENV to include named constants INT8, INT16, INT32, INT64, REAL32, REAL64 and REAL128 whose values correspond to the kinds of integer and real kinds that occupy the stated number of bits. More than one response in that newsgroup thread recommended using these instead of hard-coding integer and real kinds. In my view, this is little better than the old *n extension in that it tells you that a type fits in that many bits, but nothing else about it. As an example, there's one compiler where REAL128 is stored in 128 bits but is really the 80-bit "extended precision" real used in the old x86 floating point stack registers. If you use these you might think you're using a portable feature, but really you're not and may get bitten when the kind you get doesn't have the capabilities you need. My advice is to not use those constants unless you truly understand what they do and do not provide. Most applications should use the SELECTED_xxx_KIND intrinsics as I described above. If you want to call your kind DP to indicate "double precision", that's fine - just use SELECTED_REAL_KIND to get it.

If you have questions about the specific issues I discuss here, feel free to ask in the comments but I might not see the question for a while. If you have general questions or want to report a problem with the Intel compiler, please do that in the user forums.

For more complete information about compiler optimizations, see our Optimization Notice.


Steve Lionel (Ret.)'s picture

Raul, I was not on the committee when the ISO_FORTRAN_ENV constants for numeric kinds were added. I gather that the thinking at the time was that these were "easier to use" than SELECTED_REAL_KIND and that the syntax, at least, would be portable and were no worse than the nonstandard REAL*4 popular extension.

But as I explained earlier, I think this was misguided and not in keeping with the language's philosophy of being as platform-neutral as possible. My advice is to not use these constants.


Raul L.'s picture

Thanks, Steve, for explaining the real128 issue with GFortran (which I once found out the hard way) and how it's not strictly violating the standard. I always assumed that was a bug that will soon be fixed.

That being said, what was the reason for introducing the ISO_FORTRAN_ENV constants if there is still confusion over their meaning? How are they superior to the standard real(kind=...) notation? They define the storage size but that's not the main thing we usually care about.

FortranFan's picture


Re: "the point Steve is making: merely the storage size is not enough to characterise floating-point numbers", since this is in the context of named constants in ISO_FORTRAN_ENV intrinsic module which were introduced in the Fortran standard as recently as 2008, in your mind what can prevent the standard from being amended for these constants to mean more than just the storage size, to include their properties too?

Say a revision to the standard states REAL64 shall have "precision=15, range=307, and radix=2" i.e., the same as what Intel Fortran and gfortran have for this constant anyway (and I would think it's the same with other processors on other architectures), what problems, if any, will this change cause?

How will any of the other architectures such as Convex and IBM mini or OpenVMS be impacted?  Chances are these environments will never support a Fortran standard after 95, but even if they do, they can either choose to support REAL64 by some other means or elect not to, right?

The standard states, "If, for any of these constants, the processor supports more than one kind of that size, it is processor dependent which kind value is provided.  If the processor supports no kind of a particular size, that constant shall be equal to -2 if the processor supports kinds of a larger size and -1 otherwise."

Why can't the standard say "size and properties" instead of the current "size" and these architectures can issue -1 for the unsupported constants?

Appreciate your input,

Arjen Markus's picture

I remember using a Convex computer quite a few years ago that had two types of 32-bits reals, the difference being a different offset in the exponent, so that the same bit pattern would represent a number 4 times as large if you used type 1 than if you used type 2. One of these types followed the IEEE rules and the other, supposedly faster, type was native to Convex.

A few years before that our IBM mini computer was still around and that used 32-bits reals too, but the distribution of these 32 bits over sign+exponent and mantissa was different from the IEEE format.

Since IEEE is almost universal these days, such observations may belong to grandpa's collection of boring stories, but still it illustrates the point Steve is making: merely the storage size is not enough to characterise floating-point numbers.




FortranFan's picture


Ultimately my issue is with your statements quoted below, more importantly with what is missing in connection with them:

  1. " If you use these you might think you're using a portable feature, but really you're not and may get bitten when the kind you get doesn't have the capabilities you need."
  2.  "People who have difficulty with this probably also have very little understanding of how computational floating point works, and this lack of knowledge will just get them into trouble down the road."

Can you point to any complete example or a blog or a paper that illustrates how someone can actually "get bitten" or get themselves into "trouble down the road"?

I can sit and speculate but I can't come up with any realistic 'minimal working example' (I'm a big proponent on the MWE approach) that I can show to anyone and have them understand.  The problem is also compounded by the fact I've access to many different computers and workstations but they all now have Intel x86_64 (AMD64) architecture.

I can tell you in the present day compute platforms used in the industry with the kind of scientific and engineering computing needs being tackled, it makes no 'practical' difference whether someone codes with REAL*8, REAL(8), REAL(REAL64), REAL(C_DOUBLE), REAL(WP) where WP is some suitable SELECTED_REAL_KIND(p=N, r=M), as shown by my example below.

You have mentioned OpenVMS and VAX a few times, but I don't have access to them nor do I understand what might happen on them with the kind of simulations we run, perhaps someone can show some computations beyond the determination of PI.  I know FPU (or GPU) has been brought up, but again it's unclear what will be impacted and how.

You also bring up the aspect of "little understanding of how computational floating point works" but that's precisely the point, the levels of abstraction that have advanced since the days of assembly-level programming mean computational scientists can move beyond such arithmetic and focus on their own domain needs.

Then you mention "quadruple precision" computing whose needs we are yet to notice in our industry.  64 bits and "double precision" more than meet 99% if not 100% of our needs.  I'm asking around and no one is able to indicate any application where full 128-bit representation can genuinely help; there were some problems involving solution of PDEs where a little more than 64 bits, say extended 80 bit, seemed to help but even that was inconclusive.  But then note in such unique and rare instances, portability is the least of the concern.

My point is it's entirely insufficient to keep saying any more to someone trying to use Fortran for a technical/scientific/engineering problem that you "may get bitten" or you will yourself "in trouble down the road".  One has to explain with minimal working examples how they can "get bitten" or when is that "trouble down the road".  Maybe at some point soon you will have a Dr Fortran blogpost that gets into these matters too.


Steve Lionel (Ret.)'s picture


Your example is, more or less, portable between two specific implementations and architectures. While I can't try these myself, I know of current architectures where you'd get different results. For example, on OpenVMS Alpha with the option to use VAX D_float as double, your two "arbitrary" kinds based on SELECTED_REAL_KIND would end up with quad precision (X_float in Alpha terminiology), because VAX D_float double can't represent the range you asked for.


Yes, there are situations where the constants such as REAL64 are useful, such as what you describe where all you care about is the storage size. But is this realistic? Do you not care about the precision or range of values stored? Or even what the binary representation is?

I don't agree that kinds are hard to understand. What may be difficult is trying to force kinds into the simplistic size-based selection programmers use in most other languages. People who have difficulty with this probably also have very little understanding of how computational floating point works, and this lack of knowledge will just get them into trouble down the road. 

Clive P.'s picture


Thanks for posting such a long, informative, and well argued posting on the issue of KINDs.  But I can't help agreeing with FortranFan that if it takes such a lot of explanation then the KIND system is poorly designed (well he said rotten, I'd almost agree with that).

But there is one situation where it seems to me to be useful to have the ISO_FORTRAN_ENV constants like REAL64, when one is reading or writing a file and you want to specify how much space on the file each data item will take.  The SELECTED_REAL_KIND mechanism doesn't do that at all well.  I realize that it not guaranteed by the Fortran Standard that a REAL64 variable will actually appear in an output file occupying 64 bits, but in practice it is overwhelmingly likely that it will do so.  There are some applications where it is more important to be able to read and write files in a particular format than to get the right precision when doing calculations.

By the way, I'm new to these Intel Forums - it's ironic that the built-in spell-checker doesn't recognize Fortran as a valid word!   I had the same problem with "recognise" but it's ok if I switch to the American spelling - is there perhaps some American spelling of Fortran that works?  :-)


Arjen Markus's picture

This is not so much a comment on the text, but on the forum's features - I have not read the text yet:

I am old-school (*) when it comes to reading long texts from a computer screen: I run to the printer. And that is what I am missing here ... all the social media icons are here, but not the "printer" icon. Oh well, printing from my browser does not always work either, even when a page offers that facility. Quite often the printer has a narrower size than the browser thinks. Perhaps my pedestrian method of handling the impedance problem is not that bad :) ...

Okay, I have printed the above text and will read it as soon as convenient (see below).



(*) Yes, I have reached the age of grumpiness :). Another reason for wanting text like this on paper is that it is an easier medium when commuting.


FortranFan's picture


Consider the following code and output: in the context of numeric computing, how is your recommended SELECTED_REAL_KIND any more or less portable?

program p

   use iso_fortran_env, only : real64
   use iso_c_binding, only : c_double

   ! a couple of arbitrary kinds
   integer, parameter :: k1 = selected_real_kind( P=12, R=200 )
   integer, parameter :: k2 = selected_real_kind( P=13, R=300 )

   real*8              :: a
   real(kind=8)        :: b
   real(kind=real64)   :: c
   real(kind=k1)       :: d
   real(kind=k2)       :: e
   real(kind=c_double) :: f

   print *, "real*8 :: a"
   print *, "precision(a) = ", precision(a)
   print *, " exponent(a) = ", range(a)
   print *, "    radix(a) = ", radix(a)
   print "(*(g0))", "PI = ", 4*atan( real(1.0, kind=kind(a)))
   print *

   print *, "real(kind=8) :: b"
   print *, "precision(b) = ", precision(b)
   print *, " exponent(b) = ", range(b)
   print *, "    radix(b) = ", radix(b)
   print "(*(g0))", "PI = ", 4*atan( real(1.0, kind=kind(b)))
   print *

   print *, "real(kind=real64) :: c"
   print *, "precision(c) = ", precision(c)
   print *, " exponent(c) = ", range(c)
   print *, "    radix(c) = ", radix(c)
   print "(*(g0))", "PI = ", 4*atan( real(1.0, kind=kind(c)))
   print *

   print *, "real(kind=k1) :: d"
   print *, "precision(d) = ", precision(d)
   print *, " exponent(d) = ", range(d)
   print *, "    radix(d) = ", radix(d)
   print "(*(g0))", "PI = ", 4*atan( real(1.0, kind=kind(d)))
   print *

   print *, "real(kind=k2) :: e"
   print *, "precision(e) = ", precision(e)
   print *, " exponent(e) = ", range(e)
   print *, "    radix(e) = ", radix(e)
   print "(*(g0))", "PI = ", 4*atan( real(1.0, kind=kind(e)))
   print *

   print *, "real(kind=c_double) :: f"
   print *, "precision(f) = ", precision(f)
   print *, " exponent(f) = ", range(f)
   print *, "    radix(f) = ", radix(f)
   print "(*(g0))", "PI = ", 4*atan( real(1.0, kind=kind(f)))
   print *


end program p

Upon execution using Intel Fortran:

 real*8 :: a
 precision(a) =  15
  exponent(a) =  307
     radix(a) =  2
PI = 3.141592653589793

 real(kind=8) :: b
 precision(b) =  15
  exponent(b) =  307
     radix(b) =  2
PI = 3.141592653589793

 real(kind=real64) :: c
 precision(c) =  15
  exponent(c) =  307
     radix(c) =  2
PI = 3.141592653589793

 real(kind=k1) :: d
 precision(d) =  15
  exponent(d) =  307
     radix(d) =  2
PI = 3.141592653589793

 real(kind=k2) :: e
 precision(e) =  15
  exponent(e) =  307
     radix(e) =  2
PI = 3.141592653589793

 real(kind=c_double) :: f
 precision(f) =  15
  exponent(f) =  307
     radix(f) =  2
PI = 3.141592653589793

Using gfortran,

 real*8 :: a
 precision(a) =           15
  exponent(a) =          307
     radix(a) =            2
PI = 3.1415926535897931

 real(kind=8) :: b
 precision(b) =           15
  exponent(b) =          307
     radix(b) =            2
PI = 3.1415926535897931

 real(kind=real64) :: c
 precision(c) =           15
  exponent(c) =          307
     radix(c) =            2
PI = 3.1415926535897931

 real(kind=k1) :: d
 precision(d) =           15
  exponent(d) =          307
     radix(d) =            2
PI = 3.1415926535897931

 real(kind=k2) :: e
 precision(e) =           15
  exponent(e) =          307
     radix(e) =            2
PI = 3.1415926535897931

 real(kind=c_double) :: f
 precision(f) =           15
  exponent(f) =          307
     radix(f) =            2
PI = 3.1415926535897931


Steve Lionel (Ret.)'s picture

FortranFan, that's quite a lot of text. I point out that I replied in the newsgroup with a detailed explanation of why I disagreed with the advice of using the ISO_FORTRAN_ENV constants and a gfortran developer agreed with me.

I do not agree that the KIND feature is a problem - in fact, I rather like it as it encourages portability if used as intended. It's only when "old-timers" meet it only half-way, encouraged perhaps by vendors who took the simple route of mapping byte sizes (mostly) to kind numbers, that there is a problem. I prefer the Fortran solution to that of PL/I where every declaration had the range and precision spelled out (sort of like putting SELECTED_xxx_KIND on each one), and especially I hated PL/I's nasty rules for determining the precision of mixed-type operations.

I don't agree with your comment regarding gfortran.


Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.