4,391 Posts served

10,712 Conversations started

A nice 64-bit error in C

By Andrey Karpov (54 posts) on December 2, 2009 at 5:07 pm

In C language, you may use functions without defining them. Pay attention that I speak about C language, not C++. Of course, this ability is very dangerous. Let us have a look at an interesting example of a 64-bit error related to it. Below is the correct code that allocates and uses three arrays, 1 GB each:

#include < stdlib.h >

void test()
{
  const size_t Gbyte = 1024 * 1024 * 1024;
  size_t i;
  char *Pointers[3];

  // Allocate
  for (i = 0; i != 3; ++i)
    Pointers[i] = (char *)malloc(Gbyte);

  // Use
  for (i = 0; i != 3; ++i)
    Pointers[i][0] = 1;

  // Free
  for (i = 0; i != 3; ++i)
    free(Pointers[i]);
}

This code correctly allocates memory, writes one into the first item of each array and frees the allocated memory. The code is absolutely correct on a 64-bit system.

Now delete or comment the line "#include < stdlib.h >". The code still compiles but the program crashes after the launch. As the header file "stdlib.h" is disabled, the C compiler considers that malloc function will return int type. The first two allocations are most likely to be successful. After the third call, malloc function will return the array’s address outside the range of the first two Gbyte. As the compiler considers the function’s result to have int type, it interprets the result incorrectly and saves the incorrect value of the pointer in Pointers array.

To make it clearer, let us consider an assembler code generated by Visual C++ compiler for the 64-bit Debug version. At first look at the correct code generated when malloc function is defined (i.e. the file "stdlib.h" is included):

Pointers[i] = (char *)malloc(Gbyte);
mov   rcx,qword ptr [Gbyte]
call  qword ptr [__imp_malloc (14000A518h)]
mov    rcx,qword ptr [i]
mov    qword ptr Pointers[rcx*8],rax

Now consider the variant of the incorrect code when malloc function is not defined:

Pointers[i] = (char *)malloc(Gbyte);
mov    rcx,qword ptr [Gbyte]
call   malloc (1400011A6h)
cdqe
mov    rcx,qword ptr [i]
mov    qword ptr Pointers[rcx*8],rax

Consider the CDQE instruction (Convert doubleword to quadword). The compiler supposed the result to be kept in eax registers and extended it to a 64-bit value to write into Pointers array. Respectively, the high-order bits of rax register are lost. Even if the address of the allocated memory is inside the range of the first 4 GB, we still get the incorrect result when the high-order bit of eax register equals 1. For example, the address 0x81000000 turns into 0xFFFFFFFF81000000.

Fortunately, this type of errors is easy to define. For example, Visual C++ compiler generates two warnings informing about a potential problem:

warning C4013: 'malloc' undefined; assuming extern returning int

warning C4312: 'type cast' : conversion from 'int' to 'char *' of greater size

And PVS-Studio 3.40 analyzer generates the warning "error V201: Explicit type conversion. Type casting to memsize.".

Categories: Parallel Programming, Software Tools
Tags: 64-bit, 64-bit Coding, 64-bit migration, С

For more complete information about compiler optimizations, see our Optimization Notice.

Comments (4)

December 3, 2009 2:48 AM PST Thomas

I investigated this error on Linux with gcc because I wondered if there would be a similar error ... First of all, if I comment the line "#include <stdlib.h>", gcc complains about size_t. Obviously, VC++ has its definition from somewhere and gcc doesn't. On my 64bit system, gdb taught me that a size_t is actually a long unsigned int, so I inserted a typedef instead. Now I get some errors, too: main.c:22: warning: incompatible implicit declaration of built-in function ‘malloc’ main.c:31: warning: incompatible implicit declaration of built-in function ‘free’ But no complains about an invalid type cast (C4312 above). And the assembler looks the very same for both versions: movq -24(%rbp), %rbx movq -32(%rbp), %rdi call malloc movq %rax, -64(%rbp,%rbx,8) addq $1, -24(%rbp) I stopped there. Both compilers seems to make different assumptions about return values of unknown functions ...

December 3, 2009 7:21 AM PST jimdempseyatthecove Total Points: 73,324 Black Belt

Andrey, This is a very good example of a program that compiles without errors, and more importantly runs without errors in a limited test situation, but may fail under normal use. Not only malloc will suffer this consequence (on x32), but functions returning pointers. Jim Dempsey

December 3, 2009 2:40 PM PST jimdempseyatthecove Total Points: 73,324 Black Belt

malloc would return an uintptr_t (or intptr_t) not size_t

February 17, 2010 11:57 AM PST Martin Kosina

Just ran into this with an older libpki... this has to be the most time spent vs. LOC modified (1) on any bug. On 64bit RHEL, the underlaying malloc returned the "high" address only every 5th or so call and often never with gdb attached. Talk about pulling hairs out. Thanks for the detailed background.

Trackbacks (0)

Leave a comment