Writing Endian Independent Code in C++

What does "endian" mean?

Endians are a confusing topic for many people. Hopefully, by reading this article you will understand both what endian means and how to write code to deal with it. So I might as well start with the obvious question—"What does endian mean?"

The actual origin of the term "endian" is a funny story. In Gulliver's Travels by Jonathan Swift, one of the places Gulliver visits has two groups of people who are constantly fighting. One group believes the a hardboiled egg should be eaten from the big, round end (the "big endians") and the other group believes that it should be eaten from the small, pointy end (the "little endians"). Endian no longer has anything to do with hard boiled eggs, but in many ways, the essence of the story (two groups fighting over a completely pointless subject) still remains.

Note:  
This article was originally published to GameDev.net in 2004. It was revised by the original author in 2008 and published in the book Advanced Game Programming: A GameDev.net Collection, which is one of 4 books collecting both popular GameDev.net articles and new original content in print format.

Suppose we start with an unsigned 2 byte (16 bit) long number; we'll use 43707. If we look at the hexadecimal version of 43707, it's 0xAABB. Now, hexadecimal notation is convenient because it neatly splits up the number into its component bytes. One byte is 'AA' and the other byte is 'BB'. But how would this number look in the computer's memory (not hard drive space, just regular memory)? Well, the most obvious way to keep this number in memory would be like this:

| AA | BB |

The first byte is the "high" byte of the number, and the second one is the "low" byte of the number. High and low refers to the value of the byte in the number; here, AA is high because represents the higher digits of the number. This order of keeping things is called MSB (most significant byte) or big endian. The most popular processors that use big endian are the PPC family, used by Macs. This family includes the G3, G4, and G5 that you see in most Macs nowadays.

So what is little endian, then? Well, a little endian version of 0xAABB looks like this in memory:

| BB | AA |

Notice that it's is backwards of the other one. This is called LSB (least significant byte) or little endian. There are a lot of processors that use little endian, but the most well known are the x86 family, which includes the entire Pentium and Athlon lines of chips, as well as most other Intel and AMD chips. The actual reason for using little endian is a question of CPU architecture and outside the scope of this article; suffice to say that little endian made compatibility with earlier 8 bit processors easier when 16-bit processors came out, and 32-bit processors kept the trend.

It gets a little more complicated than that. If you have a 4 byte (32 bit) long number, it's completely backwards, not switched every two bytes. Also, floating-point numbers are handled a little differently as well. This means that you can't arbitrarily change the endian of a file; you have to know what data is in it and what order it's in. The only good news is that if you're reading raw bytes that are only 8 bits at a time, you don't have to worry about endians.

When does the endian affect code?

The short answer is, endians come into play whenever your code assumes that a certain endian is being used. The most frequent time this happens is when you're reading from a file. If you are on a big endian system and write, say, 10 long integers to a file, they will all be written in big endian. If you read them back, the system will assume you're reading big endian numbers. The same is true of little endian systems; everything will work with little endian. This causes problems when you have to use the same file on both little and big endian systems. If you write the file in little endian, big endian systems will get screwed up. If you write it in big endian, little endian systems get screwed up. Sure, you could keep two different versions of the file, one big, one small, but that's going to get confusing very quick, and annoying as well.

The other major time that endians matter is when you use a type cast that depends on a certain endian being in use. I'll show you one example right now (keep in mind that there are many different type casts that can cause problems):

unsigned char EndianTest[2] = { 1, 0 };
short x;

x = *(short *) EndianTest;

So the question is, what's x? Let's look at what this code is doing. We're creating an array of two bytes, and then casting that array of two bytes into a single short. What we've done by using an array is basically forced a certain byte order, and we're going to see how the system treats those two bytes. If this is a little endian system, the 0 and 1 will be interpreted backwards, and will be seen as if it is 0,1. Since the high byte is 0, it doesn't matter and the low byte is 1—x will be equal to 1. On the other hand, if it's a big endian system, the high byte will be 1 and the value of x will be 256. We'll use this trick later to determine what endian our system is using without having to set any conditional compilation flags.

Writing Endian Independent Code

So we finally get down to the most important part; how does one go about writing code that isn't bound to a certain endian? There are many different ways of doing this; the one I'm going to present here was used in Quake 2, and most of the code you'll see here is somewhat modified code out of the Quake 2 source code. It's mostly geared towards fixing files that are written in a certain endian, since the type casting problem is much harder to deal with. The best thing to do is to avoid casts that assume a certain byte order.

So the basic idea is this. Our files will be written in a certain endian without fail, regardless of what endian the system is. We need to ensure that the file data is in the correct endian when read from or written to file. It would also be nice to avoid having to specify conditional compilation flags; we'll let the code automatically determine the system endian.

Step 1: Switching Endians

The first step is to write functions that will automatically switch the endian of a given parameter. First, ShortSwap:

short ShortSwap( short s )
{
  unsigned char b1, b2;

  b1 = s & 255;
  b2 = (s >> 8) & 255;

  return (b1 << 8) + b2;
}

This function is fairly straightforward once you wrap your head around the bit math. We take apart the two bytes of the short parameter s with some simple bit math and then glue them back together in reverse order. If you understand bit shifts and bit ANDs, this should make perfect sense. As a companion to ShortSwap, we'll have ShortNoSwap, which is very simple:

short ShortNoSwap( short s )
{
  return s;
}

This seems utterly pointless at the moment, but you'll see why we need this function in a moment.

Next, we want to swap longs:

int LongSwap (int i)
{
  unsigned char b1, b2, b3, b4;

  b1 = i & 255;
  b2 = ( i >> 8 ) & 255;
  b3 = ( i>>16 ) & 255;
  b4 = ( i>>24 ) & 255;

  return ((int)b1 << 24) + ((int)b2 << 16) + ((int)b3 << 8) + b4;
}

int LongNoSwap( int i )
{
  return i;
}

LongSwap is more or less the same idea as ShortSwap, but it switches around 4 bytes instead of 2. Again, this is straightforward bit math.

Lastly, we need to be able to swap floats:

float FloatSwap( float f )
{
  union
  {
    float f;
    unsigned char b[4];
  } dat1, dat2;
    
  dat1.f = f;
  dat2.b[0] = dat1.b[3];
  dat2.b[1] = dat1.b[2];
  dat2.b[2] = dat1.b[1];
  dat2.b[3] = dat1.b[0];
  return dat2.f;
}

float FloatNoSwap( float f )
{
  return f;
}

This looks a little different than the previous two. There are three major steps here. First, we set one of our unions, dat1, equal to f. The union automatically allows us to split the float into 4 bytes, because of the way unions work. Second, we set each of the bytes in dat2 to be backwards of the bytes in dat1. Lastly, we return the floating-point component of dat2. This union trick is necessary because of the slightly more complex representation of floats in memory (see the IEEE documentation). The same thing can be done for doubles, but I'm not going to show the code here, as it simply involves adding more bytes, changing to double, and doing the same thing.

Step 2: Set function pointers to use the correct Swap function

The next part of our implementation is the clever twist that defines this method of endian independence. We're going to use function pointers to automatically select the correct endian for us. We'll put their declarations in a header file

extern short (*BigShort) ( short s );
extern short (*LittleShort) ( short s );
extern int (*BigLong) ( int i );
extern int (*LittleLong) ( int i );
extern float (*BigFloat) ( float f );
extern float (*LittleFloat) ( float f );

Remember to put them in a C or C++ file without extern so that they are actually defined, or you'll get link errors. Each one of these functions is going to point to the correct Swap or NoSwap function it needs to invoke. For example, if we are on a little endian system, LittleShort will use ShortNoSwap, since nothing needs to change. But if we are on a big endian system, LittleShort will use ShortSwap. The opposite is true of BigShort.

Step 3: Initialization

In order to initialize all of these function pointers, we'll need a function to detect the endian and set them. This function will make use of the byte array cast I showed you earlier as an example of a byte cast that assumes a certain endian.

bool BigEndianSystem;  //you might want to extern this

void InitEndian( void )
{
  byte SwapTest[2] = { 1, 0 };

  if( *(short *) SwapTest == 1 )
  {
    //little endian
    BigEndianSystem = false;

    //set func pointers to correct funcs
    BigShort = ShortSwap;
    LittleShort = ShortNoSwap;
    BigLong = LongSwap;
    LittleLong = LongNoSwap;
    BigFloat = FloatSwap;
    LittleFloat = FloatNoSwap;
  }
  else
  {
    //big endian
    BigEndianSystem = true;

    BigShort = ShortNoSwap;
    LittleShort = ShortSwap;
    BigLong = LongNoSwap;
    LittleLong = LongSwap;
    BigFloat = FloatSwap;
    LittleFloat = FloatNoSwap;
  }
}

Let's examine what's going on here. First we use the cast to check our endian. If we get a 1, the system is little endian. All of the Little* functions will point to *NoSwap functions, and all of the Big* functions will point to *Swap functions. The reverse is true if the system is big endian. This way, we don't need to know the endian of the system we're on, only the endian of the file we're reading or writing.

A Practical Demonstration

Ok, let's suppose we have some sort of structure we need to read from file and write to file, maybe a vertex. Our structure looks like this:

struct Vertex
{
  float Pos[3];
  float Normal[3];
  long Color;
  float TexCoords[2];
};

Nothing special here, just a typical vertex structure. Now we're going to decide that vertices will always be stored to file in little endian. This is an arbitrary choice, but it doesn't matter. What we're going to do is add a function to this struct that will fix the endian after loading or before saving it:

struct Vertex
{
  float Pos[3];
  float Normal[3];
  long Color;
  float TexCoords[2];

  void Endian()
  {
    for(int i = 0; i < 3; ++i) //our compiler will unroll this
    {
      Pos[i]= LittleFloat( Pos[i] );
      Normal[i] = LittleFloat( Pos[i] );
    }
    Color = LittleLong( Color );
    TexCoords[0] = LittleFloat( TexCoords[0] );
    TexCoords[1] = LittleFloat( TexCoords[1] );
  }
};

Let's be honest here; it's not exactly the easiest thing ever. You're going to have to write one of those painfully boring functions for each and every structure that goes in and out of files. After those functions are written, though, writing endian independent code elsewhere is going to be a breeze. Notice that we used Little* functions here because our files are all little endian. If we had decided to use big endian, we could have simply used the Big* functions.

Now what will the actual functions for working with the vertices be like? Well, to read a vertex:

void ReadVertex( Vertex* v, FILE* f )
{
  if( v == NULL || f == NULL )
    return;
    
  fread( v, sizeof(Vertex), 1, f );
    
  //now, our not quite magical endian fix
  v->Endian();
}

Notice the simplicity of correcting the endian; although our structure definitions are slightly messy, the loading code has become very simple. The code to save is equally simple:

void WriteVertex( Vertex* v, FILE* f )
{
  if( v == NULL || f == NULL )
    return;

  v->Endian();
  fwrite( v, sizeof(Vertex), 1, f );
}

And that's all there is to it. I'm including a sample C++ header/source file pair that you're free to use in your own programs; it's GPL code though (since it's actually part of a GPL project I'm working on) so you'll have to write your own if you don't want to release your code.