I have had a few glorious weeks of C coding at my day job. It is some of the best C code I have ever written, and I am quite proud of it.

During this project, I learned something new. In the past, I would commonly use #define for values such as:

#define NONE 0
#define WARNING 1
#define ERROR 2

There might be some code somewhere that made us of those values:

void HandleError (int error)
{
   case NONE:
      // All fine.
      break;

   case WARNING:
      // Handle warning.
      break;

   case ERROR:
      // Handle error
      break;

    default:
      // Handle unknown error.
      break;
}

When you use #define like that, the C pre-processor will just do simple substitutions before compiling the code — “NONE” will become “0”. Because of this, those labels mean nothing special to the compiler — they are just numbers.

But, I had learned you could use an enum and ensure a function only allows passing the enum rather than “whatever int”:

typedef enum
{
   NONE = 0,
   WARNING = 1,
   ERROR = 2
} ErrorEnum;

Now the routine can take an “ErrorEnum” as a parameter type.

void HandleError (ErrorEnum error)
{
   case NONE:
      // All fine.
      break;

   case WARNING:
      // Handle warning.
      break;

   case ERROR:
      // Handle error
      break;

    default:
      // Handle unknown error.
      break;
}

This looks nicer, since it is clear what is being passed in (whatever “ErrorEnum” is). It does not necessarily give any extra compiler warnings if you pass in an int instead, since an enum is just an int, by default.

In C, an enum is a user-defined data type that represents a set of named values. By default, the underlying data type of an enum is int, but it can be explicitly specified to be any integral type using a colon : followed by the desired type.

– ChatGPT

Thus, the compiler I am using for this test (Visual Studio, building a C program) does not complain if I do something like this:

int error = 1;
HandleError (error);

However, I just learned of an interesting benefit to using an enum for a switch/case. The compiler can warn you if you don’t have a case for all the items in the enum!

typedef enum {
	NONE,
	WARNING,
	ERROR
} ErrorEnum;

void HandleError(ErrorEnum error)
{
	switch (error)
	{
	case NONE:
		// All fine.
		break;

	case WARNING:
		// Handle warning.
		break;

}

Building that with warnings enabled reports:

enumerator 'ERROR' in switch of enum 'ErrorEnum' is not handled	CTest	

How neat! I actually ran in to this when I had added some extra error types, but a routine that converted them to a text string did not have cases for the new errors.

const char *GetErrorString (ErrorEnum error)
{
   const char *errorStringPtr = "Unknown";

   switch NONE:
      errorStringPtr = "None";
      break;
}

That is a useful warning, and it found a bug/oversight.

Because of this, I will never use #defines for things like this again. Not only does a data type look more obvious in the code, the compiler can give you extra warnings.

Until next time…

In a recent article, I discussed methods to copy variables (or structure elements) in to a packed buffer, as well as parse buffer bytes back in to variables.

Doing it manually was clunky. Using my GetPutData routines made it easier to be clunky.

But today, we’ll make it super simple and far less clunky.

Defining the message bytes

Let’s start with a simple message. It will be represented by a C structure that we’d use in the program. For example, here are the elements that might be part of a “Set Date and Time” message:

typedef struct
{
    uint16_t year;
    uint8_t month;
    uint8_t day;
    uint8_t hour;
    uint8_t minute;
    uint8_t second;
    uint8_t timeZone;
    bool isDST;
} SetDateTimeMessageStruct;

The packed message would be nine bytes and look like this:

[ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 ]
  \  /    |   |   |   |   |   |   |
  year   mon day hr  min sec tz  dst

To automate this process, a lookup table is created that represents the size of each element and the offset to where it is located within the structure.

offsetof()

Inside stddef.h is a macro that is used to calculate where in the structure’s memory a particular element exists. It is a relative offset from wherever the structure variable starts in memory. Here is an example showing offsetof():

#include <stdio.h>
#include <stdlib.h>  // for EXIT_SUCCESS
#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>  // for offsetof()

typedef struct
{
    uint8_t     a;
    uint16_t    b;
    bool        c;
    uint32_t    d;
    float       e;
    double      f;
} MyStruct;

int main()
{
    MyStruct test;
    
    printf ("sizeof(test) = %ld\n", sizeof(test));
    
    printf ("a is %lu bytes at offset %lu\n",
            sizeof(test.a), offsetof(MyStruct, a));
    printf ("b is %lu bytes at offset %lu\n",
            sizeof(test.b), offsetof(MyStruct, b));
    printf ("c is %lu bytes at offset %lu\n",
            sizeof(test.c), offsetof(MyStruct, c));
    printf ("d is %lu bytes at offset %lu\n",
            sizeof(test.d), offsetof(MyStruct, d));
    printf ("e is %lu bytes at offset %lu\n",
            sizeof(test.e), offsetof(MyStruct, e));
    printf ("f is %lu bytes at offset %lu\n",
            sizeof(test.f), offsetof(MyStruct, f));

    return EXIT_SUCCESS;
}

Try it: https://onlinegdb.com/GDvhi9TLq

It produces the following output:

sizeof(test) = 24
a is 1 bytes at offset 0
b is 2 bytes at offset 2
c is 1 bytes at offset 4
d is 4 bytes at offset 8
e is 4 bytes at offset 12
f is 8 bytes at offset 16

This is an example of how elements in a structure may be padded to ensure each one starts on an even-byte in memory. ‘a’ is one byte starting at offset 0, but ‘b’ doesn’t start at 1. As a 16-bit value, it has to skip a byte and start at offset 2. There are more skipped bytes after ‘c’ at offset 4. Since the next value is a 32-bit value, it skips three bytes.

If we just wrote the structure memory out (in this architecture), it would look like this:

[ 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15|16|17|18|19|20|21|22|23]
  |     |     |           |           |           |
  a     b--b  c           d--d--d--d  e--e--e--e  f--f--f--f--f--f--f--f

That would waste 4 bytes in that 24 byte message. Plus, if the message were parsed by a system that had different padding/alignment requirements, they wouldn’t be able to read the buffer back in to their structure variable and get the correct data.

Using offsetof() to determine where in a structure an element exists, a lookup table could be created. A C function could be written to use that table to know where to copy the data. Here’s a rough concept:

typedef struct
{
    size_t size;   // size of structure element
    size_t offset; // offset from start of structure to that element
} ElementOffsetStruct;

MyStruct test;

ElementOffsetStruct testTable[] =
{
    { sizeof(test.a), offsetof(MyStruct, a) },
    { sizeof(test.b), offsetof(MyStruct, b) },
    { sizeof(test.c), offsetof(MyStruct, c) },
    { sizeof(test.d), offsetof(MyStruct, d) },
    { sizeof(test.e), offsetof(MyStruct, e) },
    { sizeof(test.f), offsetof(MyStruct, f) },
    { 0, 0 } // end of table
};

for (int idx=0; testTable[idx].size != 0; idx++)
{
    printf ("? is %u bytes at offset %u\n",
            testTable[idx].size, testTable[idx].offset);
}

Knowing where items are in the structure allows the specific element bytes to be copied to a buffer. Something like this:

uint8_t buffer[80];
unsigned int offset;

offset = 0;
for (int idx=0; testTable[idx].size != 0; idx++)
{
    printf ("Copy %lu bytes at structure offset %lu to buffer offset %u\n",
            testTable[idx].size, testTable[idx].offset, offset);

    memcpy (buffer + offset,
        (uint8_t*)&test + testTable[idx].offset,
        testTable[idx].size);
        
    offset = offset + testTable[idx].size; 
}

“And it’s just that easy.” Running the following test program dumps the memory contents of the C structure “test” and then writes them out to a buffer using this code and then dumps the memory contents of the buffer.

#include <stdio.h>
#include <stdlib.h>  // for EXIT_SUCCESS
#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>  // for offsetof()
#include <string.h>  // for memcpy()

typedef struct
{
    uint8_t     a;
    uint16_t    b;
    bool        c;
    uint32_t    d;
    float       e;
    double      f;
} MyStruct;

typedef struct
{
    size_t size;   // size of structure element
    size_t offset; // offset from start of structure to that element
} ElementOffsetStruct;

int main()
{
    MyStruct test = { 0x11, 0x2222, true, 0x33333333, 42.42, 42.42 };
    
    ElementOffsetStruct testTable[] =
    {
        { sizeof(test.a), offsetof(MyStruct, a) },
        { sizeof(test.b), offsetof(MyStruct, b) },
        { sizeof(test.c), offsetof(MyStruct, c) },
        { sizeof(test.d), offsetof(MyStruct, d) },
        { sizeof(test.e), offsetof(MyStruct, e) },
        { sizeof(test.f), offsetof(MyStruct, f) },
        { 0, 0 } // end of table
    };
    
    for (int idx=0; idx<sizeof(test); idx++)
    {
        printf ("%02x ", ((uint8_t*)&test)[idx]);
    }
    printf ("\n");

    uint8_t buffer[80];
    unsigned int offset;
    
    offset = 0;
    for (int idx=0; testTable[idx].size != 0; idx++)
    {
        printf ("Copy %lu bytes at structure offset %lu to buffer offset %u\n",
                testTable[idx].size, testTable[idx].offset, offset);
    
        memcpy (buffer + offset,
            (uint8_t*)&test + testTable[idx].offset,
            testTable[idx].size);
            
        offset = offset + testTable[idx].size; 
    }

    for (int idx=0; idx<offset; idx++)
    {
        printf ("%02x ", buffer[idx]);
    }
    printf ("\n");

    return EXIT_SUCCESS;
}

I get the following output:

11 00 22 22 01 00 00 00 33 33 33 33 14 ae 29 42 f6 28 5c 8f c2 35 45 40 
Copy 1 bytes at structure offset 0 to buffer offset 0
Copy 2 bytes at structure offset 2 to buffer offset 1
Copy 1 bytes at structure offset 4 to buffer offset 3
Copy 4 bytes at structure offset 8 to buffer offset 4
Copy 4 bytes at structure offset 12 to buffer offset 8
Copy 8 bytes at structure offset 16 to buffer offset 12
11 22 22 01 33 33 33 33 14 ae 29 42 f6 28 5c 8f c2 35 45 40 

As I discovered when I figured this out, there are a few gotchas. First, the sample code here requires the structure variable to exist. This is because the macro sizeof() needs to have something to work on to determine the size:

sizeof(test.a)

I asked ChatGPT about this, and it provided me a clever workaround that worked on the structure itself rather than a variable:

sizeof(((MyStruct*)0)->a)

Looking at that, I think I get how it works. First, it is taking a memory location of 0 and casting that to be a pointer to a MyStruct. No such structure exists at memory 0, but you can cast a pointer to anywhere (even 0) as any kind of pointer you want. (Actually using a pointer that point to 0 would be bad, however.)

Then, it gets the offset of the “a” element from the structure we pretend is located at memory location 0. Let’s break that apart a bit:

sizeof(                 )
       (            )->a
        (MyStruct*)0

And this is enough to allow creating a table based on a structure, and not on a variable declared using that structure.

typedef struct
{
    uint8_t u8;
    uint32_t u32;
} SmallStruct;

ElementOffsetStruct smallTable[] =
{
    { sizeof(((SmallStruct*)0)->u8), offsetof(SmallStruct, u8) },
    { sizeof(((SmallStruct*)0)->u32), offsetof(SmallStruct, u32) },
    { 0, 0 }
};

This would allow an easy way to define the structure and a table of offsets for later use. I’d probably make the table structure const since it won’t need to change.

With this framework figured out, a reverse function could be done that would copy bytes from a buffer in to the correct offset inside the structure.

unsigned int offset;
    
offset = 0;
for (int idx=0; testTable[idx].size != 0; idx++)
{
    printf ("Copy %lu bytes at buffer offset %u to structure offset %lu\n",
            testTable[idx].size, offset, testTable[idx].offset);
    
    memcpy ((uint8_t*)&test + testTable[idx].offset,
            buffer + offset,
           testTable[idx].size);
            
    offset = offset + testTable[idx].size; 
}

And now, by creating a structure and a table of size/offsets, any structure can be written to a buffer, or decoded from a buffer.

Too. Much. Typing.

For the final touch, macros can be made to simplify building the table.

#define ENTRY(s, v) { sizeof(((s*)0)->v), offsetof(s, v) }
#define ENTRYEND    {0, 0}

Now, instead of having to type all that gibberish out, the table can be simplified:

ElementOffsetStruct testTable[] =
{
    ENTRY(MyStruct, a),
    ENTRY(MyStruct, b),
    ENTRY(MyStruct, c),
    ENTRY(MyStruct, d),
    ENTRY(MyStruct, e),
    ENTRY(MyStruct, f),
    ENTRYEND
};

And this is just what I have posted to my GitHub:

https://github.com/allenhuffman/StructureToFromBuffer

You can find a short README there that explains the two functions and how to use them. It greatly simplifies the process of creating message buffers or parsing them in to C structures that can easily be used in a program.

If you try it and find it useful, please let me know.

Until then…