Implementing MQTT Payload Serialization with Protobuf

Use Protocol Buffers to serialize complex sensor data into compact binary formats, significantly reducing bandwidth usage on constrained IoT devices.

In this lesson, you will learn how to optimize your IoT bandwidth usage by replacing verbose JSON payloads with compact Protocol Buffer (Protobuf) binary serialization in your MQTT messages.

When transmitting sensor data over constrained networks—such as cellular IoT or LoRaWAN—the overhead of text-based formats like JSON is significant. JSON requires sending repeated key strings (e.g., "temperature": 25.5) every time, which consumes precious bytes and radio airtime. Protobuf addresses this by using a pre-defined schema to encode data into a dense, type-tagged binary format.

Defining the Schema

First, you define your message structure in a .proto file. This acts as a contract between your embedded device and your backend:

syntax = "proto3";

message SensorData {
  uint32 device_id = 1;
  float temperature = 2;
  int64 timestamp = 3;
}

Implementing Serialization

When using the Protobuf-c library on your embedded target, the compiler generates C structures and serialization functions based on your schema. Instead of constructing a string buffer, you populate the C struct and invoke the encoder:

SensorData msg = SENSOR_DATA__INIT;
msg.device_id = 101;
msg.temperature = 22.4f;
msg.timestamp = 1672531200;

size_t len = sensor_data__get_packed_size(&msg);
uint8_t buffer[len];
sensor_data__pack(&msg, buffer);

// Now 'buffer' contains the compact binary data ready for MQTT publication
mqtt_publish(client, "sensors/telemetry", buffer, len, MQTT_QOS_1);

By sending buffer directly over MQTT, you eliminate the need for key-value labels in the payload. The broker only receives the raw values, significantly reducing the payload size—often by 60-80% compared to JSON. Because the protocol is strictly typed and position-based (using field IDs), the receiving end uses the exact same .proto schema to deserialize the binary stream back into native data types. This approach saves memory, reduces power consumption by shortening radio transmission times, and enforces a rigid schema that prevents malformed data issues at the edge.

Key Takeaway: By using Protobuf to serialize MQTT payloads, you shift from human-readable text to machine-efficient binary, drastically reducing bandwidth and airtime costs in bandwidth-constrained IoT applications.