How to Speak From Arduino to Python and Back

While I have work on my pet project( a network of sensors) I have faced with the task to send data from Arduino to python backend and back.

At first, I want to say that I didn’t use HTTP requests because my Arduino nodes don’t have the internet connections just radio network to “Hub”. And Hub retranslates what he receive by the radio channel to MQTT broker via the internet.

The first version was very simple and I used C structures. It’s quite simple from both sides. But for me, it was not useful because radio has a limitation on package size of 32 bytes and if I left some data not filled they are still in structure and use some space.

>>> from struct import pack, unpack
>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)

The second option(and last for now) was to use Google Protocol Buffer. Benefits of using this tool it’s if I left some data blank they don’t exist in result package. For example, I have bytes array with a maximum length of 128 bytes to pass settings or more complex data that float value(value from the sensor) and I didn’t put anything on that field I will not have this 128 bytes in result package.

Google provide this library with support for C++, C#, Go, Java and Python. But version for C++ too big to use with Arduino. And for Arduino exists NanoPb library that implements support of protobuf for Arduino.

Here is simple example how I use protobuf in my project.

Message structure:

syntax = "proto2";

message Message {
    enum MSG_KIND {
        UNDEFINED = 0;
        COMMAND = 1;
        REGISTRATION = 2;
        SENSOR_DATA = 3;
        SETTINGS = 4;
    }

    message SensorData {
        enum SENSOR_KIND {
            UNDEFINED = 0;
            VOLTAGE = 1;
            TEMPERATURE = 2;
            HUMIDITY = 3;
            CO2 = 4;
        }

        required SENSOR_KIND kind = 1;
        optional float value = 2;
        optional bytes data = 3;
    }

    required uint32 node_id = 1;
    required MSG_KIND kind = 2;
    optional bytes data = 3;
    repeated SensorData sensors = 4;
};

Example how to work in Arduino:

#include <pb_decode.h>
#include "Message.pb.h"

uint8_t data[128] = {'\0'};
Message msg = Message_init_zero;
pb_istream_t stream = pb_istream_from_buffer(data, sizeof(data));
bool status = pb_decode(&stream, Message_fields, &msg);

if (!status) {
    Serial.println(".. message decode error");
    Serial.println(PB_GET_ERROR(&stream));
}

Serial.print("msg type: ");
Serial.println(msg.kind);
Serial.print("new node id: ");
Serial.println(msg.node_id);

And python example:

from Message_pb2 import Message

msg = Message()
try:
    msg.ParseFromString(payload.data)
except DecodeError:
    pass # capture exeption
else:
    print('=== {} ===='.format(msg.node_id))
    print(msg)

For now Google Protocol Buffer is the best solution that I found to pack the message and that gives more or less easy way to work with from both sides.