Arduino Azure IoT library contains messages with base64 encoding

Question

I'm using the Azure IoT library for Arduino. Based on the example in the GitHub repository for ESP32, please see: https://github.com/Azure/azure-sdk-for-c-arduino/tree/main/examples/Azure_IoT_Hub_ESP32

I'm sending messages to the Azure IoT Hub based on sensor data, and from there, I'm transferring them to Cosmos DB. However, I noticed that the messages are base64 encoded. My code for sending messages in my Arduino file looks like this:

static void generateTelemetryPayload(const char* activity) {
    time_t now = time(NULL);
    struct tm* timeinfo;
    char timeStr[64];

    timeinfo = localtime(&now);
    strftime(timeStr, sizeof(timeStr), "%Y-%m-%dT%H:%M:%S", timeinfo);

    DynamicJsonDocument json(1024);

    json["category"] = "human_presence";

    if (activity != "") {
        json["activity"] = activity;
    }

    json["zone"] = zone;
    json["timestamp"] = getFormattedTime();

    char mqtt_message[128];
    serializeJson(json, mqtt_message);
    telemetry_payload = mqtt_message;
}

static void sendTelemetry(const char* message) {
    Logger.Info("Sending telemetry ...");

    if (az_result_failed(az_iot_hub_client_telemetry_get_publish_topic(
        &client, NULL, telemetry_topic, sizeof(telemetry_topic), NULL))) {
        Logger.Error("Failed az_iot_hub_client_telemetry_get_publish_topic");
        return;
    }

    generateTelemetryPayload(message);

    if (esp_mqtt_client_publish(
        mqtt_client,
        telemetry_topic,
        (const char*)telemetry_payload.c_str(),
        telemetry_payload.length(),
        MQTT_QOS1,
        DO_NOT_RETAIN_MSG) == 0) {
        Logger.Error("Failed publishing");
    } else {
        Logger.Info("Message published successfully");
    }
}

I found a possible solution that might help. To send your messages as JSON instead of a base64 string, adding UTF-8 and application/json encoding to the message. I don't know how to implement this with the Arduino library. Is there a possibility to send my messages as JSON instead of a base64 string?

Answer 1

Last week, I delved into this issue and discovered a solution. References suggesting that the solution involved adding UTF-8 and application/json encoding to the message. I couldn't find any solutions specific to my issue, especially in combination with the Azure IoT Library for Arduino. After several attempts, I found a solution: adding the string $.ct=application%2Fjson%3Bcharset%3Dutf-8 to the topic resolves the issue. If anyone else is facing the same problem, this is the solution:

static void send_telemetry(const char* message, int time = -1)
{
    az_span telemetry = AZ_SPAN_FROM_BUFFER(telemetry_payload);

    Logger.Info("Sending telemetry ...");

    // The topic could be obtained just once during setup,
    // however if properties are used the topic need to be generated again to
    // reflect the current values of the properties.
    if (az_result_failed(az_iot_hub_client_telemetry_get_publish_topic(
            &hub_client, NULL, telemetry_topic, sizeof(telemetry_topic), NULL)))
    {
        Logger.Error("Failed az_iot_hub_client_telemetry_get_publish_topic");
        return;
    }

    get_telemetry_payload(telemetry, message, time, &telemetry);

    if (esp_mqtt_client_publish(
        mqtt_client,
        (std::string(telemetry_topic) + "$.ct=application%2Fjson%3Bcharset%3Dutf-8").c_str(),
        (const char*)az_span_ptr(telemetry),
        az_span_size(telemetry),
        MQTT_QOS1,
        DO_NOT_RETAIN_MSG) < 0)
    {
        Logger.Error("Failed publishing");
    }
    else
    {
        Logger.Info("Message published successfully");
    }
}

Answer 2

Based on the retrieved documents, it seems that the messages sent by your Arduino device to Azure IoT Hub are base64 encoded. However, it is not clear from the retrieved documents whether it is possible to send messages as JSON instead of a base64 string using the Azure IoT library for Arduino.

One possible solution is to modify your code to send messages as JSON instead of a base64 string. You can try adding UTF-8 and application/json encoding to the message, as suggested in the retrieved documents. To do this, you can modify the sendTelemetry function in your Arduino file to serialize the telemetry payload as a JSON string and set the Content-Type header to application/json. Here is an example of how you can modify the sendTelemetry function:

static void sendTelemetry(const char* message) {
    Logger.Info("Sending telemetry ...");

    if (az_result_failed(az_iot_hub_client_telemetry_get_publish_topic(
        &client, NULL, telemetry_topic, sizeof(telemetry_topic), NULL))) {
        Logger.Error("Failed az_iot_hub_client_telemetry_get_publish_topic");
        return;
    }

    generateTelemetryPayload(message);

    // Serialize the telemetry payload as a JSON string
    DynamicJsonDocument json(1024);
    json["message"] = telemetry_payload;
    String json_payload;
    serializeJson(json, json_payload);

    if (esp_mqtt_client_publish(
        mqtt_client,
        telemetry_topic,
        (const char*)json_payload.c_str(),
        json_payload.length(),
        MQTT_QOS1,
        DO_NOT_RETAIN_MSG) == 0) {
        Logger.Error("Failed publishing");
    } else {
        Logger.Info("Message published successfully");
    }
}

For more details refer to JSON C Source Code Library for IoT Communication and SO.