Tuesday, September 27, 2022

(#42) Home Assistant Legacy Sensors - continued

Legacy Sensors - Configuration - Part 2

Home Assistant MQTT Device Discovery


With the MQTT Configuration topics figured out, now it's on to the message payloads. I'll have six configuration payloads to craft for Home Assistant MQTT Device Discovery to work.


There are six sensors in the Tuya Air Quality Monitor, that means six configuration messages to the six MQTT topics created in Part 1.


The unique device ID we created in Part 1 is part of these payloads.  

Something else to decide now is the MQTT topic and JSON payload for the sensor data. When those sensors send their data to the broker, what MQTT topic will they use? What will their data payload format be?

In my case, my Tuya Air Quality Monitor Sensors will publish their values on MQTT topic:

home/eb0b-5a81

And the sensor data payload will look like: 

{"topic": "home/eb0b-5a81", "dateTime": "2022-09-27T15:45:43", "version": "1.0", "state": "online", "temperature": 25.2, "humidity": 38.8, "PM25": 3, "CH2O": 0.002, "VOC": 0.008, "CO2": 359}

Note - the sensor data comes in as a single JSON message to a single MQTT topic.

Configuration Payloads

The documentation for a Sensor Configuration payload is outlined here. Recall that our device ID is "eb0b-05a81"

Each of the six Tuya Air Quality Monitor configuration message carries a payload similar to:

   "availability":[
      {
         "topic":"home/eb0b-05a81/state",
         "value_template":"{{ value_json.state }}"
      }
   ],
   "device":{
      "identifiers":[
         "eb0b-05a81"
      ],
      "manufacturer":"Tuya",
      "model":"Air Quality Sensor",
      "name":"eb0b-05a81",
      "hw_version" : "?",
      "via_device": "python"
   },
   "device_class":"volatile_organic_compounds",
   "enabled_by_default":true,
   "expire_after" : 3600,
   "force_update" : true,
   "name":"AirQuality/A volatile organic compounds",
   "state_class":"measurement",
   "state_topic":"home/eb0b-05a81",
   "unique_id":"eb0b-05a81_volatile_organic_compounds",
   "unit_of_measurement":"ppm",
   "value_template":"{{ value_json.VOC}}"
}

Let's go through the major sections. I'm sure there's a few different ways to do this, but here's what I did.

Availability

Availability - in this section I tell Home Assistant that the state of this sensor, online or off, will come in a message posted to "home/eb0b-05a81/state". And the state value will be in the JSON payload "state". 

Here's my online payload: { "state":"online" }

And my offline payload: { "state":"offline" }

Note! The topics embedded in these messages do NOT start with "homeassistant" as the root topic. The configuration messages are letting Home Assistant know that it will need to subscribe to these topics to pick up sensor data.

We use the idea of "state" again, farther down in the payload. It's a potential for confusion. A better topic for the "availability" section might have been "home/eb0b-05a81/availability" with an availability value in the payload.

As it is - what I've done is tell Home Assistant that it can find the availability status by subscribing to the MQTT topic and extracting the state message from the payload.

Note that I'm NOT using an availability/state topic specific to this sensor. It's NOT "home/eb0b-05a81/volatile_organic_compound/state".  It could be, but because all six sensors are tied to a single monitor, I chose one topic. All six sensors will go online or offline together.


Device and...


This section holds i
nformation about the device. Home Assistant uses this data to tie it into the device registry.  You can read about the attributes; I chose a handful to populate with data meaningful to me.

Moving down, "device_class" controls the Icon displayed in the UI.  For devices of type 'binary sensor' a list of the classes is here: Home Assistant Binary Sensors Device Class.

For a device type of 'sensor' a list of the classes is here: Home Assistant Sensors Device Class.

"expire_after" : means I want this sensor to show offline if no data comes in after 3600 seconds / 1 hour.  

Setting "name" to "AirQuality/A volatile organic compounds" controls how this sensor is displayed in the UI.   I call this monitor "AirQuality/A". I'll call a second monitor "AirQuality/B".

The  "state_topic":"home/eb0b-05a81" tells Home Assistant that the sensor data will be coming in on this MQTT topic. So Home Assistant needs to subscribe to this topic.

The "unique_id" for this VoC sensor is the device ID with a string appended.  And finally the    "value_template":"{{ value_json.VOC}}" tells Home Assistant to parse the VoC value out of the JSON payload's attribute VOC.


That rounds out one of the six Configuration payloads.  For brevity here are the other five.

Temperature Sensor Configuration Payload

{
   "availability":[
      {
         "topic":"home/eb0b-05a81/state",
         "value_template":"{{ value_json.state }}"
      }
   ],
   "device":{
      "identifiers":[
         "eb0b-05a81"
      ],
      "manufacturer":"Tuya",
      "model":"Air Quality Sensor",
      "name":"eb0b-05a81",
      "hw_version":"?",
      "via_device":"python"
   },
   "device_class":"temperature",
   "enabled_by_default":true,
   "expire_after":3600,
   "force_update":true,
   "name":"AirQuality/A_temperature",
   "unique_id":" eb0b-05a81_temperature",
   "state_class":"measurement",
   "state_topic":"home/eb0b-05a81",
   "unit_of_measurement":"°C",
   "value_template":"{{ value_json.temperature }}"
}

Humidity Sensor Configuration Payload

{
   "availability":[
      {
         "topic":"home/eb0b-05a81/state",
         "value_template":"{{ value_json.state }}"
      }
   ],
   "device":{
      "identifiers":[
         "eb0b-05a81"
      ],
      "manufacturer":"Tuya",
      "model":"Air Quality Sensor",
      "name":"eb0b-05a81",
      "hw_version":"?",
      "via_device":"python"
   },
   "device_class":"humidity",
   "enabled_by_default":true,
   "expire_after":3600,
   "force_update":true,
   "name":"AirQuality/A humidity",
   "state_class":"measurement",
   "state_topic":"home/eb0b-05a81",
   "unique_id":"eb0b-05a81_humidity",
   "unit_of_measurement":"%",
   "value_template":"{{ value_json.humidity }}"
}

Carbon Dioxide Sensor Configuration Payload

{
   "availability":[
      {
         "topic":"home/eb0b-05a81/state",
         "value_template":"{{ value_json.state }}"
      }
   ],
   "device":{
      "identifiers":[
         "eb0b-05a81"
      ],
      "manufacturer":"Tuya",
      "model":"Air Quality Sensor",
      "name":"eb0b-05a81",
      "hw_version":"?",
      "via_device":"python"
   },
   "device_class":"carbon_dioxide",
   "enabled_by_default":true,
   "expire_after":3600,
   "force_update":true,
   "name":"AirQuality/A carbon dioxide",
   "state_class":"measurement",
   "state_topic":"home/eb0b-05a81",
   "unique_id":"eb0b-05a81_carbon_dioxide",
   "unit_of_measurement":"ppm",
   "value_template":"{{ value_json.CO2 }}"
}

Formaldehyde Sensor Configuration Payload

{
   "availability":[
      {
         "topic":"home/eb0b-05a81/state",
         "value_template":"{{ value_json.state }}"
      }
   ],
   "device":{
      "identifiers":[
         "eb0b-05a81"
      ],
      "manufacturer":"Tuya",
      "model":"Air Quality Sensor",
      "name":"eb0b-05a81",
      "hw_version":"?",
      "via_device":"python"
   },
   "enabled_by_default":true,
   "expire_after":3600,
   "force_update":true,
   "name":"AirQuality/A formaldehyde",
   "state_class":"measurement",
   "state_topic":"home/eb0b-05a81",
   "unique_id":"eb0b-05a81_formaldehyde",
   "unit_of_measurement":"ppm",
   "value_template":"{{ value_json.CH2O}}"
}

Particulate Matter (2.5) Sensor Configuration Payload

{
   "availability":[
      {
         "topic":"home/eb0b-05a81/state",
         "value_template":"{{ value_json.state }}"
      }
   ],
   "device":{
      "identifiers":[
         "eb0b-05a81"
      ],
      "manufacturer":"Tuya",
      "model":"Air Quality Sensor",
      "name":"eb0b-05a81",
      "hw_version":"?",
      "via_device":"python"
   },
   "device_class":"pm25",
   "enabled_by_default":true,
   "expire_after":3600,
   "force_update":true,
   "name":"AirQuality/A particulate 2.5",
   "state_class":"measurement",
   "state_topic":"home/eb0b-05a81",
   "unique_id":"eb0b-05a81_particulate_matter_2_5",
   "unit_of_measurement":"ug/m³",
   "value_template":"{{ value_json.PM25 }}"
}

Now try it!

Use "mosquitto_pub" to see if you can get a sensor to appear in the Home Assistant UI. Something like this:

mosquitto_pub -h mqttbroker.local -d -t "homeassistant/sensor/eb0b-05a81/humidity/config" -m '{"availability":[{"topic":"home/eb0b-05a81/state","value_template":"{{ value_json.state }}"}],"device":{"identifiers": [ "home_eb0b-05a81"], "manufacturer":"Tuya","model":"Air Quality Sensor","name":"eb0b-05a81","hw_version" : "?","via_device": "python" },"device_class":"humidity", "enabled_by_default":true, "expire_after" : 3600, "force_update" : true, "name":"Air Quality/A humidity",   "state_class":"measurement", "state_topic":"home/eb0b-05a81", "unique_id":"eb0b-05a81_humidity",   "unit_of_measurement":"%",   "value_template":"{{ value_json.humidity }}"}'


Flip over to the Home Assistant Overview page and you should see a new sensor - labeled "AirQuality/A humidity".

It should be greyed-out to indicate that it's offline.  Let's bring it online and send it a value

mosquitto_pub -h mqttbroker.local -d -t "home/eb0b-05a81/state" -m '{ "state":"online" }'

mosquitto_pub -h mqttbroker.local -d -t "home/eb0b-05a81" -m '{ "temperature" : 20.0, "humidity" : 37.2 , "CO2" : 364, "state" : "online", "PM25" : 4, "CH2O" : 0.004, "VOC": 0.012 }'

If all went well, you should have a new sensor in the UI!





After all six sensors are configured, you should see something like this:



Note that I'm not using the "Retain Flag" when publishing configuration messages. I'm still debugging things and didn't want this configuration messages persisted by the broker.  Yet.

Monday, September 26, 2022

(#41) Home Assistant - Legacy Sensors

Adding in Legacy Sensors - Part 1

Those Eaton Home Heartbeat sensors, mentioned in post #2 from May 2014, are still working! 


So, it'd be nice to get them feeding data into my new Home Assistant system.  After reading and researching, my current (as of September 2022) thought is to use "MQTT Device Discovery."


MQTT Device Discovery, in Home Assistant, is the ability to add sensors using carefully formatted MQTT messages sent to the broker.  I've already got an MQTT integration into Home Assistant, and I'm using zigbee2mqtt to feed sensor data to Home Assistant. zigbee2mqtt uses MQTT Device Discovery -- so I'll try it too.


Send the right messages to the broker, and the sensors will appear in Home Assistant. No additional fussing required! Cool!


Reading and Sniffing

A little reading, and a little "eavesdropping" of the messages that zigbee2mqtt sends and I'm able to come up
with a prototype.


For a device, I'm going to start with a non-zigbee sensor, my Tuya WiFi Air Quality Monitor. I've already got Python code that logs into the Tuya IoT cloud, pulls the sensor data and sends it out via MQTT packets to my broker.  

The thought is, to leave that code as is, and just modify the MQTT payload so it matches Home Assistant's MQTT Device Discovery format.


Treat each sensor individually

The format is documented, but I'll share the approach I've taken. First an architectural issue of note. This Air Quality monitor has six sensors (temperature, humidity, carbon dioxide, formaldehyde, particulate matter 2.5, and volatile organic compounds).

Current Best Practice is to treat this one unit as six separate sensors. Don't try to lump them together, yet.

Two MQTT Messages 

At least two. Getting a sensor to show up involves (a) sending a "configuration" message to the "config" topic and (b) sending sensor data to a slightly different topic.  You need at least two message types, I'll be using three.

Configuration - Craft the Root MQTT Topic

MQTT Device Discovery starts with publishing configuration messages to topics, topics that Home Assistant is already subscribed to.  The topic format I'm using starts with:

homeassistant/sensor/<device ID>

"homeassistant" is the default root topic. 

"sensor" is a component that has to match one of the known Home Assistant MQTT component types. See here

And <device ID> is the unique way of identifying this sensor (or group of sensors). In my case, I'm using the unique device ID assigned to my Air Quality sensor from the Tuya IoT cloud.


The device ID assigned to my Monitor is eb0b ... 05a81 (abbreviated as "eb0b-05a81" from now on).  That's what I'll use. You can pick anything you link - MAC address, random characters, as long as its unique to the device.

One Config Message Per Sensor

Each sensor will get a subtopic appended in the format of <sensor description>/configFor the Air Quality Monitor, with it's six sensors, I end up with six configuration topics for MQTT:

homeassistant/sensor/eb0b-05a81/temperature/config

homeassistant/sensor/eb0b-05a81/humidity/config

homeassistant/sensor/eb0b-05a81/carbon_dioxide/config

homeassistant/sensor/eb0b-05a81/formaldehyde/config

homeassistant/sensor/eb0b-05a81/particulate_matter_2_5/config

homeassistant/sensor/eb0b-05a81/volatile_organic_compounds/config


The text for the sensor description appears to not matter - pick what you want. But the final subtopic must be "config". And the start of the topic must be "homeassistant/<component type>/<device ID>"

Now that we have the MQTT Configuration Topics hammered out, onto the Configuration message payloads.