(#41) Home Assistant - Legacy Sensors

Adding in Legacy Sensors - Part 1

Those Eaton Home Heartbeat sensors, mentioned in post #2 from May 2014, are still working! 

So, it'd be nice to get them feeding data into my new Home Assistant system.  After reading and researching, my current (as of September 2022) thought is to use "MQTT Device Discovery."

MQTT Device Discovery, in Home Assistant, is the ability to add sensors using carefully formatted MQTT messages sent to the broker.  I've already got an MQTT integration into Home Assistant, and I'm using zigbee2mqtt to feed sensor data to Home Assistant. zigbee2mqtt uses MQTT Device Discovery -- so I'll try it too.

Send the right messages to the broker, and the sensors will appear in Home Assistant. No additional fussing required! Cool!

Reading and Sniffing

A little reading, and a little "eavesdropping" of the messages that zigbee2mqtt sends and I'm able to come up
with a prototype.

For a device, I'm going to start with a non-zigbee sensor, my Tuya WiFi Air Quality Monitor. I've already got Python code that logs into the Tuya IoT cloud, pulls the sensor data and sends it out via MQTT packets to my broker.  

The thought is, to leave that code as is, and just modify the MQTT payload so it matches Home Assistant's MQTT Device Discovery format.

Treat each sensor individually

The format is documented, but I'll share the approach I've taken. First an architectural issue of note. This Air Quality monitor has six sensors (temperature, humidity, carbon dioxide, formaldehyde, particulate matter 2.5, and volatile organic compounds).

Current Best Practice is to treat this one unit as six separate sensors. Don't try to lump them together, yet.

Two MQTT Messages 

At least two. Getting a sensor to show up involves (a) sending a "configuration" message to the "config" topic and (b) sending sensor data to a slightly different topic.  You need at least two message types, I'll be using three.

Configuration - Craft the Root MQTT Topic

MQTT Device Discovery starts with publishing configuration messages to topics, topics that Home Assistant is already subscribed to.  The topic format I'm using starts with:

homeassistant/sensor/<device ID>

"homeassistant" is the default root topic. 

"sensor" is a component that has to match one of the known Home Assistant MQTT component types. See here. 

And <device ID> is the unique way of identifying this sensor (or group of sensors). In my case, I'm using the unique device ID assigned to my Air Quality sensor from the Tuya IoT cloud.

The device ID assigned to my Monitor is eb0b ... 05a81 (abbreviated as "eb0b-05a81" from now on).  That's what I'll use. You can pick anything you link - MAC address, random characters, as long as its unique to the device.

One Config Message Per Sensor

Each sensor will get a subtopic appended in the format of <sensor description>/config. For the Air Quality Monitor, with it's six sensors, I end up with six configuration topics for MQTT:

homeassistant/sensor/eb0b-05a81/temperature/config

homeassistant/sensor/eb0b-05a81/humidity/config

homeassistant/sensor/eb0b-05a81/carbon_dioxide/config

homeassistant/sensor/eb0b-05a81/formaldehyde/config

homeassistant/sensor/eb0b-05a81/particulate_matter_2_5/config

homeassistant/sensor/eb0b-05a81/volatile_organic_compounds/config

The text for the sensor description appears to not matter - pick what you want. But the final subtopic must be "config". And the start of the topic must be "homeassistant/<component type>/<device ID>"

Now that we have the MQTT Configuration Topics hammered out, onto the Configuration message payloads.

Adding Legacy Sensors to Home Assistant - Part 2 - Payloads

(#40) Garage Door Revisited

 Time passes - Garage Door Part Deux

Let's see: COVID, getting a son to school in Stockholm, getting the other two kids off to college.  Fixing the electric steps on the RV for the umpteenth time. Starlink testing (works great).  Work. Turning 65. Checking the 401K balance. Oh yeah - hip replacement. Blowing out the tendon in my left knee. Little stuff.

Recall that I've made a run at keeping the garage door closed before. Look here.

Switched the Eaton Home Heartbeat system from a Raspberry Pi 2 to a Pi Zero W 2 - and had things stop working. The FTDI driver on the Zero is slightly different from the driver that loads on the Pi 2. Same "version" of Raspberry Pi OS best I can tell.

On the Pi Zero - the serial port will work for a time, and then garbage data floods the line and communication with the Home Heartbeat dies.  I spent very little time looking into this but switching back to the Pi 2 and a slightly "new" rev of the FTDI driver made the problem go away.

 

Zigbee.  There's a renewed interest in Zigbee here.  More so as the number of WiFi devices multiplies. If someone could invent a wearable Faraday cage, I'd buy it.

Months ago, I grabbed the SonOff Zigbee 3.0 USB dongle and a handful of cheap
Zigbee sensors from Aliexpress.

From there was a short hop to zigbee2mqtt as my commitment to MQTT remains strong. It works. It works great - 'nuff said. Getting that installed was easy, but paring devices, moving configurations to different computers - was a bit of a chore. But it's up and running.

From that hop, a skip to Home Assistant. Same thing - installation was a bit of a learning curve. Eventually I bailed on installing "core" on an spare Ubuntu laptop and grabbed a spare Pi 3B+ and loaded their image.  That is clearly the easiest route to get things up and running.

 

 

And then the final jump to ESPHome.

Wow! What a delightful hunk of software! Again a few hiccups getting my NodeMCU to connect initially but it was because I had loaded an image already and had forgotten how to reset the device into boot mode. Once that was done, the basic firmware image was uploaded and the ESP8266 device was online. It appeared immediately in both ESPHome and Home Assistant.

[ PS: Don't use Firefox - it doesn't support WebSerial.  I downloaded MS Edge and got things closer. I also went through the steps to build and install "esphomeflasher" and that worked great. ]

Sweet. Fantastic job on the development!

See here:

And here:

There are quite a few examples of how to write a YAML configuration for controlling a momentary relay. That's how my garage door works - to raise or lower, it's a momentary press of the button.  Here's my version 0.1 YAML that's working on my test bench:

esphome:
  name: test-8266-hiletgo

esp8266:
  board: nodemcuv2

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "blah blah blah"

ota:
  password: "blah blah blah"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Test-8266-Hiletgo"
    password: "blah blah blah"

captive_portal:

switch:
  - platform: gpio
    pin:
      number: 14
      inverted: true      
    id: relay
    name: "Garage Remote"
    icon: "mdi:car-connected"
    restore_mode: always_off
    on_turn_on:
    - delay: 1000ms
    - switch.turn_off: relay

Let's get right to the wiring since that's where I learned a few more things. On these relays, the pin outs are (Thank you RandomNerd Tutorials!):

I run the two wires to the garage door from the relay's common and "normally open" connections. This seemed to have the desired behavior when the power to the NodeMCU was cut (the relay did not trigger).  For now the JD-VCC / VCC jumper is on - meaning the power to the relay is coming from the NodeMCU board. I think this will be fine, as there should be little power on the relay. The door trigger is simply a momentary close (continuity) of the circuit.

Ground on the 4 pin connector goes to GND on the NodeMCU. VCC (on the 4 pin) goes to the 3.3V pin on the NodeMCU.

And IN1 is connected to D5 - GPIO 14. Again thank you to RandomNerd Tutorials for pointing out that the pin you pick does matter!  Some pins are triggered on boot. And this would trigger the door if the NodeMCU lost power, or rebooted. We don't want that.

Quote:  "Some ESP8266 pins output a 3.3V signal when the ESP8266 boots. This may be problematic if you have relays or other peripherals connected to those GPIOs.

Additionally, some pins must be pulled HIGH or LOW in order to boot the ESP8266.

Taking this into account, the safest ESP8266 pins to use with relays are: GPIO 5, GPIO 4, GPIO 14, GPIO 12 and GPIO 13."

 

Note the two lines highlighted in red in the script. "inverted: true" was the only way I could find to have the pin power up as Low.  Without this, when the NodeMCU booted, the relay was powered up and continutiy established -- ie. like holding the garage door remote button down.

Not good.  Using "inverted: true" now powers up with the relay not engaged; no continuity; button not mashed.

And "on_turn_on" seems to be a great way to get the behavior I want from the relay - energize (mash button), hold for a second, then release.

It's time to head out to the garage, unplug the Raspberry Pi solution that's been there for years and try this. One thing I'll lose is the automation and rules I had about closing the door.  I'll have to see if they're able to be recreated in Home Assistant.

(#39) NodeMCU Analog Multiplexer

Sharing is Caring

There's but one analog input on the NodeMCU V3 board I purchased some months ago. And I'd like to connect two (or more) analog sensors.

NodeMCU V3 Lolin Board

 

This is where an Analog Multiplexer, like the Sparkfun 16 Channel Analog/Digital Multiplexer/Demultiplexer can help. Quoting from their product's webpage:

This chip is like a rotary switch - it internally routes the common pin (COM in the schematic, SIG on the board) to one of 16 channel pins (CHANxx). It works with both digital and analog signals (the voltage can't be higher than VCC), and the connections function in either direction. To control it, connect 4 digital outputs to the chip's address select pins (S0-S3), and send it the binary address of the channel you want (see the datasheet for details). This allows you to connect up to 16 sensors to your system using only 5 pins!

See (https://www.sparkfun.com/products/9056).

Wiring Diagram

As a quick test, I wired the multiplexer to the NodeMCU and sensors as depicted:

Wiring Diagram

Note I'm using pins D0 through D3 for controlling the multiplexer - these may not be the best pins to choose for a production application. But they worked for the test.

Pin D0 on the NodeMCU connects to pin S3 on the multiplexer, D1 to S2, D2 to S1, D3 to S0.

The Signal (S) pin on Voltage Sensor 1 connects to pin C0 on the multiplexer. For Voltage Sensor 2, the S pin connects to C1.

VCC is connected to the 3.3V pin on the NodeMCU and the grounds are tied together.

Note that I had to connect the ground on the Voltage Sensor GND input to the NodeMCU's ground too to get an accurate reading.

The Code

Selecting the channels on the multiplexer is easy. First ensure the pins are enabled for output:

{

   Serial.println("Setting up the Output pins");

   pinMode( D0, OUTPUT );

   pinMode( D1, OUTPUT );

   pinMode( D2, OUTPUT );

   pinMode( D3, OUTPUT );

}

Consult the Chip's documentation for a Truth Table on how to toggle the pins to select a channel.

CD74HC4067, CD74HCT4067 Truth Table

Note that I did not connect the "E" pin on the board, so I cannot turn off all channels. One will always be connected.

Selecting Channels is straightforward. To pick Channel 0, set all four S pins low (to zero):

void selectChannel0 ()

{

   setS0( 0 );

   setS1( 0 );

   setS2( 0 );

   setS3( 0 );

}

To select Channel 5, S0 and S2 are set to high, the others are set to low:

void selectChannel5()

{

  setS0( 1 );

  setS1( 0 );

  setS2( 1 );

  setS3( 0 );

}

The code to drive a S pin High or Low is also easy:

// ---------------------------------------------------------------

void setS0 (int value)

{

   // S0 is connected to D3/GPIO 0

   if (value == 1)

     digitalWrite( D3, HIGH );

   else

     digitalWrite( D3, LOW );

}

// ---------------------------------------------------------------

void setS1 (int value)

{

   // S1 is connected to D2/GPIO 4

   if (value == 1)

     digitalWrite( D2, HIGH );

   else     

   digitalWrite( D2, LOW );

}

// ---------------------------------------------------------------

void setS2 (int value)

{

   // S2 is connected to D1/GPIO 5

   if (value == 1)

     digitalWrite( D1, HIGH );

   else

     digitalWrite( D1, LOW );

}

// ---------------------------------------------------------------

void setS3 (int value)

{

   // S3 is connected to D0/GPIO 16

   if (value == 1)

      digitalWrite( D0, HIGH );

   else

     digitalWrite( D0, LOW );

}

Code is intentionally verbose. Feel free to save a few bytes and collapse the functions into bit manipulation ones. Reassigning the order of the D0 to D3 pins would be helpful too.

Finally, a little bit on reading voltages from these sensors. There are a lot of examples on the 'net, but bear in mind many of these examples are for a 5V Arduino based board. The NodeMCU as I'm using it, is 3.3V. Thus the voltage calculation needs to change slightly:

float readVoltage ()

{

const float R1 = 30000.0; // Resistor 1 Ohm value on the voltage senso

const float R2 = 7500.0; // Resistor 2 Ohm value on the voltage sensor

const float adjustment = -0.6; // Voltage off when compared to                                         // an accurate voltmeter

int analogValue = analogRead( A0 ); // Read the Analog Port 0

float Vout = (analogValue * 3.3) / 1024.0; // 3.3 is because we're a 3.3V                                         // device, not 5V like an Arduino

float Vin = Vout / (R2/(R1+R2));

Vin += adjustment;

return Vin;

}

 (#38) RV Phone Home

Making Connections

Part of the problem with a Computerized and Connected RV is the connected part. The promised ubiquity of wireless connectivity is still plagued with holes. Holes in coverage. 

What good is an RV loaded with Raspberry Pis doing their automation-thing if they can’t communicate with the outside world?

That’s a rhetorical question. The answer is “Not much”, of course.

Making A Connection

There are a gazillion (or thereabouts) commercial solutions to this. You’ve undoubtedly have a solution in your home. It’s called a WiFi Router. One end of the router has a cable that plugs into “The Internet”. The other end of the router has WiFi antennas that you can connect your computer(s) to.

The RV makes things a little more interesting. First there’s rarely a way to plug a “cable” into The Internet. Moreso if you’re rolling down the road; very hard to keep that wired connection to the Internet. RV owners sometimes invest in WiFi Extenders, or WiFi Repeaters to solve this.

One end of the WiFi Repeater connects, wirelessly, to another WiFi Router that, in turn, is wired to the Internet. For example, if you’re parked next to a Starbucks, your WiFi Repeater would work something like this:

 

Bob’s your uncle – your RV (and the computers inside) are connected to the Internet.

You can buy a WiFi Repeater.

And frankly you probably should.

It’ll be less work and less reading.

However, if you’re like me, and don’t want to pop $300 for something that should be $50, then you decide to roll your own.

Rolling your Own WiFi Repeater

You might be lucky enough to have an old, unused but still functional router lying around in the house. I did. Obviously, or this blog post would end right here.

I had a Netgear N750 (Model WNDR4300) going unused. This model has a couple of things going for it that’ll make it ideal to turn it into a WiFi Repeater:

1. There are two antennas in the router – one for the 2.4GHz band and the other for 5GHz
2. This router can run the OpenWRT firmware

Google up OpenWRT if you’re curious but it’s free and open software that will replace the Netgear software and unlock the full potential of your router.

There’s NO WAY I’m going to cover off installing OpenWRT.

Nope. If you mess it up, you could brick your router.

“Bricking” is where you screw up and render the router unusable. Turning it into an expensive “brick”.

No. Figure out how to install OpenWRT on your router if you wish.

The come back here.

If you wish.

 

Getting to the WiFi Repeating Part

OpenWRT makes it easy to turn the N750 into a wireless repeater.

What’s great about the N750 are the two wireless bands: 5GHz and 2.4GHz. You can use one band to “connect to Starbucks and out the internet” and use the other band to connect the computers in the motorhome.

I’ll quickly gloss over the steps to get things all connected, but briefly:

- in the OpenWRT setup screens you select the 5GHz radio and tell it to scan for WiFi networks.

- you pick the wireless WiFi network you want to join (e.g. Starbucks), type in the password for that network and click “Join”.

- you then flip over to the 2.4GHz radio setup screens and make it into an Access Point. This is very easy, and only needs to be done once. Give it a new SSID (it’s ‘OpenWRT’) by default. Make it password protected

Finally, For each RPi in the RV, you have it join the 2.4GHz network. And test that each Rpi can access the internet.

 

In the End

We’ve reused and old router and effectively created our own WiFi Wanderer, WiFi Explorer, WiFi Arranger. You get the drift.

In a subsequent post, I’ll cover off the OpenWRT setup in more detail. But rest assured, it’s not hard.

No, no I won't.  Because...

 

 

 

 

(#37) What a long strange trip it’s been 

 

RVs, Pandemics and Time

There are upsides to a pandemic. I’m not so callous as to gloss over the horrific impacts on people, the potential personal impact. But there are upsides. To name one, we’ve spent more time together as a family, than well, I can recall. Ever. And my kids are college age – the time when I’d expect them to be out more than in.




Another upside has been more time to work on side projects. Like making a home smarter with complex event processing (CEP) technology. So while it’s been months since I’ve written anything on this topic, it’s not been time passing without progress. Au contraire mon frère, there’s been much mischief afoot! Much raspberry-pi’ing to coin a verb.

Complex Event Processing

It was probably fifteen or so years ago when I was told about CEP technology while visiting Tibco headquarters. It was nearly ten years ago when I downloaded EsperTech’s Esper product and put together my own-non trivial project.


And as of today, my quest for a smarter home that leverages the power CEP still runs. With one tiny twist: the “home” part has become “motorhome”.

Can I make my motorhome less dumb?

Maybe.

Probably.


I've killed 12 months on this already. So let’s catch up on what I’ve done in that regard, what I’ve done to instrument the RV and leverage Esper’s CEP capabilities. There’s a lot to cover; I’ll tackle it in this order:

1. RV Phone Home - Get the RV online! Reliable internet connectivity was a challenge. When the RV sits, it sits in a storage lot with one public access point accessible. Task 1 was to assemble a WiFi extender/repeater so the RV could “phone home”.
2. With “phone home” complete, task 2 was to “Home - Phone RV” - reverse the connection. Make it so I could connect to the computers in the RV from the comfort of my home office.
3. Make one RV computer a WiFi access point. All other computers in the RV would connect to this access point.
4. Go IoT with more Sensors -- Scatter useful computers and sensor about the RV to gather data about the conditions. One would monitor the Solar Charge Controller, a second would be GPS enabled for geolocation information, a third would monitor conditions (temperature, humidity) in the coach cabin, a fourth would connect via ODB2 to the engine, and so on.
5. Use IoT - MQTT to Publish Sensor Events. Have the sensors send their values to a broker inside the RV. The bridge the MQTT broker in the RV with my MQTT broker at home, so those values could make their way back to my desktop.
6. Feed those sensor values into InfluxDB, a Time-Series Database. Use Telegraf to ingest the JSON formatted events, and Chronograf to create dashboards to display sensor data.
7. Feed those sensor values as events into Esper, our Complex Event Processing engine.
8. The coup de grâce, build up a series of Esper queries to construct “Meaningful Conclusions” from the events and patterns.

So next up – Putting the RV online.

(#36) Solar Charge Controller Update   

It's Dead, Dammit! 

Sometime around the middle of 2019, I noticed that my EPSolar Landstar 1024B controller had stopped working. I'm not sure why but it was an inexpensive proof-of-concept that showed I could command the controller and get the data out of it I wanted.


That controller, the LS1024B was also a PWM based model and only 10A.  A great baby step but not a long term solution.  So, after some thought I decided to buy another EPSolar product. [ And an extended warranty. ]



August 6th 2019, I bought the EPEver Triron 3210N Controller. This is an MPPT 30A controller and should support my projected needs for the RV.



Most of the code written for the LS1024B worked on the 3210N. EPEver/EPSolar technical support sent me a new document that outlined the new protocol.  There were a few changes, a handful of deletions and two handfuls of additions.  But it didn't take long to have the code working again.

New - A EPSolar Library

I took the time to move the base code into a library (shared and static).  That seemed like a useful step.

I'm content with the library design but already see a few things I'd do differently. The 'pound-defines' (#define) are a first attempt to abstract out the dependency on knowing Modbus. 

Yes, you'd always need to link with the libmodus library, but at least you wouldn't have to know anything about the modbus context object.

Client Applications

With the library done, the next application to recreate was the application that polls the Solar Charge Controller (SCC) for status and publishes MQTT messages to the broker.

This application can also receive MQTT based commands and change parameters in the controller.

This application is written in C and runs on a Raspberry Pi Model B+ connected by a EPEver Branded RS485/USB cable to the controller.

Status messages are published to the broker every minute.

Curses! Foiled Again

The second application was cobbled together quickly and provides a simplistic, curses-based UI for displaying SCC values and changing parameters.

Home

Here's the startup, Home Screen with the essential data displayed:

 

 

At a glance are data from the Solar Panel (PV), the House Battery and any load that is connected to the MPPT Controller.

The controller has a connection for an external temperature sensor. And the controller keeps tally of the stats around generated and consumed power.

One thing I noticed after short usage was that the internal clock in the controller drifts quickly.  If this was an AC based appliance, I'd suspect a 50/60Hz issue. But it's not and I'm not sure why the controller time falls behind so quickly.

 

Battery

Pressing 'B' brings you to the Battery Screen:

Battery parameters are modified here.  These are the values fo my 100AH Group 27 Sealed Lead Acid battery.

Load

Press 'L' for the Load Screen:

 

The controller has a four useful modes for turning the connected load on and off. There's full manual mode. Then there's sunrise and sunset, timers and a combination thereoff. 

The last one is hard to put into words. The manual has a pretty good picture of how it works.

Note that Dawn/Sunrise is not defined by a time but instead by a minimum threshold voltage from the solar panel. In this case, when the panel is putting out 5.0V (or better) the controller calls that "Dawn".


And the same for Dusk/Sunset. When the panel voltage output is 6.0V or less, then that means it's dusk.


[ Yes - these are the default values which makes you wonder what happens when the panel voltage is 5.5V! ]

Finally, note that if you chose to use the 'Dusk Plus Timer' mode you need to set the "Time of Night" parameters that will define Dawn.

Device / Controller

Press 'D' to see Device (the MPPT Controller) settings:

The controller can turn itself off if it gets too hot, can stop charging if the battery gets too hot, can stop draining if the battery is too cold.

You can see that the two clocks have drifted off by over an hour after about 10 days.

In a future post we'll discuss the RS485/USB adapter in more detail and the steps I took to rebuild the driver as a kernel module.