Wednesday, June 13, 2018

(#30) Solar Charge Controller Communication


The last post covered off on the hardware and computer connection for creating a communication link with the EPSolar LandStar LS1024B controller.

Here we'll briefly cover off on the basic software approach to communicating with the controller.

As mentioned last time, the communication protocol used by the controller is an archaic one called Modbus.

With our USB/RS485 Adapter connected and plugged into a Linux machine we can start coding up the software.

There are Python libraries for modbus, but I chose to use a 'C' library for initial exploration.  I used libmodbus V3.1.4 from libmodbus.org on Ubuntu 16.04 LTS.

[ November 2018 Update: complete code for my solution is available in my GitHub Repository ]


Modbus and Libmodbus

Fortunately the API exposed by libmodbus is small; there's not a lot to master. And the documentation is good. Unfortunately, the terminology is modbus specific and I've already taken the stance that I have no desire to master an interface protocol for industrial devices from the late 70's.

So rather than dig into why a register isn't a coil and why there are three or four ways to actually read data, it was easier to just throw some code and it and see what works.


The C code


Right off the bat, include the right header files:

#include <errno.h>
#include <modbus/modbus.h>


#define LANDSTAR_1024B_ID       0x01

Then try to open the port to the controller. The settings are 15200 Baud, 8N1. My USB/RS485 adapter shows up as /dev/ttyUSB0:

int main (int argc, char* argv[]) 
{
    modbus_t    *ctx;
    
    puts( "Opening ttyUSB0, 115200 8N1" );
    ctx = modbus_new_rtu( "/dev/ttyUSB0", 115200, 'N', 8, 1 );
    if (ctx == NULL) {
        fprintf( stderr, "Error opening\n" );
        return -1;
    }

    printf( "Setting slave ID to %X\n", LANDSTAR_1024B_ID );

    modbus_set_slave( ctx, LANDSTAR_1024B_ID );
    <...>

If the port opens successfully, then try to connect:

    puts( "Connecting" );
    if (modbus_connect( ctx ) == -1) {
        fprintf( stderr, "Connection failed: %s\n", 
                 modbus_strerror( errno ) );
        modbus_free( ctx );
        return -1;

    }

Once the port is open, you need to consult the vendor documentation to see what 'modbus registers' hold the data we're interested in. 


Here's a snippet that shows how to pull what EPSolar calls the "real time" data:

static 
void    getRealTimeData (modbus_t *ctx)
{
    int         registerAddress = 0x3100;

    int         numBytes = 0x13;  // 0x14 and up gives 
                                  // 'illegal data address'
    uint16_t    buffer[ 32 ];

    // zero everything out to start!    
    memset( buffer, '\0', sizeof buffer );
    
    // call 'read-input-registers', other read calls error
    if (modbus_read_input_registers( ctx, 
            registerAddress, numBytes, buffer ) == -1) {
        fprintf(stderr, "Read failed: %s\n", 
                modbus_strerror( errno ));
        return;
    }
    
    // ---------------------------------------------
    //  Photo Voltaic Values - Volts, Amps and Watts
    float pvArrayVoltage =  ((float) buffer[ 0x00 ]) / 100.0;
    float pvArrayCurrent =  ((float) buffer[ 0x01 ]) / 100.0;
    
    //
    // Assemble the Power (watts) value from two words
    long    temp = buffer[ 0x03 ] << 16;
    temp |= buffer[ 0x02 ];
    float pvArrayPower   =  (float) temp / 100.0;
    <..snip..>


A couple of notes...
There are a handful of 'read' calls in the modbus library. Again, rather that learn modbus details, I just whacked at the code until one of the calls worked.

But the 'read-input-registers' call did not always work. More on this later.




Note the 'register-address' variable that's set to 0x3100. Again, this value comes from the vendor documentation.  And the amount of data read, numBytes, can vary from 1 to MAX (where MAX seems to depend). 

This number of bytes values sometimes fell short of what the documentation said I could expect. For example, there are three additional fields documented:

float batterySOC = ((float) buffer[ 0x1A ]) / 100.0;
float remoteBatteryTemp = ((float) buffer[ 0x1B ]) / 100.0;

float systemRatedVoltage = ((float) buffer[ 0x1D ]) / 100.0;

But it didn't work for my controller - setting numBytes higher that 0x13 gave an error.


I mentioned that 'read-input-registers' didn't always work. For the Settings stored at 0x9000, I had to use the 'read-registers' call.  Looking at little into the Modbus specifics, I can see that one call uses Modbus function call 0x04 and the other uses 0x03.   It's most likely because the settings at 0x9000 are both read and write. 

At some point in the future, I'll try the function write-registers (modbus function 0x10) to see if I can update the settings. 

(#29) And now for something completely different


Tinkering on the Home Automation front has slowed. I've got a number of projects in various states of completion: IKEA Tradfri lighting, ODB2 and CANBUS monitoring, GPS work, etc.  And Solar.  



Since we bought the motorhome, my interest in solar charging of 12V systems has been growing.  A few months ago, I purchased a 100W solar panel, and a simple 10A solar charge controller to get started.

While this approach taught me a few things about solar charging systems at a low entry cost, the charge controller was dumb.  More sophisticated charge controllers, with communication capabilities are out there.

Given my interest in IoT, a smarter controller was a necessity!


The Solar Charge Controller

Beijing Epsolar Technology Co is a popular manufacturer of affordable solar charging products. Their LandStar series of PWM Charge Controllers met my needs of being affordable and having a communication interface. 



Twenty dollars and three weeks later, the LS1024B PWM Solar Charge Controller from EPSolar arrived.



Other models from this vendor used a serial interface. This one uses a protocol called "modbus".  I knew nothing of this protocol before the controller arrived and I still know precious little. And I have little desire to master an archaic protocol, so take this next section with a huge 'grain of salt'.

Apparently you can use the Modbus protocol over a variety of interface specifications, but the LandStar uses Modbus RTU over RS485.  While I have no idea what that really means, I was able to Google up a number of matches to get something cobbled up that works.

USB / RS485 Adapter

The first purchase was a USB/Modbus dongle.  There are dozens available on eBay, Aliexpress, etc. I spent $3.50 for one that looks like this and when plugged into Linux, it registers as "ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter":

Plug the adapter into your Linux machine, check 'dmesg' to ensure the device is recognized.  And you should have a "/dev/ttyUSBx" device appear.

[ ] usb 1-2: new full-speed USB device number 3 using ohci-pci
[ ] usb 1-2: New USB device found, idVendor=1a86, idProduct=7523
[ ] usb 1-2: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[ ] usb 1-2: Product: USB2.0-Serial
[ ] usbcore: registered new interface driver usbserial
[ ] usbcore: registered new interface driver usbserial_generic
[ ] usbserial: USB Serial support registered for generic
[ ] usbcore: registered new interface driver ch341
[ ] usbserial: USB Serial support registered for ch341-uart
[ ] ch341 1-2:1.0: ch341-uart converter detected

[ ] usb 1-2: ch341-uart converter now attached to ttyUSB0


Wiring it all up

Wiring of the device is simple.  Grab any old Ethernet cable (the connection on the LandStar controller is RJ45.)  Cut one of the RJ45s from the cable, strip the wires.  Find the wires that connect to pins 3 and 5 - and trim the insulation.



Wire 5 goes the the connection marked "A" on the adapter. Wire 3 goes to the connection marked "B".  Use a multimeter to ensure continuity and a good connection.  Then plug the adapter back into the Linux machine.

Plug the other end of the Ethernet cable into the COM port on the controller.  Connect the 12V battery to the charge controller and the LandStart will power up.



[ Ignore the DC-DC buck converter in the photo as I use this setup to power a Raspberry Pi. ]


With the hardware wired and connected, we can move on to the software. Which is discussed in the next post!






Thursday, August 10, 2017

(#28) That doggone total eclipse is gonna mess me up


It just dawned on me (pun soon to be intended) that the looming Total Eclipse is going to break one of my more useful web services for about an hour.  That web service is my "is it dark outside?" web service.


Sunrise, sunset...

In Home Automation, it's handy to know when the sun rises, when it sets and where we are at any given time between those two boundaries.  Here's my implementation in PHP:

<?php
 $aLatitude = htmlspecialchars($_GET["lat"]);
 $aLongitude = htmlspecialchars($_GET["lon"]);
 $aTimeZone = htmlspecialchars( (isset($_GET['tz'])) ? $_GET['tz'] : 'America/Denver' );

 //
 // uncomment one zenith setting: http://en.wikipedia.org/wiki/Twilight
 $zenith = 90+(50/60); // True sunrise/sunset
 //$zenith = 96;           // Civilian Twilight - Conventionally used to signify twilight
 //$zenith = 102;         // Nautical Twilight - the point at which the horizon stops being visible at sea.
 //$zenith = 108;         // Astronomical Twilight - the point when Sun stops being a source of any illumination.

 //
 // Good enough approximation at offset
 $tzOffset = $aLongitude * 24 / 360;

 //
 // But evidently, I need to deal with Daylight Saving Time myself
 if (date( 'I' )) {
    $tzOffset += 1;
 }


 $sunriseStr = date_sunrise( time( date_default_timezone_set( $aTimeZone )), SUNFUNCS_RET_STRING, floatval($aLatitude), floatval($aLongitude), $zenith, $tzOffset );
 $sunsetStr  = date_sunset(  time( date_default_timezone_set( $aTimeZone )), SUNFUNCS_RET_STRING, floatval($aLatitude), floatval($aLongitude), $zenith, $tzOffset );

 $isDark = FALSE;

 $date1 = DateTime::createFromFormat( 'H:i', date( 'H:i' ));
 $date2 = DateTime::createFromFormat('H:i', $sunriseStr );
 $date3 = DateTime::createFromFormat('H:i', $sunsetStr );

 if ($date1 < $date2 || $date1 > $date3)
 {
    $isDark = TRUE;
 }
 
 $result_json = array( 'sunrise' => $sunriseStr,
                       'sunset' => $sunsetStr, 
                       'currentDateTime' => $date1, 
                       'timezone' => $aTimeZone,
                       'inDST' => (date( 'I' ) ? '1' : '0'), 
                       'isDark' => ($isDark ? '1' : '0') );

 // headers to tell that result is JSON
 header('Content-type: application/json');

 // send the result now
 echo json_encode($result_json);
?>


It fits my needs. Pass in a Lat/Lon for the sunrise and sunset times at that location. Leave them off and it assumes you live near me.

It also returns the current date, time and timezone data for, you guessed it, me.




Invocation is pretty simple. Here's today's sunrise/sunset request for Seattle:

http://<yourwebserver:port>/<path>/sunrise.php?lat=47.6lon=-122.3

The result comes back as JSON:

 
   "sunrise":"05:48",
   "sunset":"20:21",
   "currentDateTime": 
      "date":"2017-08-10 09:29:00.000000",
      "timezone_type":3,
      "timezone":"America\/Denver"
   },
   "timezone":"America\/Pacfic",
   "inDST":"1",
   "isDark":"0"
}

Again - that currentDateTime object is for me.  Change it or remove it to fit your needs.  Once this webservice is in place, having other applications check to see whether it's dark outside or not is also pretty easy. A contrived python example:

import json
import urllib2

# My website needs authentication
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
use it instead of None.
top_level_url = "http://<yourserver>/<path>/sunrise.php?lat=40&lon=-105"
password_mgr.add_password(None, top_level_url, <usr>, <passwd>)

handler = urllib2.HTTPDigestAuthHandler(password_mgr)
opener = urllib2.build_opener(handler)
urllib2.install_opener(opener)

data = urllib2.urlopen(top_level_url)
json_payload = json.load(data)
isdark = json_payload['isDark']

# do something
if isdark == "1":
   print "It is dark - turn on the outside lights"
else:
   print "It is light (not dark) - turn off the outside lights"





Wednesday, August 9, 2017

GOSUB 200 - Another Interruption - Daemons and Python


Plunking "instructions to myself" in a blog is useful, so here it comes again. This time it's about Python, Daemons and "systemd".

For code that I want to run all of the time and have it managed by the operating system there are a handful of approaches: write it as a daemon, write it as a executable and work up some cron scripts to keep it going, and now there's systemd.  Like it, revile it -- it looks like systemd is here to stay.

So, briefly, here's what I did to make my Python code managed by the systemd service.

It starts with a 'unit' file.  Bear in mind I have precious little information on what all of the options are, what the ones I've used actually do and the ramifications of the choices made.  

But here's what's working for me.

Step 1 - Create <useful-name>.service

$ more garagedoorcontroller.service
[Unit]
Description=Garage Door Controller Service
After=syslog.target

[Service]
Type=simple
User=myname
Group=mygroup
WorkingDirectory=/complete/path/GarageDoorController
ExecStart=/usr/bin/python /complete/path/GarageDoorController/GDController.py
StandardOutput=syslog
StandardError=syslog
Restart=always
RestartSec=60

[Install]
WantedBy=default.target

#
# After this the steps are:
# Create the unit file in /etc/systemd/system (the file name
# before .service will be the name you call to start/stop/restart the service)
# Set file permissions to 644
# Tell systemd that there is a new unit file: systemctl daemon-reload
# Start service: systemctl start <service>
# Check the service status: systemctl status <service>


Steps 2 and 3 - Copy it to the right spot


foo: $ sudo cp garagedoorcontroller.service /etc/systemd/system/.
foo: $ sudo chmod 644 /etc/systemd/system/garagedoorcontroller.service

Step 4 - Tell systemd that there's a new (or altered file)


foo: $ sudo systemctl daemon-reload


Step 5 and 6 - Start it, check the status


foo: $ sudo systemctl start garagedoorcontroller
foo: $ sudo systemctl status garagedoorcontroller

garagedoorcontroller.service - Garage Door Controller Service
   Loaded: loaded (/etc/systemd/system/garagedoorcontroller.service; disabled)
   Active: active (running) since Wed 2017-08-09 09:35:17 MDT; 18min ago
 Main PID: 12961 (python)
   CGroup: /system.slice/garagedoorcontroller.service
           └─12961 /usr/bin/python /complete/path/GarageDoorController/GDController.py

Aug 09 09:35:17 rpi4 systemd[1]: Started Garage Door Controller Service.



Tail /var/log/syslog if you have problems. The error messages are descriptive.



The BSD daemon, also called Beastie (a near homophone of the letters B-S-D pronounced slurred together), as drawn by John Lasseter. His widely known and popular take on the BSD mascot first showed up on a book cover in 1988. 

[ From the Wikipedia entry on BSD Daemon. ]

Monday, August 7, 2017

(#27) - Home Automation, Lights and Normal People


Normal

Here's how normal people turn on the lights in the kitchen:

Not Normal

And here's how people enamored with Home Automation turn on the lights in the kitchen:

  1. Fumble around in the dark, pull smartphone from pocket
  2. Swipe around until Home Automation App found, launch app
  3. Wait while Home Automation app starts
  4. Swipe around and find the Lighting Tab on the Home Automation app
  5. Swipe around and find the room labeled Kitchen
  6. Press the On button



And the industry wonders why adoption just won't take...

Still, that's far too simple and straight-forward for me, so continuing our TCP Connected Lighting Gateway programming from the last post, lets cover two of the four commands we use.

DeviceSendCommand


DeviceSendCommandLevel


To control individual bulbs, you use one of these two commands. The first turns a bulb on and off. The second command will set the dimming level.

#------------------------------------------------------------
def set_light_value(self, did, value=1, update=True):
    rc = self.my_gateway.send_device_command(did, value)
    if update:
        self.load_rooms_and_devices()
    return rc

#-----------------------------------------------------------
def set_light_level(self, did, level=100, update=True):
    rc = self.my_gateway.send_device_command_level(did,level)
    if update:
        self.load_rooms_and_devices()
    return rc



And then the HTTPS calls to the gateway are:

















The two strings, DeviceSendCommand and DeviceSendLevelCommand were shown back here.


The last two commands of the four affect an entire room and they're used in the same manner:

RoomSendCommand

RoomSendCommandLevel

#------------------------------------------------------------
def set_room_value(self, rid, value=1, update=True):
    rc = self.my_gateway.send_room_command(rid,value)
    if update:
        self.load_rooms_and_devices()
    return rc

#------------------------------------------------------------
def set_room_level(self, rid, level=50,update=True):
    rc = self.my_gateway.send_room_command_level(rid,level)
    if update:
        self.load_rooms_and_devices()
    return rc

No surprise here, the HTTPS calls to the gateway are:

















The "update" variable is used to potentially postpone a call to reload the room and bulb status values. That call, the RoomGetCarousel call is a bit 'thick' and I wanted to avoid calling it if I'm doing multiple bulb updates,

(#26) - Let there be light! 



Insufficient coffee to fuel any witticisms or snarky remarks means I'll cut to the chase. The lighting controller in my house is from a company called TCP Connected. I've discussed the challenges I've had with their product back here.  


Tuesday, June 20, 2017

(#25) My IoT Garage Door Closer

Problem with a 'p'

There's a big problem with leaving your garage door open at night.

And the problem is that the thieves never take the stuff you want them to take. 

Do they grab the box of outdated college text books from 1985? 
No. 

Do they grab the mostly-empty can of Oldham Tudor Earth Tone brown paint you bought in 1995 when that color was in vogue? 
No.