(#12) ESP 

And not the whacko stuff JB Rhine wrote about - but Event Stream Processing. From now on, when I type ESP, I mean Event Stream Processing and not Extrasensory Perception.

I've been taking a long, leisurely path to this point - a good home automation system should make inferences. An inference is a conclusion based on data and reasoning.  So let me be direct - a good home automation system should make inferences.

There.


Oh hell, one more time: a good home automation system should make inferences. That's Point #1.

An inference is a conclusion ("some one is home!") based on data ("the garage door opened, then the house door opened and closed -- all within a few minutes.").

Point #2, we've talked about how the rules that drive the conclusion could change over time. So they shouldn't be hard-coded as some sort of Finite State Machine in the application.

Point #3, tools exist that allow you to draw conclusions based on the appearance of data (in my case, specifically, MQTT event message). 

And these tools allow you to express the event sequence easily.  
At run time.

I know, right?
It's almost as cool as when Smuckers put peanut butter and jelly in the same jar!

Esper estas programaro por okazaĵo rivereto prilaborado.
(I slay myself...)  Esper is an open source, Java based, Event Stream Processor.  Event Stream Processing is a subset/superset/cousin/sibling of Complex Event Processing which we introduced earlier.

You can learn more about Esper from their website: http://www.espertech.com/esper/

Pulled right from their homepage:

Drawing Conclusions in 3, no 5, Easy StePs!

For you Emergenetics Greenies, here's the recipe

1. Have your Home Automation Sensors publish events on the network. I'm using MQTT
2. Download and install Esper (You can code Java, right?)
3. Write Java code
1. You'll need a main() that instantiates and runs the Esper Engine
2. You'll need a POJO that represents your Events
3. Write code to subscribe the the MQTT event stream
4. Write a handler object, that you want triggered when Esper makes an inference
5. Write the EPL that represents the event sequence you're after
6. Tell Esper to associate the handler code and the EPL statement
4. Run
5. Debug, lather rinse repeat

See? Five steps.

Esper's a cool tool.  And it comes with (as of this blogging) 733 pages of documentation. Esper's the brain child of a genius, and unfortunately, the documentation assumes you're a genius too.

There are few of the obligatory Hello World examples out there.  Esper comes with plenty of examples, but they're source files.  What mere mortals, like you and me, need are pretty simplistic step-by-step examples.

That's what I'll do next. I'll post my code.
Oh, I use NetBeans. So you JetBrains and Eclipse users will just have to suffer along.

(#11) S.P.O.F. 

Here’s a real life example of how a Home Automation system failure can really stink. Let’s say your carefully crafted Home Automation system is humming away as you leave the house on an extended vacation.  And let’s say you’re heading into the remote wilderness of Southern Utah.

Well, the destination is immaterial, but let’s say you’re heading far, far away from home. 

Oh yes – you've got wireless connectivity. Our example doesn't work if you can’t connect to the ‘net.

For the first four days of your vacation, you check your Home Automation Status page and see that your system is working well.  The doors are all reporting closed, the water leak sensors say they’re dry and the motion sensors see nothing.

Now – the fun part. On day five, at 9:32 pm, your system tells you that the big garage door has just opened. Whoa!  You’re not home, you’re sure of that! And you’re pretty sure no one else has good reason to open the door.

Quick – check the Home Automation Status page.  Yes – the garage door is reporting that it’s Open. It’s saying the Battery is OK and that the sensor is reporting its On-line and functioning.  Check the other sensors: have they entered into the house?  No – the door sensors are closed and have been closed. The motion sensor says No Motion.  Are they rummaging around in the garage?  Maybe the rat-bastards are after my TwentyYearOld, WillNotDie John Deere Weed Whacker!  Or maybe they’re going for the Pizza Hot Pocket snacks in the freezer out there.

Recheck the other sensors – the doors are still showing closed and there’s still no motion.

False Alarm?

But why now?  

The garage door sensor has been working well for weeks.  

Why now?  

Why, at 9:32 pm when I’m 1400 miles from home?

Reliable, Dependable and Trustworthy!

So let’s switch gears and talk about reliability.

Reliability engineering is something I know nothing about. So, like movie critics, I’m allowed to pontificate and make wild assumptions.  I know it’s hard.  Reliability engineering, that is. It’s got to be hard – or Three Mile Island wouldn't have had a problem; Chernobyl wouldn't have blown up; the Therac-25 wouldn't have killed people.

• How do you make a system reliable?  
• How do you make it reliable and still affordable?  

Add a second sensor? No – there’s that old proverb: “Man with two watches never knows what time it is.” Add a third sensor?  I've read that’s what they do in Avionics, reliability through redundancy.

Wait – reliability is not at the heart of this system. It’s something else. It’s trust.

In this case, what we've lost is trust. I can’t trust my system anymore.  

Oh -- it works, mostly. 

It works usually. 

And you know what?  

That was the same problem we had with X10 technology.  It worked, usually.

Avionic systems also have enough sensors it’s possible to make deductions on whether a sensor has failed. If one sensor reports something that might claim the plane is flying upside down, yet all of the other sensors indicate normal flight, odds are that we have a sensor failure.

In my house, I can make some deductions.  If an interior door suddenly opens, yet the nearby motion sensor indicates stillness and the exterior doors have remained closed – then odds are it’s a false alarm.  

But not with the garage door sensor.  It’s perfectly reasonable for the door to open at 9:32pm.  Just not when we’re all on vacation.

• Trust.
• Trust is earned.
• Trust is earned over time.  

Once lost, it’ll take a while to earn it back.  We’ll be home from vacation in about two days.  I’ll debug the garage door sensor.  Hopefully the failure will be obvious. Maybe the sensor fell off the door.

Two Days Later -- Epilog

That was it – mechanical failure. The Velcro that held the sensor to the door gave way, the sensor fell off the door and landed in a tilted (open) state.  One #4 wood screw and we’re back in business.

For want of a nail, err, screw...
To borrow the Wisconsin State Motto: foreward!

(#10) Raising the Alarm 

If you're a glutton for punishment, or you're doing graduate work in the space, or if you're both a glutton and a grad student, you'll eventually stumble across the concept of event types.  Just Google "event stream vs event cloud" and start reading about event types.

But that's not the point of this post. I've got two types of events coming off the various publishers scattered around the house.  And it comes from the sense that I'll have "status" events and "alarm" events.  And they'll be easily winnowed (or 'sifted') from the event cloud/stream.  

Because I'll want to react to Alarm Events differently from Status Events.

"A subtle thought that is in error..."
The distinction between the two, an alarm and a status event, can be conspicuous or subtle.

1. "The Door just opened."
2. "The Door is still open."
3. "The Door is still open and has been open for two minutes."

By my sophistic thinking, the first one is an Alarm.  The second one a Status Event.

The third - "The door is still open and has been open for two minutes" - could be either.  It depends on whether you consider a Two-Minute-Left-Open Door in your house worrisome or not.

(One of) my point(s) being that whether something is an Alarm or a Status will depend. Depend on a few things.  A few things that will make sense to you and your house and your situation. And your few things that will probably not match my few things.

So?

So don't code this.  This needs to be flexible, changeable, adaptable.

A door open for two minutes in winter's cold December - is an alarm.  A door open for two minutes in Summer might just mean your hands are full taking burgers to the grill.

Wait, no - a door open for two minutes in December in Australia is...


Well, you get the gist.  As programmers we make decisions, lots of them, based on our context. And that leads to inflexibility, complexity and a lack of adoption.


Details, Boy, Details
Ok - so here's an example of two ALARM Events in my system:

WS2308/ALARM | 2014-05-08 20:29:00 -0600 | OUTDOOR HUMIDITY HIGH | 89.0 |

WS2308/ALARM/OUTDOOR/PRESSURE | 2014-05-08 20:29:04 MDT | RAPID OUTDOOR PRESSURE DROP | 0.07 |

Take note of my current, but may change, convention when I publish events

• The MQTT topic is part of the message payload
• The MQTT topic starts with the system that created the event -- WS2308 is the name of my weather station
• The MQTT subtopic is ALARM in this case
• Vertical bars separate the fields
• Information is human readable for debugging, instead of in XML or JSON
• I'm still inconsistent in my MQTT Topic usage. The first alarm could also have been published on the "WS2308/ALARM/OUTDOOR/HUMIDITY" topic.

My weather station publishes weather STATUS events about twice a minute. But it only publishes ALARM events when a threshold has been reached or exceeded.

There's one more thing that I do by convention for my ALARM events, and that is they are published ONCE.

Fire and Forget, Spray and Pray
That's right - my ALARM events are published once.  When a door opens in my house, you'll get one and only one of these:


HHB/ALARM | 2014-05-09 16:19:57 -0600 | 03 | OPEN-CLOSE SENSOR | Front Door | OPEN | 1 | 0000010228 |

I think that's what you want.  I don't think you want your event generator solving for network reliability issues.

And speaking of reliability issues...