(#16) Writing the Wrongs - Esper Patterns 

I made a mistake.  After watching my Esper code run for several weeks I was consistently seeing triggers on event patterns that I didn't expect.  It's not that the triggers were wrong -- I'm not asserting a bug in Esper -- it's that I wasn't seeing results that I expected to see.

The ABC's 

Let's simplify what I'm after. If you recall, I'm looking for a Garage Door Open event, a Door Open event and then a Motion event as a pattern that I'm interested in.  

Simplify this to event A, event B then event C, where event A is the Garage Door Open Event, event B is the Door Open event and event C is the motion event.

I'm interested in A, then B, then C.  In Esper's EPL parlance, this is noted as "A -> B -> C".  Which is useful notation to adopt.

The Real World Intrudes

What was happening, in my house, occasionally was this:

1. Garage Door Opens (e.g. A-1)
2. Garage Door closes (don't care about this event)
3. Time passes
4. Garage Door Opens again (e.g. A-2)
5. Door Opens (B-1)
6. Motion Sensor triggers (C-1)

In my psuedo EPL notation I was seeing A1 -> A2 -> B1 -> C1.

What was I expecting?  I was expecting that my Esper would trigger on the last three: A2 -> B1 -> C1.   And that's not what I was seeing.  In my update listener code, I saw that the three events that were coming in were A1, B1, and C1.  I was getting the first A event, not the last as I had thought I'd see.

The Fault, Dear Brutus

I've just finished up an hour of playing with Esper and patterns. As you'd surmise, the results I'm getting are because of the EPL I used.  So let's explore a bit more on the EPL varations I tried and the results I got.

The test cases I used were all variations on the arrivals of A, B and C events with delays in between.  Test case 3, to pick one, is noted as: "A1, A2, d10 A3, B1, C1".

In English this would be: 

• send A event (A1)
• send A event (A2)
• Delay 10 seconds
• send A event (A3)
• send B event (B1)
• send C event (C1)

I created 4 Esper patterns in EPL and ran the application.

EPL-1

SELECT * FROM PATTERN

[ every 

 (eventOne = HHBAlarmEvent( macAddress = '000000B357', deviceStatus = 'OPEN' ) 

-> eventTwo = HHBAlarmEvent( macAddress = '0000012467', deviceStatus = 'OPEN' ) 

-> eventThree = HHBAlarmEvent( macAddress = '000007AAAF', deviceStatus = 'MOTION' )) where timer:within( 2 minutes )];

EPL-1 can be noted in my shorthand as [ every ( A-> B -> C) where timer:within ]

EPL-2

SELECT * FROM PATTERN

[ (eventOne = HHBAlarmEvent( macAddress = '000000B357', deviceStatus = 'OPEN' ) 

-> eventTwo = HHBAlarmEvent( macAddress = '0000012467', deviceStatus = 'OPEN' ) 

-> eventThree = HHBAlarmEvent( macAddress = '000007AAAF', deviceStatus = 'MOTION' )) where timer:within( 2 minutes )];

EPL-2 can be noted in my shorthand as [  ( A-> B -> C) where timer:within ]

EPL-3

SELECT * FROM PATTERN

[ ( every eventOne = HHBAlarmEvent( macAddress = '000000B357', deviceStatus = 'OPEN' ) 

-> eventTwo = HHBAlarmEvent( macAddress = '0000012467', deviceStatus = 'OPEN' ) 

-> eventThree = HHBAlarmEvent( macAddress = '000007AAAF', deviceStatus = 'MOTION' )) where timer:within( 2 minutes )];

EPL-3 can be noted in my shorthand as [  (  every A-> B -> C) where timer:within ]

EPL-4

SELECT * FROM PATTERN

[ every eventOne = HHBAlarmEvent( macAddress = '000000B357', deviceStatus = 'OPEN' ) 

-> eventTwo = HHBAlarmEvent( macAddress = '0000012467', deviceStatus = 'OPEN' ) 

-> eventThree = HHBAlarmEvent( macAddress = '000007AAAF', deviceStatus = 'MOTION' ) where timer:within( 2 minutes )];

EPL-1 can be noted in my shorthand as [ every A-> B -> C where timer:within ]

You can see I'm moving the "every" and the parenthesis around and see what results I find.

The Test Cases

With the four EPL patterns ready, I modified the code to read patterns from a file, to add real-world timings. Then I created six test cases:

(Recall "d10" is my short hand for delay (wait) 10 seconds, d123 means delay 123 seconds.)

All four EPL patterns will be run against the test cases.  Next, I thought about the results that I wanted.  For example on test case 3, what I'd like to get is a trigger on the event sequence A3 -> B1 -> C1.

So I added a column to indicate what I was hoping to see from Esper:

Now, we could certainly disagree over what's to be expected or desired. Your needs / expectations could be different from mine.  For example in test case 5, you might possibly want to get notified on A1->B1-> C1 or A1->B3-> C1 or A3->B3-> C3.  What you expect is up to you.  I've just put down what I think I'd want to meet my needs.

The Results - Grouped by EPL

Test Case 1

Recall EPL-1 is [ every ( A-> B -> C) where timer:within ]. Let's look at the results:

Green denotes that I got what I was wanting to get.  Red means that I got something that I didn't want. Please, please, please note - red does not mean the results are wrong. Red means that the EPL I used didn't give me the results I was hoping for.

Think about Test Case 1, where I was hoping to get A3->B1->C1 and instead I got triggered on A1->B1->C1.  It's obviously correct but it's also reasonable to get that response from Esper.  So the chore becomes "what EPL will produce the results I'm after?"

Let me beat this horse a bit more - Esper responded. The EPL worked, my updateListerner method was called.  But when I examined the three events that triggered the listener object, sometimes I got the event objects I wanted (green) and sometimes I did not (red).

Let's keep going

Test Case 2

Recall EPL-2 removes the "every" keyword: [ ( A-> B -> C) where timer:within ]. The results were:

Again - green the results match what I expected. The red, I got something other than what I had wanted and now, yellow, means the trigger did not fire. Yellow means the updateListener method was not called.

The Yellow results on Test Case 5 puzzle me.  Yes, the time delta between A1 and C1 is outside the window. No trigger should fire.  But A3, B3 and all of the C events are within the two minute window and best I can tell, the updateListener did not get called.  That strikes me as odd.

Test Case 3

Recall EPL-3 puts the "every" keyword back but inside the parenthesis: [  (  every A-> B -> C) where timer:within ]. The results were:

Interesting,  More cases where there was no trigger fired.  Again not what I was expecting.

Test Case 4

Finally EPL-4 removes the parenthesis: [  every A-> B -> C where timer:within ]. The results were:

Conclusions

First I need to read more and get a better understanding of the EPL pattern syntax to see if my errors are obvious.  

The Community Responds!  Switch your EPL to use The Match Recognize Syntax!