Jump to content

Welcome to Smart Home Forum by FIBARO

Dear Guest,

 

as you can notice parts of Smart Home Forum by FIBARO is not available for you. You have to register in order to view all content and post in our community. Don't worry! Registration is a simple free process that requires minimal information for you to sign up. Become a part of of Smart Home Forum by FIBARO by creating an account.

 

As a member you can:

  •     Start new topics and reply to others
  •     Follow topics and users to get email updates
  •     Get your own profile page and make new friends
  •     Send personal messages
  •     ... and learn a lot about our system!

 

Regards,

Smart Home Forum by FIBARO Team


  • 0

EventProxy


jgab

Question

-There is always this issue with device IDs needed to be declared in the header. Then a device is reincluded and gets a new id and code breaks.

-Then there is the issue of an integer being a bad way to refer to devices when coding and thus we build hometables or declare variables.

 

This code is an example of trying to solve both issues. It works but is more of a concept for the moment as some more checks and more type of triggers need to be supported. (It supports value, power,sceneActivity and simple CentralSceneEvent for now)

 

Anyway, we have a scene that builds a "hometable", lets call it the "HomeTableBuilder". It builds the table from meta data stored in the 'Device description' field for devices (not for virtual devices, they don't have a description field and they don't trigger scenes anyway).

 

The format is very flexible to specify where in the home table 'tree' the deviceId should be inserted..

Please login or register to see this attachment.

'Device description' = ["dev.%room.tableLamp.%id"]

will set HomeTable.dev.kitchen.tableLamp=417, if the device is in room 'kitchen' and has id 417

'Device description' = ["dev.%section.%room.tableLamp.%id"]

will set HomeTable.dev.upstairs.kitchen.tableLamp=417, if the device is in section 'upstairs' and room 'kitchen' and has id 417

'Device description' = ["dev.remote.%id"]

will set HomeTable.dev.remote=362, if the device is a fibaro remote with id 362

The code in the HomeTableBuilder that needs to be setup is:

Please login or register to see this code.

So this take care of creating a hometable (HT in the code) from meta data stored in the devices themselves. We can then add more static configuration data to the hometable etc.

After that we declare which of our other scenes should receive events. (Test1 has id 313, and Test2 has id 316 in this example). sceneId(string) is a convenience function to get the ID from a scene name.

Please login or register to see this code.

This says that we want to receive property.value events from "tablelamp" and send it to the scenes named "Test1" and scene named "Test2"

We declare the same for "power"

And our fibaro remote should send its events to "Test1" and "Test2" too.

 

The HomeTableBuilder then dynamically creates another scene in the HC2 named "EventProxy"..softwareVersion. (it's always created hidden for me!?, do "show hidden")

with the following code:

Please login or register to see this code.

This "EventProxy" scene receive triggers for the devices declared and send them to the scene we declared having interest in these events, Test1 and Test2 in this case.

The code in a receiving scene (like Test1) then looks like below. (No declared triggers in the header, it gets all triggers from startScene from the EventProxy)

Please login or register to see this code.

Scenes should still declare globals, weather etc... triggers that are not connected to devices IDs.

 

So, what have we achieved? 

-We don't have to deal with numerical device IDs in our code

-We can add meta data to devices that tells where in the HomeTable they should be inserted

-We don't have to declare numerical IDs in headers

 

Drawback: A small performance penalty to let the EventProxy trigger other scenes instead of they being triggered directly. (Seems to be very small penalty though, logs are always on the same second in EventProxy and receiving scenes for me). One could generate different EventProxy scenes for different triggers to distribute the load a bit and ensure that the EventProxy doesn't hit the maximum 10 active instances... However, an activated EventProxy instance is very quick to dispatch the event and then terminate.

 

Improvement: Let scenes dynamically register which events they want when they start up. Only issue is how often the EventProxy need to be rebuilt...

 

Code for the complete HomeTableBuilder:

Please login or register to see this code.

 

Edited by jgab
  • Like 1
  • Thanks 1
Link to comment
Share on other sites

Recommended Posts

  • 0
  • Inquirer
  • Some notes on setting up and running the EventRunner framework...

     

    First off I really recommend setting up an IDE off-line on a PC/Mac to test and debug scenes created with the framework before deploying them on the HC2.

    • I recommend ZeroBrane Studio. It's free and cross platform. I guess that any other would work.
    • Download and install [Lualibs](https://forum.fibaro.com/topic/24319-tutorial-zerobrane-usage-lua-coding/) with Fibaro/JSON APIs. The link also have a tutorial how to setup ZeroBrane.
    • Please login or register to see this link.

      and put them in your working directory
    • Fire up the IDE and load 'EventRunner.lua' and run it. It is setup to run test examples from the 'example_rules.lua' file. Play around with enabling other scenarios in that file.
    • ...and tell me about all the bugs you find :-) 
       

    There are a few variables in the 'EventRunner.lua' that affect the execution.

    • _sceneName. Set to the name of the script/scene. Only used for debug output when scene starts.
    • _debugLevel. 1=sparse output, 4=log everything
    •  _deviceTable. Name of the fibaro global holding your json device table. Only used by your own code to read in data.
       

    In 'EventRunnerDebug.lua' there are some more important variables. Note that EventRunnerDebug.lua is only included if running off-line as HC2 doesn't support 'dofile'. However, this is the point as the file only contain code for off-line running. Like 'setTimeout', 'net.HTTPClient' etc, and implementations of fibaro:* functions.

    • _REMOTE. If set to true will use FibaroSceneAPI to call functions on HC2, else emulate them locally...
    • _MEM. If true adds memory usage info to log output.
    • _speedtime. If nil will run the clock in normal speed. If set to number of hours will speed through that many hours. Very convenient for debugging schedulers running over days... Ex. 24*7 will speed through 1 week.
    • __fibaroSceneId. Set to scene ID. On the HC2 this variable is defined to the current scene's ID
    • hc2_user. HC2 credentials, used for api.x/FibaroSceneAPI calls
    • hc2_pwd. HC2 credentials, used for api.x/FibaroSceneAPI calls
    • hc2_ip. ip address to HC2, used for api.x/FibaroSceneAPI calls

     

    When the scene is fully debugged and you are happy with how it behaves off-line, move the 'EventRunner.lua' code to the HC2, and test it again. 

     

    A typical scene will have the below structure (using the basic Event.* API)
     

    Please login or register to see this code.

     

    Some implementation notes...

    The reason for developing this framework is to try to find an abstraction model for programming home automation systems, Fibaro's HC2 in this specific case. This means dealing with sensors and actuators and time, and be able to describe logic involving those elements. This has driven a few design choices:

    • An underlying event based programming model, could also be thought of as a message based model. The home automation environment is highly asynchronous. Sensors and switches goes on and off at will. The scene model chosen by Fibaro tries to help by spawning new instances of scenes for every new sensor and switch trigger happening. This allow a scene to not block other triggers, but it makes it difficult to synchronise logic across many triggers. Using an event/message based model for scenes makes it much more natural to deal and handle with the inherent asynchronicity.
    • A mapping of the event model to a script language based on 'rules'. Rules comes in 3 flavours, rules that fire at specific times during the day, rules that triggers on state changes from sensors and actuators, and rules that trigger on any type of (user) defined event. The last flavour allows for mixing the event model with the rule model in scenes.
    • A script language with a semantic/syntax (abstraction level) that makes it easy to deal with time and events. Time has its own data type, e.g. "10:40+01:20 == 12:00", and to check if a rule is triggered within a time interval the '..' operator can be used e.g. "11:00..12:00". Events have its own syntax, e.g. "#event1 => post(#event2)". Devices' properties are accessed (and set) with the ':' operator. e.g. "55:on", "lamp:isOn", "lamp:value=99". The ':' operator also handles operations on sets of devices. e.g. "{55, lamp}:on". etc...

    The framework implements a model where a "main" scene always run and looks at an "mailbox" (global fibaro variable) if there are any new incoming fibaro triggers (sensors actuators etc). If so they are sent to defined event handlers and/or rules in the framework. Events posted within the framework is handled within the framework and don't need the mailbox. A new incoming extern/fibaro trigger will spawn a new instance of the framework scene, and that instance will quickly post that trigger to the mailbox and quit. This allows the main scene to fetch the trigger and hand it to event handlers and rules. The BIG advantage here is that all triggers will be handled within the same scene instance (main). This allows handlers and rules to "remember" state between invocations. One way to think about the framework is that it converts triggered scene instances to timer instances within a single scene (main).

    There are some trickery to do synchronous mailbox handling, and even though Fibaro/Lua don't have primitives to handle that the chosen implementation have empirically proven to work...

     

    High level framework logic in the picture below

    Please login or register to see this attachment.

     

    The script rules are compiled to a simple stack based 'byte-code'. The advantage is that the byte-code loop is linear/flat allowing the execution of the code to be paused at anytime (code is just an array of instructions). A traditional recursive code interpreter over some tree-structure is tricker to suspend. This is used by script functions like 'wait' and 'for' to suspend the code execution and setup a timer to be able to later resume executing the code where it left off. The interpretation of the byte-code in general is surprisingly fast.

    "trace(true); b=3;a=8*b+10" translates to

    Please login or register to see this code.

    and when run produces the following trace (the tracecommand turns on instruction tracing, so the first instruction that is traced is the 'pop' of the result from trace)

    Please login or register to see this code.

    It does some simple compile-time optimisations, like folding constants arithmetic expression.

     

    There are still a lot of improvements to do, one such area is "error handling" and better "error messages" when there are compilation/runtime errors...

    Edited by jgab
    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Some simple use-cases to further demonstrate the syntax.

    Running actions on specific times

    Please login or register to see this code.

    Running actions on triggers

    Please login or register to see this code.

    Action when conditions have been true for a period of time

    Please login or register to see this code.

    Including Lua functions in scripts

    Please login or register to see this code.

     

    Edited by jgab
    Link to comment
    Share on other sites

    • 0

    Jan

     

    I have been meaning to explore this ever since seeing your first posts.   So today following through your setup advice here on and GitHub to enable ZeroBrane offline testing.   Stumbling through it ... are my solutions on the right track to get this up and running to explore your intriguing event framework concept?

     

    obstacle #1

    Please login or register to see this code.

    Digging around looks like need to set ZeroBrane to use LUA 5.2 intepreter (setting through project menu for Lua Interpreter)  ... that cleared the _ENV issue.

     

    obstacle #2

    Please login or register to see this code.

    ah ha.  conf is related to the Hometable variable - your instructions did describe this and - although running only on PC - I think this is setup correctly in event runner source (below).  

     

    Please login or register to see this code.

    feels like I missing something or some information to explore your example rules?    I am guessing your definition of hometable?    Have I missed a step in the setup?

     

    Otherwise this looks so powerful ... I will go for a start from scratch on the rules based on my own setup but perhaps this will help others yet to explore your EventRUnner concept.  Many thanks!

     

    all - I did not spot the very helpful wiki that Jan has set up on Github initially - this brings the threads together well ... I hope! 

     

     

    Please login or register to see this link.

    Edited by ipsofacto
    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Oh gosh! a user! :)

    It would be really great if you would manage to get the examples to run.

    Ok, obstacle #1. Mea culpa, I hadn't committed the last change where I define _ENV if its not defined (in EventRunnerDebug.lua). I try to stay 5.2/5.3 neutral. If you pull the last one that will work.

    obstacle #2. also a bit sloppy of me. I have a file "devicemap.data" that contains a json encoded hometable (it's in the GitHub). Download that file to the working directory and the examples should work.

    When the framework starts up (in ZeroBrane) it reads in that file and sets the fibaro global _deviceTable to that content, so the rules can do a fibaro:getGlobal(_deviceTable).  Replace it with your own home table value if you like. However, the 'example_rules.lua' has dependencies on the predefined values I have.

     Now things should work, and in the beginning of examples_rules.lua there are variables enabling the various scenarios., and tRules should be enabled by default.

    local tExpr = false
    local tRules = true
    local tShell = false
    local tGEA = false
    local tEarth = false
    local tTest1 = false
    local tTest2 = false
    local tPresence = false
    local tHouse = false
    local tScheduler = false
    local tRemoteAsync = false

     

    It's possible to run many scenarios at the same time but it's difficult to tell things apart then...

    So, please try again and I promise to guide and answer any questions. I know that when it comes to writing rules and the semantics the documentation is not that great - but I will try to support you as much as I can. I'm myself running this for quite some time and it's working smooth... however, my rules my not touch all of the code so it's great if someone else will try it out.

    Let me know how it goes.

    Link to comment
    Share on other sites

    • 0

    Jan

     

    aha the data file  - thanks.   That has got me up and running on the demo.  

     

    I was hoping to point to my Hometable and simply replicate the tRules IF block pulling definition remotely from HC2 i.e.  local conf = fibaro:getGlobalValue("HomeTable") and _Remote TRUE in EventRunnerDebug i.e.

     

    In eventrunner.lua

    Please login or register to see this code.

    Please login or register to see this code.

    in eventrunnerdebug.lua

     

    I guess I am missing something in how this hangs together as this leads to dev null error on execution.    Have to run - will be back trying to think it through.  Cheers

    Edited by ipsofacto
    more details
    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • 3 hours ago, ipsofacto said:

    Jan

     

    aha the data file  - thanks.   That has got me up and running on the demo.  

     

    I was hoping to point to my Hometable and simply replicate the tRules IF block pulling definition remotely from HC2 i.e.  local conf = fibaro:getGlobalValue("HomeTable") and _Remote TRUE in EventRunnerDebug i.e.

     

    In eventrunner.lua

    Please login or register to see this code.

    Please login or register to see this code.

    in eventrunnerdebug.lua

     

    I guess I am missing something in how this hangs together as this leads to dev null error on execution.    Have to run - will be back trying to think it through.  Cheers

     

    Your intuition looks right.

    The data file is only used if _REMOTE is false to initialise the (simulated) global home table variable.

    If you have _REMOTE true and have setup the IP address and username/pwd in eventRunnerDebug, fibaro:getGlobalValue("HomeTable") or fibaro:getGlobalValue(_deviceTable)  should fetch the variable value from the HC2.

    Try and set a break point in ZeroBrane and see what you get back... 

    However, you can't run the example_rules with you own table as they depend on the devices declared in the davicemap.data file. See line 23-27 in example_rules.lua. I may try to fix that dependency a bit better.

    I encourage you to put your rules directly in the main() function in the beginning of EventRunner.lua - that's what you need to do anyway when moving the code to the HC2.

    Pay attention also how you home table is structured, because that will decide how you access them in the rules. Look at the example below.

    Ex.

    Please login or register to see this code.

     

    Link to comment
    Share on other sites

    • 0
    Guest mszewcz

    @jgab I think you should change gsubs in T.makeIdTable() to following ones:

    Please login or register to see this code.

    If you use single % sign some incorrect thing happen, ex.: dev.%section.%room.xxx.%id becomes dev.%section.%ROOMNAME.xxx.%12 and not dev.SECTIONNAME.ROOMNAME.xxx.12

    You could also use some sanitizeString function to keep section's and room's names correct when written using with non ascii chars and spaces, ex:

    Please login or register to see this code.

    In above one I'm jut replacing polish chars with their ascii equivalents and removing spaces. Thanks to that "Sams room" is translated to "samsroom" in HT table, and not to "Sams":{"room".

    Hope it improves your code a bit. Anyway - good work. Keep doing!

    Edited by mszewcz
    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • 15 hours ago, mszewcz said:

    @jgab I think you should change gsubs in T.makeIdTable() to following ones:

    Please login or register to see this code.

    If you use single % sign some incorrect thing happen, ex.: dev.%section.%room.xxx.%id becomes dev.%section.%ROOMNAME.xxx.%12 and not dev.SECTIONNAME.ROOMNAME.xxx.12

    You could also use some sanitizeString function to keep section's and room's names correct when written using with non ascii chars and spaces, ex:

    Please login or register to see this code.

    In above one I'm jut replacing polish chars with their ascii equivalents and removing spaces. Thanks to that "Sams room" is translated to "samsroom" in HT table, and not to "Sams":{"room".

    Hope it improves your code a bit. Anyway - good work. Keep doing!

    Thanks for spotting the error - strange, because I have double '%%' in my own code...

    Yes, something like 'sanitizeString' is good if you have localised names.

    Link to comment
    Share on other sites

    • 0

    @jgab Just to give feedback, this project is super interesting, though I will probably never implement it given that my system - 138 devices, 31 scenes - runs fine.

     

    However I had the idea of a central log scene for some time, and some attempts using a global to store log entries lead to some of the issues that you mentioned, particularly lost messages. I have no proof but I suspect race conditions. Your project and the _MAILBOX approach gave me some fresh inspiration, let's see if I manage to work something out.

     

    Cheers,

     

    jayrock

     

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • 21 minutes ago, jayrock said:

    @jgab Just to give feedback, this project is super interesting, though I will probably never implement it given that my system - 138 devices, 31 scenes - runs fine.

     

    However I had the idea of a central log scene for some time, and some attempts using a global to store log entries lead to some of the issues that you mentioned, particularly lost messages. I have no proof but I suspect race conditions. Your project and the _MAILBOX approach gave me some fresh inspiration, let's see if I manage to work something out.

     

    Cheers,

     

    jayrock

     

    Thanks,

    Never touch code that runs fine :-)

    /J

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • @jompa68Are we talking the EventRunner framework?

    The easiest is the script rules.

    Please login or register to see this code.

    '@time' runs at the specified time every day and can be qualified with more test to target specific days

    Please login or register to see this code.

     

    Link to comment
    Share on other sites

    • 0
    1 minute ago, jgab said:

    Are we talking the EventRunner framework?

    Yes, EventRunner Framework is the one i am trying to understand now. Looks promising so far. Nice with all the examples in example_rule.lua
    Guess i will come back with more question :)

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Guess you have looked at 

    Please login or register to see this link.

     ? Lots of examples in the left hand menu - script tutorial etc.

     

    • Like 1
    Link to comment
    Share on other sites

    • 0
    31 minutes ago, jgab said:

    Guess you have looked at 

    Please login or register to see this link.

     ? Lots of examples in the left hand menu - script tutorial etc.

     

    Yes i have seen that, very helpful.
    If i want to use data from "HomeTable", does Util.defvar support it?

    Please login or register to see this code.

     

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Yes, you can drag in the whole home table with

    Please login or register to see this code.

    Here everything hangs under the root 'dev'. Typically you want to define variables a level down which is why I usually do:

    Please login or register to see this code.

    Also if you do 

    Please login or register to see this code.

    The logger will print the hometable name instead of the integer value of the device.

    Edited by jgab
    Link to comment
    Share on other sites

    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.

    Guest
    Answer this question...

    ×   Pasted as rich text.   Paste as plain text instead

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×
    ×
    • Create New...