Jump to content

Welcome to Smart Home Forum by FIBARO

Dear Guest,

 

as you can notice parts of Smart Home Forum by FIBARO is not available for you. You have to register in order to view all content and post in our community. Don't worry! Registration is a simple free process that requires minimal information for you to sign up. Become a part of of Smart Home Forum by FIBARO by creating an account.

 

As a member you can:

  •     Start new topics and reply to others
  •     Follow topics and users to get email updates
  •     Get your own profile page and make new friends
  •     Send personal messages
  •     ... and learn a lot about our system!

 

Regards,

Smart Home Forum by FIBARO Team


SDK for remote and offline HC3 development


jgab

Recommended Posts

19 minutes ago, jgab said:

 

 

The point is to make it transparent for the QA. You can develop the master QA in ZBS and move it to the HC3 and the QA will not see the difference. You can have a "standalone" program and a QA with the same code.

What I probably need it's just emulation for fibaro functionality that's all.

Like emulation for fibaro.call, fibaro.getValue, api.get/put/post, fibaro.getVariableValue, fibaro.debug/error/trace and etc.

Edited by cag014
Link to comment
Share on other sites

  • Topic Author
  • 7 minutes ago, cag014 said:

    No... I don't want to run anything on HC3 or any other controller, just to run the code on PC.

    Right now I run a code that controls HC3 and HC2 (as a slave) and everything looks fine (I mean the values from HC2 are correct).

    Could you suggest how to stop internal refershStates  just for testing.

     

    if dofile and not hc3_emulator then
      hc3_emulator = {
        poll=false, 

        :
     

    7 minutes ago, cag014 said:

    Another issue, it looks like the Http timeout doesn't accept milliseconds? When I set timeout to 1000 it works, but by setting to 999, I never receive response from Http(refreshstate) 

    http requests in ZBS are not really asynchronous. So I. have to "fake it"

    You can try to do

    net.HTTPClient = net.HTTPAsyncClient

    before you create your first. request, It will make it look like an asynchronous requests, and your code will probably not hang when. you poll refreshState.

    • Thanks 1
    Link to comment
    Share on other sites

  • Topic Author
  • 7 minutes ago, cag014 said:

    What I probably need it's just emulation for fibaro functionality that's all.

    Like emulation for fibaro.call, fibaro.getValue, api.get/put/post, fibaro.getVariableValue, fibaro.debug/error/trace and etc.

    Don't you want your code to run on a HC3 too?

    If it's just to control HC3/HC2 boxes I would probably just use the rest APIs or?

    Link to comment
    Share on other sites

    13 minutes ago, jgab said:

    Don't you want your code to run on a HC3 too?

    If it's just to control HC3/HC2 boxes I would probably just use the rest APIs or?

    First of all thanks for previous answers... looks like it works fine... will tested a little bit further.

    No, I don't want to run that on HC3. 

    The idea it to run it totally on PC  to achieve three  goals

    1. Using entire LUA library and capability to work with 

    2. HCL users can use this code as well and use some Lua functionality that supported by All-in-One.

    3. Same code can run on the controllers as well.

     

    Yes, I do use rest APIs, but I don't want to invent the wheel... You did an excellent job with the emulator.

     

    Edited by cag014
    Link to comment
    Share on other sites

    @jgab

    My main problem with the code is the Html colors and fonts.

    While your emulator supports html colors by replacing the <font color=xxx>(...) </font>, I need your help to make it more "smart" (if you have spare time)

     Html supports fonts "recursively", the emulator does not.

    for example

    <font color=red>RED-TEXT<font color=blue>BLUE-TEXT</font> BACK RED TEXT</font>

    This line in emulator  shows text ass following

    REDTEXT<font color=blue>BLUE-TEXT</font> BACK RED TEXT</font>

    Need some complicated loop to extract and set correctly colors.

     

    I'm going through my code to set that correctly. Not an easy task, to make it work, the line in example should be :

    <font color=red>RED-TEXT</font><font color=blue>BLUE-TEXT</font><font color=red> BACK RED TEXT</font>

     

    Again, nothing urgent or serious issue...

     

    Edited by cag014
    Link to comment
    Share on other sites

  • Topic Author
  • 11 hours ago, cag014 said:

    @jgab

    My main problem with the code is the Html colors and fonts.

    While your emulator supports html colors by replacing the <font color=xxx>(...) </font>, I need your help to make it more "smart" (if you have spare time)

     Html supports fonts "recursively", the emulator does not.

    for example

    <font color=red>RED-TEXT<font color=blue>BLUE-TEXT</font> BACK RED TEXT</font>

    This line in emulator  shows text ass following

    REDTEXT<font color=blue>BLUE-TEXT</font> BACK RED TEXT</font>

    Need some complicated loop to extract and set correctly colors.

     

    I'm going through my code to set that correctly. Not an easy task, to make it work, the line in example should be :

    <font color=red>RED-TEXT</font><font color=blue>BLUE-TEXT</font><font color=red> BACK RED TEXT</font>

     

    Again, nothing urgent or serious issue...

     

    Parsing nested html tags is doable.

    I think the main problem is that the ZBS log console is very limited. In principle only a limited set of colors.

    Starting to do extensive html parsing for log output may not be worth the effort.

     

    Another approach I'm thinking of is to use the emulator's web interface. 

    While the emulator is running it also runs a small web server. The web server is mainly used to receive triggers pushed from the HC3 (like buttons pressed etc). However, the server also publish a simple interface to the emulator (but only when the emulator is running).

    On the PC, while the emulator runs, you can open your web browser at 

    http://127.0.0.1:6872/web/

    and there are some pages for devices, the QA UI etc. The intention with the interface is to provide some possibilities to simulate the QA UI and turn on/off devices to trigger callbacks to the emulator.

    I guess it would be possible to support a "log" tab that would support full CSS/HTML. The question is just what update model one would choose for the page? Continuously reload or some kind of ajax model?

     

    Because running the emulator in ZBS comes with some limitations (can't run something in the background while ZBS is not executing the emulator) I have also been thinking about moving the emulator to a docker container. It would be a real "standalone" emulator that could also run a real web server like nginx in the background. ZBS would still allow for remote debugging with mobdebug.

    However at the moment the emulator is easy to use, just one file to put in your development directory. A docker setup would require more from potential users...

    Edited by jgab
    Link to comment
    Share on other sites

    1 hour ago, jgab said:

    Starting to do extensive html parsing for log output may not be worth the effort.

     

    I agree..

    Link to comment
    Share on other sites

  • Topic Author
  • 1 minute ago, cag014 said:

    I agree..

    because I was already scanning for single color tags in log output it turned out that it was not more expensive to scan for nested color tags - so I added that,

    • Thanks 1
    Link to comment
    Share on other sites

    On 1/28/2021 at 10:36 AM, jgab said:

    because I was already scanning for single color tags in log output it turned out that it was not more expensive to scan for nested color tags - so I added that,

    Thanks... works fine

    Finally everything runs absolutely great and stable.

    One question, do you know how to run ZeroBrane in parallel? I mean to run few programs at the same time?

    In addition I want to execute in parallel some tasks inside All-in-One. Using Lua coroutines is not exactly multi-thread

    Edited by cag014
    Link to comment
    Share on other sites

  • Topic Author
  • 1 hour ago, cag014 said:

    Thanks... works fine

    Finally everything runs absolutely great and stable.

    One question, do you know how to run ZeroBrane in parallel? I mean to run few programs at the same time?

    In addition I want to execute in parallel some tasks inside All-in-One. Using Lua coroutines is not exactly multi-thread

    Well, Lua's runtime engine doesn't like to run in parallel (it's not re-entrant). The only way to do it is to spawn multiple Lua runtimes in parallel with separate linux processes - like scenes and QAs are run in the HC3 - but they can't share Lua code or state....

    If it was possible to run the same Lua runtime with multiple processes accessing the variables and functions there would be a need for some advanced synchronisation primitives like in languages that support multithreading. 

    Multi-threading is not all what it's hyped up to be. With multi-threading you may be able to run stuff in parallel but it creates a lot of head ache to not create race conditions and there is always subtle bugs that hit you when you think that everything is working. Coroutines requires your "processes" to explicitly give up time to other processes - but you have much better control when that happens and it's easier to prevent race conditions.

    The coroutine model in Lua is actually extremely efficient and used in some hight performance web server engines.

     

    It's pretty simple to do "parallel" lua functions in fibaroapiHC3.lua

    Please login or register to see this code.

     

    test1 and test2 are programs with infinite loops that makes sleeps to give up time to other processes.

    You could do

    Please login or register to see this code.

    and as long as myprogram.lua does a sleep here and there to give up time it will run "in the background".

    even 'sleep(0)' would give other coroutines waiting the time to run.

    You could automatically instrument all fibaro.* and QuickApp:* to do a sleep(0) just to make sure that sleep is called.

    Please login or register to see this code.

    ...or you may know better where to put these sleeps.

     

    Imagine if Fibaro would allow coroutines in QuickApps/Scenes we could write some really cool code. fibaro.sleeps that doesn't block all other threads. Remote calls without fibaro globals. My EventRunner could cut ~300 lines of code now used to implement my own pseudo coroutines...

    • Like 2
    Link to comment
    Share on other sites

    @jgab

    I'm trying to use VS code, but receive too many errors about missing dll and lua files

    Have tried to import the files from ZeroBrane, but in several cases the error is that the application doesn't support Win32!?

    Do you know by the chance how to create correct environment for Lua?

    I did install Lua debug extensions...

    Thanks

    Link to comment
    Share on other sites

  • Topic Author
  • 7 minutes ago, cag014 said:

    @jgab

    I'm trying to use VS code, but receive too many errors about missing dll and lua files

    Have tried to import the files from ZeroBrane, but in several cases the error is that the application doesn't support Win32!?

    Do you know by the chance how to create correct environment for Lua?

    I did install Lua debug extensions...

    Thanks

    @10der is using the emulator with VS on Windows. Is that what you use too? I’m a strictly Mac guy and VS on Mac isn’t that good for Lua.

    @10der any hints?

    Link to comment
    Share on other sites

    17 hours ago, cag014 said:

    I'm trying to use VS code, but receive too many errors about missing dll and lua files

     

    I think I understand what you are saying. With Zerobrane "out of box" experience is good, no issue with paths and dlls.

     

    With VSC extension... It is possible to get it working but I can only give approximate instructions because every environment will be different. The key is the paths used by the Lua extension.

     

    so in launch.json I have

     

    Please login or register to see this code.

     

     

    The path, cpath and cwd are all very important and you can see I am using Lua 5.4 and I am on a Mac -> lots of references to /usr/local/lib.

     

    But you do not have to guess or understand the paths, you can start Zerobrane and print the paths. Here's an example after I've selected an older Lua in Zerobrane:

     

    Please login or register to see this code.

     

    To be honest I cannot remember why I set the current working dir to ${workspaceFolder}/../.. but I can say I used to have a tree structure. I would highly recommend to store all files (credentials.lua, fibaroapiHC3.lua, your code) in a single directory and keep things simple...

     

    Another "to be honest"........ When I try this now, the debugger seems to "start" then after a few seconds "stop" without message or trace - not good. I haven't used this VSC plugin in a while (errrrrrrm several months probably) and either something in my environment changes, or the actboy168 extension changed, VSC version for sure has changed and possibly it is due to an upgrade to "Big Sur" (though not likely).

     

    And lastly it is worth checking where "luarocks" has stored your extensions, and where,

     

    Please login or register to see this code.

     

    Good Luck

    • Thanks 1
    Link to comment
    Share on other sites

    BTW because I couldn't get the debugger going in VSC I downloaded and added this to one of my target scripts:

     

    Please login or register to see this link.

     

    A simple, highly embedabble CLI debugger for Lua 5.x, and LuaJIT 2.0.

     

    Trivial to "install". Can be integrated as a single .lua or .c file.
    The regular assortment of commands you'd expect from a debugger: continue, step, next, finish, print/eval expression, move up/down the stack, backtrace, print locals, inline help.
    Evaluate expressions and call functions interactively in the debugger with pretty printed output. Inspect locals, upvalues and globals. Even works with varargs ... (Lua 5.2+ and LuaJIT only).
    Pretty printed output so you get {1 = 3, "a" = 5} instead of table: 0x10010cfa0
    Speed. The debugger hooks are only set when running the step/next/finish commands and shouldn't otherwise affect your program's performance.

    ........ and some more

     

    It's got that "gnu dbg" feel to it, but I've used this library a few times and it does the job.

     

    Link to comment
    Share on other sites

  • Topic Author
  • I never used anything than ZB on Mac. I did struggle with VS on Mac a last summer but gave upon as the Lua plugin didn't seem that "great"...

     

    The emulator really doesn't care about the debugger except that it's probably useful if the debugger supports coroutines as it is used in the emulator. I do try to load mobdebug.lua if it exists as debugging of coroutines needs to be enabled explicitly. Trying to load it should be an option as it may interfere with other debuggers.

     

    • Thanks 1
    Link to comment
    Share on other sites

    Guys. 1st of thank you for the question.

    the second moment I am not happy with flim-flam FIBARO company (and ZigBee native support) and do not read this forum at all.

    only just got the some notification from my friends like a Jan / Peter /  Peter #2 :) 

    that's sorry.

     

    about VSCode

    Please login or register to see this attachment.

    main here - please OPEN fibaro.code-workspace  as VSCode workspace.

    otherwise  VSCode will use default LUA env what is not compatibly with libs

    • Thanks 3
    Link to comment
    Share on other sites

    @jgab, @10der, @petergebruers

    Thanks for your help.

    Now it runs on VS code.

    To my opinion VS code more powerful editor and it could execute several instances of the program w/o any problem.

    That gives an option to keep running the program to control smart home and to to develop more features in parallel.

     

    Again thanks a lot for all your help

     

    • Like 1
    • Thanks 1
    Link to comment
    Share on other sites

  • Topic Author
  • Just now, cag014 said:

    @jgab, @10der, @petergebruers

    Thanks for your help.

    Now it runs on VS code.

    To my opinion VS code more powerful editor and it could execute several instances of the program w/o any problem.

    That gives an option to keep running the program to control smart home and to to develop more features in parallel.

     

    Again thanks a lot for all your help

     

    I'm in the process of rewriting the core of the emulator. The core is a really good timer model that allow us to run setTimeout but also keep other processes going - like a web server. I have used my own implementation and it has been fun to code but it has always been the part of my code suffering from bad Feng Shui.

    I'm rewriting it now to use the Copas library for co-routines and network IO. I will include relevant parts of the Copas code in my own code as to not create dependencies (and I have to make some small adjustments). It will still depend on LuaSocket for the base network IO.

     

    The good news is that the performance should be much better for user code (QAs/Scenes) as well as for network IO (web server net.* etc). 

     

    • Like 1
    • Thanks 1
    Link to comment
    Share on other sites

    Hello Jan,

    I have a QA who send a <fibaro.call(1082"sendCommand"170)>

    Its working good but if 1082 is restarted or HC3 restarts it will change the value due to:

    <function QuickApp:onInit()    self:sendCommand()>

    How to prvent or bypass it?

    Thanks for all your valuable adivices,

    //Sjakie

    Link to comment
    Share on other sites

    • 3 weeks later...
  • Topic Author
  • Next generation of the emulator.

    Please login or register to see this link.

     

    I've tacked on a 'C' on the filename, but as it matures it will replace the old fibaroapiHC3.lua that I will freeze now.

    It's a much bigger rewrite than I originally intended, so there are still bits and pieces lacking.

     

    What's new.

    -Internal timer engine based on Copas.

        --gives better overall performance and non-blocking IO

    -Multitasking - allows for multiple QAs and Scenes to be loaded and run in parallel (and be debugged).

        --testing interactions between QAs and QAs and Scenes

    -QA can downloaded, converted, installed in the emulator, and uploaded back to the HC3 in several formats.

       Ex:

       -- loadQA(33):install() -- Downloads QA 33 from HC3 and installs it in the emulator

       --loadQA(33):save("faqs/","unpacked"):install() -- Saves as unpacked in dir /fqas. Opening the saved unpacked files allows debugging...

     

    What's old. 

    - Old emulator format, "if dofile..." header still supported

     

    The new code also allows for new features to come...

     

    ----------------------------------------------------------------

    Header for QAs. and Scenes that should be emulated look like in the. previous version

    Please login or register to see this code.

    For a Scene the 'type', 'proxy' and 'UI' is ignored as they are only relevant for QuickApps. I'm considering allowing proxies for Scenes too, but it's not prioritised at the. moment.

     

    The emulator, when running, opens two ports.

    hc3_emulator.webPort=6872 for a web server. This server both receives events pushed from the HC3 and also serves up a Web GUI on

    Please login or register to see this link.

    that can be used to interact with QAs and devices

    hc3_emulator.webPort=6972 is a socket listener that can be connected to (with ex. telnet) and interact with the emulator when it's running. ZeroBrane Studio is a bit limited when it comes to running commands when the application is running. This listener allows lua commands to be typed into the emulator. when it's running.

    Please login or register to see this code.

    Two commands are treated special.

    'quit' - closes the port.

    'log <pattern>' - register a log listener for log output with the tag <pattern>

    log 120

    will output all log commands with tag 'QUICKAPP120' on this port. This allows many terminals to be setup to capture the logs from different QAs and/or Scenes.

    The port can also be used to pipe commands into to turn on/off devices in specific patterns to test a QA ex.

     

    To include files in a QA use the directive --FILE:<file name>,<module name>;

    Ex.

    Please login or register to see this code.

    The files will be loaded when the QA is emulated and when built/uploaded the files will be added with the module names specified in the .fqa created.

     

    The main file should have an emulator header

    Please login or register to see this code.

    At startup the emulator is loaded with the dofile(...) statement in the header. The emulator then takes over and finds out what file it was included from (using the Lua debug interface). It then looks into the file and extracts the header and evaluates it so that the hc3_emulator table becomes populated with default values specified by the user.

    It then looks further into the file and if it discover a 'hc3_emulator.actions' statement it decides that the file is a scene. If it discovers a "QuickApp:" statement it decides that the file is a QuickApp.

    If it is a scene it then continues with

    Please login or register to see this code.

    to run it as a scene.

    If it was a QA it does

    Please login or register to see this code.

    If it was neither a Scene or QA file it just loads the code and runs it, but with the environment available for QAs (json.*, fibaro.*, setTimeout etc). This kind of "plain" emulator file can be useful for testing lua code or loading in other code.

    Ex

    Please login or register to see this code.

    This loads QA with deviceID 33 from the HC3, install it in the emulator and runs it...

     

    The functions for loading and installing QAs and Scenes in the emulator are

    Please login or register to see this code.

     

    The load* functions returns an "object" with the methods :install(), :save(), :upload(), and :args().

    That's why we can do 

    Please login or register to see this code.

    Each methods returns the object so we can chain the methods.

    loadQA(33):save("unpacked","faqs/"):install()

    This loads the QA, saves it as unpacked in the faqs/ directory and then installs it.

    Edited by jgab
    • Like 3
    • Thanks 1
    Link to comment
    Share on other sites

    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.

    Guest
    Reply to this topic...

    ×   Pasted as rich text.   Paste as plain text instead

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×
    ×
    • Create New...