Wait until http:request returns

[wpeters 1]

In order to obtain information from "non FIBARO" devices, I have setup a small Webserver which I would like to call from my HC2 via FIBARO LUA scenes. Unfortunately my HC2 executes the "http:request" after executing the main LUA scene (pls. see a small example and scene output). I currently have created a workaround to start another scene (which submits the http:request) and wait for it´s result. This seems to me quite complicate My questions to the experts are: 

 

• Am I doing something wrong with that "http:request" call?
• Is there any other way to request and receive a web response within a LUA scene?   
• Is there a way to put the main scene execution "on hold" and to enable "http:request" execution? I have tried "fibaro:sleep" without success.

 

Thank you for your help!

 

Wilfried

 

 

--[[
%% properties
%% events
%% globals
--]]

function sendRequest(url)                      -- url to execute
  fibaro:debug("... start sendRequest.");
  local http = net.HTTPClient();
  http:request(url, {
      success = function(response)
        if ((response.status == 200 or response.status == 201)) then
          fibaro:debug("Status:  " .. response.status);
          fibaro:debug("Data:    " .. response.data);
        else
          fibaro:debug("Error: " .. responce.status);
        end
      end,
      error = function(err)
          fibaro:debug("Error: " .. err);
      end
    })
    fibaro:debug("... end sendRequest.");
end

 

fibaro:debug(os.date("%x") .. " ENTER");  
sendRequest("http://192.168.1.190:8080/info/myDevive001");
fibaro:debug(os.date("%x") .. " EXIT");

 

The output is:

[DEBUG] 19:53:26: 03/15/19 ENTER
[DEBUG] 19:53:26: ... start sendRequest.
[DEBUG] 19:53:26: ... end sendRequest.
[DEBUG] 19:53:26: 03/15/19 EXIT
[DEBUG] 19:53:26: Status: 200
[DEBUG] 19:53:26: Data: {"datapoints" {"idp0000":"22","idp0001":"1","idp0002":"0","idp0003":"0"}}

 

 

 

 

[jgab 2,105]

10 hours ago, wpeters said:

In order to obtain information from "non FIBARO" devices, I have setup a small Webserver which I would like to call from my HC2 via FIBARO LUA scenes. Unfortunately my HC2 executes the "http:request" after executing the main LUA scene (pls. see a small example and scene output). I currently have created a workaround to start another scene (which submits the http:request) and wait for it´s result. This seems to me quite complicate My questions to the experts are: 

 

• Am I doing something wrong with that "http:request" call?
• Is there any other way to request and receive a web response within a LUA scene?   
• Is there a way to put the main scene execution "on hold" and to enable "http:request" execution? I have tried "fibaro:sleep" without success.

 

Welcome to an asynchronous world

 

You have to think differently about the flow in your program.

Instead of thinking that your program executes from start, first line, to end, last line - your program consists of functions calling each other from start to end.

That means that every part of your program more or less  needs to be a function...

A common program structure is (there could be more or fewer program parts, but at least a main() )

Please login or register to see this code.

Assume now that in setup we need to do a sendRequest to get some values from an external server before we can run our main

First we implement sendRequest to also take a continuation function like in the example above. A function that takes as argument the result of the requests (the data), so we could reuse the sendRequest function for other types of requests.

Please login or register to see this code.

We can now easily modify our program structure to insert the sendRequest in setup before the main()

Please login or register to see this code.

The continuation function given to sendRequest sets a global 'myData' here before calling main, and main makes use of that global. An alternative would be that the continuation function gives data as an argument to main(data). In any case, a very non-intrusive addition to our program, and we still have a "sequential" feel of the flow between the parts of our program.

 

You need to make 2 sendRequests in setup before main? easy

Please login or register to see this code.

So, you need to start to have a "functional" mindset - it is a very powerful way to think about program structure. The setTimeout function also fits in natural here as it is just seen as a way to insert a delay in your chain of function calls.

 

It's actually possible to define a 'join' function for 2 or more parallel asynchronous calls. The last example above can be implemented as a "join" between two parallel sendRequests instead and we should continue with main when both are finished. ’Now they are run in sequence which are not particular efficient. Implementing a ‘join’ is left as an exercise for the reader...

Edited March 16, 2019 by jgab

[jgab 2,105]

ok, a 'join' would look like this,

The 'continuation' function will get a table with the results from all the requests.

Please login or register to see this code.

'join's own continuation function just store the result in the results table and counts down a variable holding the number of requests there are, and when reaching 0 knows that all requests are done, and then call the original continuation with a table of all the collected results.

Btw, this is not faster (internal threads are cooperative lua threads), but it's a "cleaner" way to code fork and joins in a functional continuation model.

Edited June 7, 2021 by jgab

[wpeters 1]

Hi jgab:

Thank you so much for your quick answer and your education. I‘m using LUA scenes and Virtual Devices within my HC2s quite a regular. I have got in touch with FIBARO LUA by reading articles in the internet (e.g. FIBARO documentation), however, cannot recall any of that details. Since standard LUA is in my view quite different from FIBARO’s LUA implementation could you recommend someone like me a place (e.g. books) to go to educate himself?

 

I‘m excited to start experimenting with the things I learned from you today.

[jgab 2,105]

On 3/16/2019 at 10:19 AM, wpeters said:

Hi jgab:

Thank you so much for your quick answer and your education. I‘m using LUA scenes and Virtual Devices within my HC2s quite a regular. I have got in touch with FIBARO LUA by reading articles in the internet (e.g. FIBARO documentation), however, cannot recall any of that details. Since standard LUA is in my view quite different from FIBARO’s LUA implementation could you recommend someone like me a place (e.g. books) to go to educate himself?

It's really standard Lua. ...or it's standard Lua where Fibaro has chosen to disable some standard features.

Call-backs that you get from net.HTTPClient is pretty standard programming concepts, and I suspect that the reason we get an asynchronous call-back is that net.HTTPClient wraps the call-backs in a Lua coroutine. coroutines are standard in Lua but Fibaro has chosen to hide them from us (I suspect setTimout also wrap the functions in coroutines). Coroutines are like processes but only one can run at the time - that's why the "main flow" of the program has to finish before the call-backs are executed. Then the way Fibaro has chosen to implement scenes we can't blame Lua for  

[Mateo 2]

On 3/16/2019 at 8:52 AM, jgab said:

ok, a 'join' would look like this,

The 'continuation' function will get a table with the results from all the requests.

Please login or register to see this code.

'join's own continuation function just store the result in the results table and counts down a variable holding the number of requests there are, and when reaching 0 knows that all requests are done, and then call the original continuation with a table of all the collected results.

Btw, this is not faster (internal threads are cooperative lua threads), but it's a "cleaner" way to code fork and joins in a functional continuation model.

 

 

Hello @jgab

 

how you find the time to bring us all those recommendations is a mystery for me

thanks again

 

i'm having i guess the same kind of issue and struggling with for more than 24 hours... need some help

 

i'm creating a QA to control hue lights in a room.

i'm collecting in multiple global variables all the information i need for (hue user, ip, port ....) and store in QA variables

THEN i'm requesting the hue bridge to get from him:

* data around the hue group (name, lights...) that i store in the QA variable

* with the data of the hue group (from getVariable), details around light (colormode, reachable ...)

 

i'm doing all this in a linear approach in oninit()and the issue is that all this is asynchronous and doesn't do the job in the correct order.

i have to play onInit 3 or 4 times to get it all

 

i read above but don't understand this cont thing... 

 

please see my code below. How would i hierarchize all this?

creating external function to onInit() and ?

 

thank in advance

br

mde

Please login or register to see this code.

 

[jgab 2,105]

You could structure your code like this

Please login or register to see this code.

Btw, never do fibaro.sleep inside a success handler. Doesn't help..

Please login or register to see this code.

 

Edited August 4, 2020 by jgab

[Mateo 2]

3 hours ago, jgab said:

You could structure your code like this

Please login or register to see this code.

Btw, never do fibaro.sleep inside a success handler. Doesn't help..

Please login or register to see this code.

 

 

you are superman !

will have a detailed look on all this in the coming hours

so great

 

i've tried to separate those hue accesses out of the onInit function but was not looking as bringing the expected results....

will try again with your detailed approach, may have (surely have) miss something !

thanks a lot... so eager to test this after hours of tests

 

have a great day

 

 

3 hours ago, jgab said:

You could structure your code like this

Please login or register to see this code.

Btw, never do fibaro.sleep inside a success handler. Doesn't help..

Please login or register to see this code.

 

 

just understood main difference and surely the basics of the cont approach you were evoking ...

I was launching function step 1 and 2 from onInit where you launch step 2 from step 1... that makes all the difference

 

thanks again,

will test as soon as possible 

 

 

 

 

[Mateo 2]

2 hours ago, Mateo said:

 

you are superman !

will have a detailed look on all this in the coming hours

so great

 

i've tried to separate those hue accesses out of the onInit function but was not looking as bringing the expected results....

will try again with your detailed approach, may have (surely have) miss something !

thanks a lot... so eager to test this after hours of tests

 

have a great day

 

 

 

just understood main difference and surely the basics of the cont approach you were evoking ...

I was launching function step 1 and 2 from onInit where you launch step 2 from step 1... that makes all the difference

 

thanks again,

will test as soon as possible 

 

 

 

 

 

hello @jgab

 

unfortunately it doesn't work and brings the same issue...  ... was so excited

 

Actually, my final goal is to use this QA for all my different rooms, just changing the QA variable content relative to the room and then getting all device id and other information for a central and mapping table in global variable.

To test this capacity to replicate, i do have added a "delete All" function to reset all QA variables and restart from scratch for a new room. In this function, i've integrated a self:onInit() to replay set up of everything and launching hue request accordingly to your recommendation.

 

But the results remains the same:

* deleting QA variable is very long (almost 1 sec per variable) <> perhaps not the correct way doing it

* getting global variable details for all devices and setting it as QA variable very long also (almost 30 secs) <> that's a lack of reactivity i don't understand

* and contents relative to the hue bridge returns remains at 0 value in the debug returns (but are partly correctly created as QA variable... needing one more onInit execution to have it all...) <> i would like this delete function to reset to 0 all QA variable (except room and floor) and get in one clean move all info for the new room, including hue one.

 

What is the thing i'm missing?

it's killing my brain

Should i fragment all my onInit in separated function to secure continuity ?

 

please find below my code update.

Tell me if fqa is better and easier for you than copy and paste code (from zerobrain, following your recommendation  )

 

thanks

br

mde

 

Please login or register to see this code.

Please login or register to see this attachment.

 

Please login or register to see this attachment.

 

Edited August 4, 2020 by Mateo

[Taz 0]

I'm confused... I'm trying to do POST request and then GET just after that to see if change reflected, but I get "old" value:

Function for API calls

Please login or register to see this code.

 

code:

Please login or register to see this code.

terminal output:

Please login or register to see this code.

so my understanding is `GET` request should be executed only if `POST` is completed, but it doesn't seem to wait for post to be finished, because `current_temp` is not updated (22 instead of 23)? If I execute it 1-2 secs later then both temperatures are 23:
 

Please login or register to see this code.

Am I missing something?