curious behaviour of net.HTTPClient

[maxdreamland 1]

Hi,

 

I came across a behaviour of net.HTTPClient that i don't understand, maybe someone can shed some light on this?

 

have a look at this simple example, where i define a helper function to print out a table and a function to make an HTTP call and then i make two HTTP calls with a delay of 5 seconds inbetween them:

 

Please login or register to see this code.

 

When you start this scene you'll notice that for 5 seconds nothing happens and then the two HTTP calls are made right after one another with no delay inbetween them.

 

So it's as if the code was actually written like this:

 

Please login or register to see this code.

Any idea why this is?

 

thanks in advance!

 

best regards

Max

 

[petergebruers 1,268]

This topic is related to you question, it contains explanation + solution.

Please login or register to see this link.

I know it's rather long, so I understand if you are still uncertain after reading that. But I think it contains important aspects of this "callback" style of programming.

[maxdreamland 1]

Ok, I read that thread and I understand the concept of the callback functions. And I understand why you can't be sure of the order of execution if you have several http-calls in sequence, beause the success/error-callbacks are actually executed on a different thread.

 

but i think there is more to this problem. In my example above I have a sleep(5000) between the first and second call, and it is an HTTP call to the local/internal HTTP-Server, so it should be very quick. Certainly i expect the callback to come back quicker than 5 seconds, right? So i would expect the debug-output from the callback of the first http-call to arrive within those 5 seconds and show up in the debug console. but that is not the case, the debug output of both calls appears simultaneous (same timestamp) after the 5 second sleep is over.

 

and you could also look at this from the perspective of the receiver of the HTTP-calls. For example, If i replace the HTTP-Call from a simple HC2-API call to an HTTP-PUT call to one of my AEOTEC LED bulbs i can see that the HTTP-call doesn't seem to be SENT before the 5 seconds are over.

 

For example, i have one HTTP-call to set the color of the LED to red, then wait 5 seconds and make the second call to set the color to green.

 

what happens now is, for 5 seconds nothing will happen with the LED and then it will shortly flash red and immediately switch over to green. 

 

so it seems to me there is more going on than just the asynchronous callback.

 

thanks in advance for elaborating.

Max

[petergebruers 1,268]

The http calls start as soon as the main code ends. So putting "sleep" in the main code delays the start of ever http call. Instead of putting every http call in main, like you do for "usual" Fibaro commands, chain the http calls via the callback functions. For your led example: in main do a http call to set the led "red" and nothing more. In the callback function of that first http call, sleep for 5 seconds, then call the http code to set the led to "green". If you want to do something else, put whatever sleep you want in this second callback function, and call a third url. And so on.

[maxdreamland 1]

Wow. That's pretty fucked up

Please login or register to see this image.

/emoticons/default_icon_smile.gif" alt=":-)" /> I never would have guessed that the HTTP calls are delayed until the scene finishes. I can't even desribe how baffled I'm about that.

 

I never worked with LUA before i bought the HC2 a couple of weeks ago (but I'm a software developer by trade)...is this behaviour "normal" for the LUA script environment or is it a special kind of "stupid" we have to deal with just in the scene context?

 

I noticed there is a second "Net" namespace (Net instead of net) that is only visible from within a virtual device but not a scene, does that have the same restrictions?

 

regards

Max

[petergebruers 1,268]

The VD http implementation is different and predates the version used in scenes. The same behavior exists for "setTimeout". See:

Please login or register to see this link.

In lua, the concept of "coroutines" exists, but this is not available in the Fibaro implementation. There are no thread manipulation and synchronization primitives.

[ipsofacto 8]

This is a really interesting topic that also demonstrates quite how complicated it can quickly become to develop meaningful scenes in Fibaro. Whilst it is great that Fibaro have implemented their REST API, it is hardly an intuitive work with collections of devices etc with LAU itself eg. Switch off all lights except those in room 2 and 3.

Three questions to the Fibaro team:

1. Why is the .net implementation different in VD and scenes - do you plan to make this consistent and improve the documentation related to using within LUA

2. Any plans to extend the Fibaro:LAU functions eg to provide more access to devices within system. I would be really happy if we were able to query / filter and process against collections of devices

3. Allowing users to define global functions for use across an installation would be invaluable - at least allow an "include" instruction to allow common functionality to be imported into VD and scene code blocks!

I have pretty much stopped extending my implementation - magic scenes are far too simplistic for real use, blocks are OK for very specific rules and actions. LUA has great potential but simply too many restrictions to be manageable.

I won't mention the plug-in fiasco!

Seasons Greetings

[maxdreamland 1]

3. Allowing users to define global functions for use across an installation would be invaluable - at least allow an "include" instruction to allow common functionality to be imported into VD and scene code blocks!

 

you could place your "+1" in this bugzilla ticket that "petitions" a solution for exactly that:

 

Please login or register to see this link.

 

if it will make a difference, i don't know, but i imagine it has more impact there than in a random forum post.

 

I'm still baffled about the Fibaros motivation behind implementing net.HTTPClient in this way. Peter said it is the more recent implementation (comared to the VD Net variant) and that the VD version behaves different. From the context we're talking about i assume he means that it behaves "synchronous" - you call it and execution waits till the respones comes in. And from me looking at sme example code thats also the impression i get from that.

 

so i'm wondering why would Fibaro re-implement an already available functionality in a different and inherently more complicated way for the scene context while simultaneously blocking access to the older more straight forward implementation? I'm only part of the Fibaro community for a couple of weeks and maybe this is obvious to an "old timer", but it is not to me.

 

I mean, Fibaros goal with the magic and blocked scenes is obviously to make things simpler for the non-programmer enduser. so why would they implement this more complicated version and deny access to the simpler version at the same time?

 

but apart from these more philosophcal questions...I initially came upon this whole mess because i implemented a LUA library that allows easy control of an AEOTEC RGBW bulb. I know that there is a virtual device for that, but i don't like the idea of having two devices per bulb (the "real" bulb and the virtual device). And it was fun to figure out how the bulb interface works, too

Please login or register to see this image.

/emoticons/default_icon_smile.gif" alt=":-)" />

 

so now i have this basically ready to publish library but it turns out because of the curious HTTP implementation it's basically useless unless you make your call into my library your last and only call in your scene. thats not completely useless but it's not as versatile as i was hoping in the beginnig.

 

so, maybe we can salvage this. I'll post the library at the end of this post and maybe someone more experienced with LUA has an idea how to restructure/refactor this so it might work better.

 

The problematic HTTP code is in the activate()-function.

 

Oh, by the way...if someone finds it usefull in its current state, feel free to use it. It was my intent to give it to the community from the beginning.

 

regards

Max

 

[EDIT]: temporarily removed the library to include additional safety checks (read Peters post below)

[petergebruers 1,268]

Thanks for sharing. Maybe two small warnings... Running this on a device that's not a Aeotec RGBW bulb will result in destruction of the config (parameters) of that device. Also, your script forces the parameter set to a few parameters, I think it would be safer to read the parameter set, modify what you want and write it back. I am going to share a script that does this. I haven't published it before, because I think you have to be very cautious with modifying parameters. But because you published yours, I feel inclined to publish mine. It checks the device type and does read-modify-write.

This script tests if your device is a FGD211 with the right firmware. This script then sets "Max dimming level" to 25. There is no "undo" but this parameter seems safe to me

Please login or register to see this image.

/emoticons/default_icon_wink.gif" alt=";-)" />

 

Please login or register to see this code.

[maxdreamland 1]

Thanks for the advice! I'll include the additional checks and until then I'll remove the library in my post above so people don't screw up their configs. I'll repost it when i made it safer with your additions.

[LordOfFalcons 5]

wow. i wish i found this post a month ago. very clear explanation of my the http asynchronisity annoyance i've been struggling with!

 

however, somewhat annoying to see that this ~3 year old problem still hasn't been addressed properly...