Wishlist for Lua development environment improvements (Anno 2022)

[jgab 2,052]

Ok, here is my Wishlist for improvements of the Lua development environment. Mostly concerning QAs. Some of them are really easy to implement and are really bug fixes...

In general, I think the QA system is a huge leap forward (been coding since 2012), but it's still quite frustrating at times.

In no particular order (will update the list as they come...)

Btw, if you have a problem with your QA or Scene it may be worthwhile looking through this list if it's a known problem  

 

-----------------------------------------------

1. Add error messages to setTimeout and setInterval. Wrap the function in a pcall and log if there are errors. A lot of posts on this forum are from confused users that see the code crash silent. It's a really easy improvement. Do it also with the callbacks in net.* functions. Seen similar (silent errors) behaviour there.. Yay, we got this in 5.151.47

 

2. Wrap json.* functions with a pcall too and throw error(...,2) to give the user a line number from where the json function was called. Now we get a line number deep in json where it crashed that is not very useful. Oh, and use the same json library for both QAs and Scenes (see also point 16) - the difference is annoying ,ex null handling, sparse arrays etc.

 

3.

Please login or register to see this link.

. That's how events work in real system... Now they are severely limited in use. 

With custom events carrying value we could also solve the issue with starting scenes with parameters - something that people from HC2 are missing. Just make the scene trigger on the event and the parameters are the event's value...

 

4. Give us Lua's coroutines. They are extremely light-weight and safe in version > 5.1. Then we could provide "helper" functions to let users program more in a synchronous way for http request etc. without blocking the code. A lot of trouble in the forum for people to get their head around asynchronous callbacks that could be avoided. (and of course, start QuickApp:onInit in a coroutine). Home automation programming is a highly asynchronous problem, but we can hide some of the complexity if we have coroutines.

 

5. Enable metatables. It's really useful for code abstractions and because it's a separate process for the QA, if something gets f*d up it won't affect others. plus, a lot of code on the Internet that would be nice to use, uses metatables and are a pain to convert if even possible. Thanks, we got that!

 

6. More libraries. LPEG and some crypto/encoding libs, stuff that benefits from C/C++ backends, and something for OAuth would be nice. 

 

7. Some kind of user/QA defined http ingress point to be able to work with external systems with web hooks

Ex. In a QA.

   self:registerIngress("/foo",callback)

would allow incoming requests to   "http://<IP>/api/device/<self.id>/ingress/foo"

supporting both GET/PUT/POST/DELETE and the callback would receive the data/parameters/headers.

The API seems to be fuelled by Nginx, so it should be easy to add this dynamically and when the QA is stopped/deleted the ingress is removed

 

8. More, and more flexible, UI elements - it's 2022 and the QAs are not pretty and very limited in the way they can interact with users.

 

9.  With coroutines we could even make

Please login or register to see this link.

so that it could return a value from the called QA, and it can be made backwards compatible/optional. I believe this could improve life considerable for not so experienced, and experienced developers. I do believe that the

Please login or register to see this link.

.

 

10. Enable

Please login or register to see this link.

so that we can do remote debugging of the code. Other embedded systems uses it , and it's 2022 and developers expect better tools. Only be enabled by the HC3 user per QA for security reason(?). Why not have a HC3 developer mode that supports some more debugging libraries and is not enabled in production system? Why not even integrate with VS Studio or Zerobrane?

 

11. Some way to really share code. We have QuickApp files. Why not shared QuickApp files ? A separate place where shared files can be stored and can then be selected in the QuickApp file panel in each individual QA.

 

12. One of the great things with the Fibaro framework is that everything is represented/accessible via the REST API - kudos for that. Please continue with that. So, what happened with fibaro.getSceneVariable and fibaro.setSceneVariable, no API ?

 

13. Why can't Scenes be downloaded and uploaded?

 

14. Minor nuance. QuickApp:getVariable should return nil when the variable doesn't exist. Not "". Now we can't tell if user set the variable to "" or hasn't set it at all (can actually matter). I assume Fibaro want to be helpful but it doesn't really help and break Lua coding conventions. If people expected an IP number from the variable and it returns "" instead of nil, it's of no help. AND we get a warning in the log anyway if it doesn't exist(!) 

 

15. Fix the

Please login or register to see this link.

from scenes and QAs. Now it has a tendency to mangle arguments. string "66" becomes number 66 etc. It's a real pain. Solution is to json encode the whole argument list table at caller and unpack the table at receiver.

 

16. Have the same Lua version for Scenes and QAs. Why 5.3 for QAs and 5.2 for Scenes??. And keep the same fibaro.* api implementation. Separate out Scenes specific stuff like how it's done for QAs.

 

17.

Please login or register to see this link.

? (recompile for 64 bits?)

 

18. Choosing to compile the HC-lite Lua with 32 bits makes it a mess to write portable code. os.time() barely fits in 32 bits and typically fail if you do arithmetic with it. Some values returned

Please login or register to see this link.

...

 

19. Some callback mechanism for events in QAs. Like refreshStates but maybe refine it to a sourceTrigger level? It can be done with Lua in the QA today using refreshStates but it would be useful to provide some filter on the callback, so the system filtered before sending to the QA, aka less stress on the QA and no repeating calls to refreshStates. In fact, it would probably be useful to add what types of events (filter) is of interest to the /api/refreshStates call too. Well, we got refreshStateSubscriber() as a wrapper around /refreshStates, but we would like to have source side filtering - now currently burning as many cpu cycles.., 

 

20. Add an MQTT broker to the HC3. There are light weight brokers out there. Would simplify some MQTT use case and we may even start to use it between QAs?

 

21. Let us have access to Lua's debug.* api  in QAs (or at least if development mode enabled?). Could write our own small tracers and profilers. Once again, if QAs have their own processes, what's the problem?

 

--------------------------------------

 

People in the forum have other wishes? (concerning Lua coding)

Edited January 9 by jgab
spelling errors...

[jgab 2,052]

Added some more points to the list and clarified others...

Edited July 17, 2022 by jgab

[petrkl12 214]

@jgabwhat about to add native support for events without repeating calls of 

Please login or register to see this code.

 

[jgab 2,052]

3 minutes ago, petrkl12 said:

@jgabwhat about to add native support for events without repeating calls of 

Please login or register to see this code.

 

Yes, let me add it to the list

[jgab 2,052]

~80% of this list could be done in an afternoon.... and 100% could be made backward compatible to not break existing QA/Scenes... and it would result in 200% improvement in productivity, quality , and functionality of QAs... and a 300% increased developer satisfaction.

Edited July 17, 2022 by jgab

[Sjakie 138]

If Fibaro will dom this the effect will be that many users wil feel themselves "a happy camper" by using HC3!!

I hope they will response.

Thanks JGab!

[Bodyart 231]

Hi Jan @jgab,

great wish list! I hope Fibaro will not only respond to this, but also realize at least some of them!

[Neo Andersson 184]

On 7/17/2022 at 12:02 PM, jgab said:

~80% of this list could be done in an afternoon.... and 100% could be made backward compatible to not break existing QA/Scenes... and it would result in 200% improvement in productivity, quality , and functionality of QAs... and a 300% increased developer satisfaction.

And please give us the possibility of encrypting the LUA code, or hide it, when selling the system.

[precioz 1]

Yes Yes Yes

13. Why can't Scenes be downloaded and uploaded?

I would like to join - I really miss the TXT scene documentation. I need it to save documentation of the state of the scenes to date. I submitted the proposal:

Please login or register to see this link.

Please consider the proposal.

 

 

Yes Yes Yes

12. One of the great things with the Fibaro framework is that everything is represented/accessible via the REST API - kudos for that. Please continue with that. So, what happened with fibaro.getSceneVariable and fibaro.setSceneVariable, no API ?