how to log scene events/variables for later debug/system health check

[AutoFrank 371]

Hi,

I am finally getting close to a decent level of automation around the house with lights, music, security, etc.

The challenge is that by its very nature a lot of decisions are being made by the scenes and a lot of events firing as a result.

 

I am looking for a way to keep track of some of the main variables and device states just to see that the 'engine' is working as designed/desired. I see as being helpful when something goes awry and I need to debug

 

I reckon there may be 15- 20 'things' I would like to keep track of and I can only think of a couple of ways to accomplish this...

 

Notifications - Popup service (

Please login or register to see this link.

) - I don't think this will work as I don't want a notification for each event (more like a summary) and I don't want to have to acknowledge each one

 

Global Variables

Log/append each one to a string stored in a global variable and after a number of entries dump out the content

Log event to a json encoded table) and read back / dump anything older than 24 hrs (this would take a bit of research/help but think I could do this with some help)

 

VD/Fibaro:log

perhaps create a VD and use fibaro:log() function t capture the data but I'm not sure if this is even possible

 

Are there other ways that people think would work or does anybody have anything like this already in place.

 

Thanks

Frank

[cag014 626]

I didn't' understand what exactly you're looking for... 

But may I suggest to create VD or scene that runs at loop every minute and checks if everything works/behaves as expected. You're the best person to know what to expect from your system, so you could create test conditions accordingly 

In case something went wrong it'll send notification by email - no acknowledge required.

 

Few  examples:

1. Since the Z-wave time to time stuck or doesn't response (I have this on few releases and same issue with current 4.090 release) , I'm monitoring Z-wave traffic and responsiveness.

2. Verifying if there are any dead devices.

3. Check motion sensors work as expected and not jammed.

and etc.

If anything goes wrong I send email and perform auto restart services. Till now it works perfectly.

Here a partial example of VD code which runs at loop every minute, but events verified at different time slots, every 10, 15, 90 minutes or any other value - see check Interval() function)

Note: all my device IDs are store as global json variable, you'll see a lot of them below, but it's just ID numbers. By doing that I don't need to change the code in case device has been re-included. 

Please login or register to see this code.

If it doesn't what you meant, please disregard. Again, it's just an idea.

[AutoFrank 371]

 

I didn't' understand what exactly you're looking for... 

But may I suggest to create VD or scene that runs at loop every minute and checks if everything works/behaves as expected. You're the best person to know what to expect from your system, so you could create test conditions accordingly 

In case something went wrong it'll send notification by email - no acknowledge required.

 

Few  examples:

1. Since the Z-wave time to time stuck or doesn't response (I have this on few releases and same issue with current 4.090 release) , I'm monitoring Z-wave traffic and responsiveness.

2. Verifying if there are any dead devices.

3. Check motion sensors work as expected and not jammed.

and etc.

If anything goes wrong I send email and perform auto restart services. Till now it works perfectly.

Here a partial example of VD code which runs at loop every minute, but events verified at different time slots, every 10, 15, 90 minutes or any other value - see check Interval() function)

Note: all my device IDs are store as global json variable, you'll see a lot of them below, but it's just ID numbers. By doing that I don't need to change the code in case device has been re-included. 

Please login or register to see this code.

If it doesn't what you meant, please disregard. Again, it's just an idea.

 

 

 

I think this may help me... thanks very much

- this is a very interesting way to tackle the issue of "is everything running okay" ?

Your code above is a little beyond what I'm used to but I think I get what you're doing and I'll work my way through it.

 

BTW - could I ask how you get all your devices into a global json variable ?

I was thinking about using this approach to log various device and scene events/status and dumping the result to an email every 24 hrs

[cag014 626]

Here is an example of my json scene code.

1. make sure to set Run Scene parameter to Automatic mode.

2. In my case I'm using "jTable" global predefined variable. It's very important to use global predefined variable and not just global variable.

Please login or register to see this code.

Now in order to get the table in scenes or VDs you need to have follow line at beginning of your code (converting stored JSON string to table). After that you can use the parameters from the table.

Please login or register to see this code.

To press button at VD

Please login or register to see this code.

As I said,  if from some reason your device ID has changed (re-included), you don't need to change the code, but you still need to update the device number at scene's header. (Fibaro doesn't support variables in header)

Please login or register to see this code.

Let me know if you need more info.

[AutoFrank 371]

 

Here is an example of my json scene code.

1. make sure to set Run Scene parameter to Automatic mode.

2. In my case I'm using "jTable" global predefined variable. It's very important to use global predefined variable and not just global variable.

Please login or register to see this code.

Now in order to get the table in scenes or VDs you need to have follow line at beginning of your code (converting stored JSON string to table). After that you can use the parameters from the table.

Please login or register to see this code.

To press button at VD

Please login or register to see this code.

As I said,  if from some reason your device ID has changed (re-included), you don't need to change the code, but you still need to update the device number at scene's header. (Fibaro doesn't support variables in header)

Please login or register to see this code.

Let me know if you need more info.

 

 

Thanks, this looks like a great way to handle all the ID's

I assume the first scene you included above is just run to initially load all the ID's and then if you update an ID ?

 

I've never worked with pre-defined variable (never knew what they were used for TBH)

How do I set up jTable

 

Please login or register to see this attachment.

 

thanks again 

[cag014 626]

1. Yes, the first scene is to load the IDs. By the way when you change any data and save it it changed on the fly (no nee to restart the system)

 

2. Yes, just save the variable as shown at attachment. To make it clean, after saving go back to edit and remove the second value (not must)

 

Since when the system reboots, it is not clear what the order of the execution of the scene, so you may see some initial error at scene (device not found or nil) if json scene ran after. Don't worry... it's OK and going forward no errors.

 

Another advise, after json scene executed, you should have the debug information of the table as a string. Please copy and paste that in jTable variable value. It helps in two situations:

 

1. When the system startup and the json scene not executed yet. (not must)

2. When you adding new global variable and push the save button (on the right of the screen), the jTable takes the value as appears in global variables panel. By the way in this case you must to run the json scene.

    My suggestion when you adding global variable, please do not use save button. The variable updated in any case.

 

Hope all above is understandable !

             

Please login or register to see this image.

/emoticons/default_icon_curve.gif" alt=":-/" />

Please login or register to see this attachment.

Please login or register to see this attachment.

[AutoFrank 371]

1. Yes, the first scene is to load the IDs. By the way when you change any data and save it it changed on the fly (no nee to restart the system)

 

2. Yes, just save the variable as shown at attachment. To make it clean, after saving go back to edit and remove the second value (not must)

 

Since when the system reboots, it is not clear what the order of the execution of the scene, so you may see some initial error at scene (device not found or nil) if json scene ran after. Don't worry... it's OK and going forward no errors.

 

Another advise, after json scene executed, you should have the debug information of the table as a string. Please copy and paste that in jTable variable value. It helps in two situations:

 

1. When the system startup and the json scene not executed yet. (not must)

2. When you adding new global variable and push the save button (on the right of the screen), the jTable takes the value as appears in global variables panel. By the way in this case you must to run the json scene.

    My suggestion when you adding global variable, please do not use save button. The variable updated in any case.

 

Hope all above is understandable !

             

Please login or register to see this image.

/emoticons/default_icon_curve.gif" alt=":-/" />

 

yep, most of it

 

I added the new perdefined variable called jTable and removed the second value

 

I create a scene with your data (will update with mine later) and ran the scene - I can see the output in the debug window and have copied it. (Presumable the string is stored in the value field of the jTable predefined variable)

 

The next part I'm not sure what you mean...

Do I create a 'standard' global variable (also called jTable ??)  and paste into that ??

 

and do I need to do this (copy/paste string) everytime I update the id data in the string

 

I'm not sure what the connection is between a variable called jTable and a predefined variable called jTable (if any)

 

Thanks again

Frank

[cag014 626]

No, no need to create additional variable.

Just if you can copy the entire string of json table in jTable value, that's all.

The reason for that, when the system reboots it takes the values from the global table, so till your scene will executed there are no values... 

And no you don't need to copy every time the table.

In general this is just an suggestion, the system will work fine without this actions.

[AutoFrank 371]

No, no need to create additional variable.

Just if you can copy the entire string of json table in jTable value, that's all.

The reason for that, when the system reboots it takes the values from the global table, so till your scene will executed there are no values... 

And no you don't need to copy every time the table.

In general this is just an suggestion, the system will work fine without this actions.

 

sorry still a bit confused

Please login or register to see this image.

/emoticons/default_icon_sad.gif" alt=":-(" />

This is what my predefined variable looks like 

 

Please login or register to see this attachment.

 

How do I 'copy it in ??

also when I run the scene to update the table does the value field of the variable not get to updated with the data from the scene anyway  and would not copying it in not simply over-write it

[cag014 626]

No need to copy..

[AutoFrank 371]

No need to copy..

 

sorry about the confusion and basic questions

I still have a lot to learn and thanks for taking the time to share and explain

 

Rgds

-Frank

[Lambik 192]

Sorry to bump in.....

 

Am I right if you're looking for a way to generate one message with multiple log-lines? I did some testing in the past and discovered that a Fibaro email message can not contain more then 18 lines or so. That's why I use Telegram for logging. That way I can receive one Telegram message with 50 lines (could be more though) of log-lines. The lines are separated (CR, next line) with %0A for Telegram. For email you have to use \n

 

I don't use scenes to get a trigger from devices, instead I use in the mainloop of a VD to check a global variable change:

Please login or register to see this code.

For devices change you can use:

Please login or register to see this code.

Be aware that the mainloop of a VD is executed once in 3 seconds. Try to avoid fibaro:sleep functions.

 

All lines I want to send to a log you can store it in an ascending table with table.insert, adding the CR at the end. To send the log just 'loop' the table.

 

For an example, check my signature link to the Presence Detector VD.

 

BTW, in the Presence Detector I do store a table to a global variable. The content normally gets corrupted when the the WebGui variable panel is saved. I solved this by capturing the error and reload the content. The solution you suggested looks a nice way to get around this panel-save-bug.

 

I hope I understood your questions, in the first posting, right, if not, just ignore me...

Please login or register to see this image.

/emoticons/default_icon_mrgreen.gif" alt=":mrgreen:" />

[AutoFrank 371]

Sorry to bump in.....

 

Am I right if you're looking for a way to generate one message with multiple log-lines? I did some testing in the past and discovered that a Fibaro email message can not contain more then 18 lines or so. That's why I use Telegram for logging. That way I can receive one Telegram message with 50 lines (could be more though) of log-lines. The lines are separated (CR, next line) with %0A for Telegram. For email you have to use \n

 

I don't use scenes to get a trigger from devices, instead I use in the mainloop of a VD to check a global variable change:

Please login or register to see this code.

For devices change you can use:

Please login or register to see this code.

Be aware that the mainloop of a VD is executed once in 3 seconds. Try to avoid fibaro:sleep functions.

 

All lines I want to send to a log you can store it in an ascending table with table.insert, adding the CR at the end. To send the log just 'loop' the table.

 

For an example, check my signature link to the Presence Detector VD.

 

BTW, in the Presence Detector I do store a table to a global variable. The content normally gets corrupted when the the WebGui variable panel is saved. I solved this by capturing the error and reload the content. The solution you suggested looks a nice way to get around this panel-save-bug.

 

I hope I understood your questions, in the first posting, right, if not, just ignore me...

Please login or register to see this image.

/emoticons/default_icon_mrgreen.gif" alt=":mrgreen:" />

Thanks Lambik

 

I like the idea of using a VD to capture state changes in GV and the telegram idea sounds like what I am looking for ...

 

-Frank