Z wave monitor to prevent freeze

[cag014 585]

Hi all,

I see that some users have downloaded

Please login or register to see this link.

script and I think it could be a good idea to share our data and know-how to optimize our Z-wave performance.

In my case I have manage to reduce Z-wave traffic from average of event per 1.7 sec. to 4.8 sec (24 hours monitoring, 78 physical devices, 382 IDs), but I don't know if this is a good number. It will be interesting if anyone could share  an average of his system. That way we can compare and might be to achieve the right number and stable performance which may be could prevent Z-wave freeze in the future.

I think average of Z-wave traffic somehow depends on number of the devices in the system and again I believe we need team work to find correct formula for that.

Few users have shared with me that they have found devices, which "bombarding" the traffic with unnecessary reports and they fixed the issue.

It could be very helpful to all of us to share our solutions and fixes...  we all can learn from others

If you think it could violate your privacy,   please ignore this message...

Thank you

 

 

[petergebruers 1,231]

@cag014 you have a point. That parameter 43 does not increase periodic reporting but it can limit it. The description of the parameter says exactly what it does and I can confirm that this explanation is correct. It limits the number of reports to 5 per interval and 30 s default.

 

If the user has a very small load (0.1 to 1.0 W) the system might indeed report frequently, but not exactly once per 30 seconds.

 

If this plug reports frequently without any load, then this parameter applies:

 

Parameter 47

Time period between reports on power load and energy consumption.
Parameter defines time period between reports sent when changes in power load have not been recorded.

 

But the default is 3600 seconds.

 

Lets dig a little deeper...

 

22 minutes ago, perjar said:

when there is no load nor any change in power consumption.

 

AFAIK the Fibaro Wall Plugs do not send many power reports when the load is zero (see p 47).

 

But they can send many updates (see other parameters) if "the load does not change" - it depends what you mean by that.

 

For instance... If you have a very low load like a phone charger with no load, the plug might register 0.3 W then 0.4 W. That is a small change, but relatively speaking it is 33% so it triggers a report. It has been reported for example for low power LED lamps and printer(s) in standby.

 

My espresso machine pulses its boiler on and off, it reports many changes per minute because of that...

[petergebruers 1,231]

18 hours ago, perjar said:

Even if a light looks like it is glowing steadily, the power consumption can easily fluctuate between say 1,4W and 1,7W which is 21%. So, working with the power reporting settings is critical if you want to avoid flooding the system with power updates.

Exactly!

 

18 hours ago, perjar said:

The good news is that with the tweaks mentioned above, the number z-wave messages has taken a deep dive

I'm glad we could help... You can show your appreciation by occasionally clicking on a like or thanks button on one of the earlier posts - if you think that is deserved. I am 100% end user and earn no money with this, but "likes" make me smile...

 

18 hours ago, perjar said:

I have a few other devices that generate too much traffic (power meters), but they are of my own design, built on the z-uno, so I can't really blame anyone else there.

Ah, another Z-Uno lover! It's probably an interesting challenge to come up with a universal algorithm that uses fewer packets but still makes your scenes run as expected. For example, the boiler of my espresso machine pulses on 1 second, off 1 second, then on 1 second and 5-8 seconds off...

[drboss 107]

It's good idea, i'm in optimization process now. I things on end of week I will finish and of course I will share my results before and after. Thanks again for your program, with them is much easy to do this and more complex that only with USB z-wave connected as secondary controller (I use it  ex. for faster check if any association not presented by HC2 exist in network).

btw optimization process, form me the one of not resolved problem is rapports event (energy) from devices like TKB, they raports all very small change of measured value. Any chance to change by parameters for ex minimal interval between rapports.

Edited December 27, 2018 by drboss

[cag014 585]

@drboss

Thank you for join forces

Unfortunately I don't have any TKB devices, but if you could provide the device model I'll try to find solution... two heads better than one.

Meanwhile I have found below parameters set  -  might be not exactly your model, but usually same company uses same parameters set for different models

Please login or register to see this spoiler.

Looks like parameter 3 is the one you need to change

Edited December 27, 2018 by cag014

[petergebruers 1,231]

8 hours ago, drboss said:

problem is rapports event (energy) from devices like TKB, they raports all very small change of measured value.

This was confirmed to me for some TKB devices, via PM.

 

I don't know the specifics, some companies seem to care more (or less) about proper documentation...

 

You can try:

 

Please login or register to see this link.

 

I will also invite you via PM to test one of my private scripts, not ready for public release, aptly called "sanity check".

[Comfortica 16]

Don't forget the greenwave powernodes... They have a history of zwave network pollution

[petergebruers 1,231]

9 minutes ago, Comfortica said:

Don't forget the greenwave powernodes... They have a history of zwave network pollution

Confirmed... And I am not sure about the parameters. There are different revisions of the device and there are documents, floating on the internet, that mention certain parameters but don't apply to all revisions.

 

But basically any power reporting device can be "noisy" ar "spamming" if it has a parameter based on % change and is connected to a low load like LED or a device with low and fluctuating standby. For example, I have a plug that sends 0.3 then 0.4 W because by that is a 33% increase...

[drboss 107]

Yes the parameters is the same but option 3 and 4 is time period if any change in power no occurs. But if power network change voltage (1,2,3 volts in small period of time or connected device to outlet change consumption continuously like washing machine, dishwasher) then TKB products send to controller every change (1V or 1W or less!). Any parameters like in Fibaro Switch to set minimal interval between update or set minimum change of value for reporting (by percentage or value).

Last night my dishwasher send 800 update in 1 hours when work, on standby it send +/- 200 update by hour (power consumtion around 2.3W but change +/-1 1W ever and again

Edited December 27, 2018 by drboss

[petergebruers 1,231]

12 hours ago, cag014 said:

I think average of Z-wave traffic somehow depends on number of the devices in the system and again I believe we need team work to find correct formula for that.

 

I am on holiday, so please forgive me my terse comments. I have to be brief. We recently talked about "burstiness" and I did not give you a clear answer. I'll give some hints right now...

 

• The basic rule IMHO is "lower is better". I fanatically reduce the number of reports (packets, events). It limits collisions, CPU, ... everything!
• As a rule of thumb... A controller can handle maybe 10 - 20 (maybe 50 peak) message exchanges per second, assuming there are not any collisions. I think many users don't know how slow it actually is, because we kind of get blinded by the spec of the signal speed, being for recent devices on a HC2: 40 kbit per second.
• As a rule of thumb... Starting a scene takes about 250 ms and I would say a HC2 cannot start more then 5 - 10 scenes per second.
• If your scene toggles devices, all those devices report state and possibly power, voltage change, ... It depends how you write your scene, the device, the load, parameters, how many hops are used. Many of my scripts have a sleep(250) between commands. I wouldn't say you always need one, nor should it be 250 ms, but it is worth trying if specific scenes cause trouble.
• When in doubt, Fibaro support should be able to look at Z-Wave logging but that log is not accessible to end-users. I don't have to tell them how to do their job, I am only telling this because the possibility exists. For end-users, the only option is a Z-Wave sniffer. The "event log" and "power history" only partially represent real traffic...

 

 

Edited December 27, 2018 by petergebruers

[Theo 6]

Thank you @cag014 for these nice scripts. Your documentation a bit (too) compact. Maybe we can work on some more extended documentation. It would also be nice to have some author referral ("published by @cag014 on forum.fibaro.com at 21/12/2108").

When I use the start button to switch mode, in first instance I get the message "something went wrong" and some seconds later a report will show anyway.

As an immediate result I searched for a parameter to lower the DS18B0 temperature readout cycle through a universal sensor from 20 to 120 seconds.

[Theo 6]

My results after three hours:

[DEBUG] 19:56:13: Home (HC2-034000) Z-wave traffic summary report.

|  id  | total # | total % |  value  |  power  |  unit  |  energy  |  parameters  |  offset  |  color  |  Device description       
|   33 |  59     |  45.39  |       1 |      30 |        |        4 |              |          |      24 |
|   83 |  28     |  21.54  |      28 |         |        |          |              |          |         | 
|   62 |  22     |  16.93  |       5 |      15 |        |        2 |              |          |         |
|   47 |  10     |  7.7    |       4 |         |      6 |          |              |          |         | 
|   84 |  6      |  4.62   |       4 |         |        |          |            1 |        1 |         | 
|    5 |  2      |  1.54   |       1 |       1 |        |          |              |          |         | 
|   48 |  2      |  1.54   |       2 |         |        |          |              |          |         | 
|   58 |  1      |  0.77   |         |       1 |        |          |              |          |         | 

Z-wave events = 130
Devices Num.  = 8
Sample time   = Sun Dec 30 16:56:26 2018 - Sun Dec 30 19:56:03 2018
Elapsed time  =   2:59:47
Avg. traffic  = 82.9 seconds.

 

[barend121 13]

after 20 hours

 

Z-wave events = 21658
Devices Num.  = 105
Sample time   = Mon Jan  7 22:54:03 2019 - Tue Jan  8 19:37:52 2019
Elapsed time  =  20:43:52
Avg. traffic  = 3.5 seconds.

[cag014 585]

Thank you. It gives me some reference for my system. Looks like we're have same traffic and load.

By the way, I have uploaded new version which includes graphical chart of the system behavior over a time. Now the table is actually HTML table, makes  easier to navigate.

It is still waits for Fibaro approval....

 

[cag014 585]

On 12/30/2018 at 2:05 PM, Theo said:

Maybe we can work on some more extended documentation

@Theo

Have released new version (already available in download center) which includes charts for devices, scenes and user defined view.

I think this one probably needs more extended documentation, any help from you will be very appreciated.

Thank you

[Theo 6]

8 hours ago, cag014 said:

@Theo

Have released new version (already available in download center) which includes charts for devices, scenes and user defined view.

I think this one probably needs more extended documentation, any help from you will be very appreciated.

Thank you

@cag014: see PM

[alex88 68]

I'm running a cut down system and I'm in the process of expanding and have turned a lot of old scenes etc off as lot of devices were removed with new ones added back, but results seem similar to yours and cag014.

 

Z-wave events = 26982
Devices Num.  = 111
Elapsed  time   =  22:00:32   [ Jan 25 23:41  -  Jan 26 21:42 ]
Avg. traffic  = 3 seconds

 

I was recording peak traffic of 0.3 seconds to 0.6 seconds, but I guess overnight it drops to a very low number to bring the average up a lot.

 

EDIT: I posted the text below here: 

 

 

"My next task is to figure out what is a "normal" range of commands for a device.

I have 3x Fibaro RGBW that are taking up over 35% of my zwave traffic. One of these devices I deleted and added back to fix an issue I had with zwave network comms not going through, however I'm not sure whether it's normal for Fibaro RGBW modules to take up so much traffic. They each reported around 4000 commands, in the total # column, on your zwave viewer in less than 24 hours.

I'm tempted to delete the other 2x Fibaro RGBW modules and add them back, but it takes time updating code etc and given the deletion one of the three didn't reduce the traffic, I'm not sure what good it will do.

My other devices range from 10's to 1000 of commands in the same period of less than 24 hours. With most devices falling below 250 commands.

Can you or anyone share what is a normal range for the number of commands per device per 24 hours?"

 

4,000 commands for about 22 hours for an RGBW sounds very high from looking at other peoples traffic.  I will need to look into them to see what can be done, but I don't recall changing any of the settings, so not sure how to reduce this.

 

The next issue is working out what is calling each RGBW 4000 times in less than a day and why. I suppose that is one that only Fibaro support can provide an answer to....

Edited January 27, 2019 by amilanov

[Momos 98]

Ran it for 5h.. guess the results are pretty good.  

Elapsed time = 5:01:47 [ Feb 01 09:57 - Feb 01 14:59 ]

Sample log rate = 5 minutes. Total samples 60
Z-wave events = 1899 on 62 devices.
Average traffic = 9.6 sec.
Z-traffic range = 5.6 - 30.7 sec.

 

24h test next  

[D-Vine 64]

Result after 4.44 hours

 

 

Please login or register to see this code.

 

 

Please login or register to see this code.

[T.Konopka 7]

@cag014, you have the thread for your file here:

No need to gather feedback in two places

[cag014 585]

@T.Konopka

OK, got it

thanks