HC2 Energy Consumption data just disappeared. What to do?

[MonkY 9]

Hello everyone!

As i have posted before, i am having hard times with my electric energy provider, and i have used the Fibaro data to compare the consumption usage.

Today i had to send the last month usage to the provider, but i was shocked to see my Fibaro Home Center 2 it's only showing data since December 24, 2018. And i needed the data from at least December 12, 2018. Plus, i was used to compare monthly usage... which is now impossible!

 

What in the world happened? How did all the data before December 24 disappeared? I had years of consumption data in there. I thought Fibaro was a reliable smart home system...

 

Please help me fix this. Is there a way to recover my lost consumption data?

 

Thank you in advance!

Please login or register to see this attachment.

[Thomasn 77]

Did you update your system just before the 24th of December? 

[MonkY 9]

It is possible to be the update from 4.510 to 4.520, but i am not sure.

Either way, i need my data back...

[DBETestLab 0]

Hi @Monky. Do you usually export this data? Could you tell me how? Thanks.

[MonkY 9]

DBETestLab: no, i don't usually export this data. I just write down the values every month.

 

Anyway, it seems it's not a problem of losing data after an upgrade. I just lost more data, and i haven't upgraded anything. 

It looks like Fibaro now only keeps last week of data or something. And that's really... really bad!

How can i fix this? Is there a setting for how long to keep the energy consumption history?!?

Please login or register to see this attachment.

[MonkY 9]

What in the world is going on? It seems like the energy usage data just keeps deleting itself after a period of time.

Last time i checked, i could see the data from January 19. Now i can only see it from January 28.

Is there a way to fix this?

 

P.S. The Fibaro tech support is really bad! They didn't even bothered to reply to my emails...

Please login or register to see this attachment.

[MonkY 9]

Well, i finally received an email from Fibaro. And my ticket status already changed to "Solved" (wow!). Here it is:

 

Quote

Hello.
I am really sorry for that.
Most likely due to heavy amount of readings Home Center is currently rewriting the readings in order to keep limited database tidy. Strongly suggest you to decrease frequency of reading using parameters starting from 50 for Fibaro Dimmer, starting from 40 for Roller Shutters and 50 and higher for Fibaro Switch.
I hope it helps in any degree.
Have a good day.

 

Now, that's a little vague for me, specially since i don't have too many Fibaro Dimmers, no roller shutters and no switches. But i have about 20-30 Fibaro Plugs.
I have noticed the following parameters:
 

Quote

Parameter 11 Standard power report

This parameter determines the minimum percentage change in active power that will result in sending power report to the main controller
Compared to parameter 10, the maximum number of reports sent is reduced (parameter 12) to 5 in a specified time interval. In addition, the reports are not sent in mode, which may result in overloading the Z-Wave network. 
By default changes in power load may be reported up to 5 times per 30 seconds, when power load changes by 15%.

Set at: 15%

Quote

Parameter 12 Power reporting interval

This parameter defines the time interval of sending at most 5 standard power reports when the power changes by the value set in parameter 11. The higher the value of parameter 12, the fewer reports the device will send.

Set at: 30s

Quote

 

Parameter 13 Energy reporting threshold

This parameter specifies the minimum change in energy consumption (in relation to the previously reported), that will result in sending a new report.

Set at: 0.1KWh

Quote

 

Parameter 14 Power and energy periodic reports

This parameter defines time period between reports sent when changes in power load have not been recorded or if changes are insignificant. By default reports are sent every hour.

Set at: 1h

I could try to change these to:
Parameter 11:  25%
Parameter 12: 60s
Parameter 13: 0.2KWh
Parameter 14: 2h

Would that be much better? Also, is there a way to change all my 30 Fibaro Plugs at once, and not having to modify each and every one of them manually?

[Guest PeterCogen]

Just as I was beginning to feel reasonably happy with the HC2 I bought after struggling with the HC Lite for one year...

 

I ran into this same problem. Energy data disappearing after approx 40 days.

 

What exactly are developers thinking when they decide to "delete power consumption records to keep the database tidy" without clearly documenting this feature for the end user ?

 

Controlling consumption is among the core functionalities of home automation.

 

You cannot just decide to throw away valuable user data at undefined intervals without any warning !

 

A simple solution would be to automatically perform some kind of data aggregation (eg daily, weekly, monthly kWh data) after a user-definable period.

 

That way, users could see detailed info for recent events (last week or last month) and have summary data for a much longer period.

 

 But the very first thing to do is certainly to mention the current behaviour on the Energy Panel.

[MonkY 9]

I agree. For me, controlling power consumption is a must have feature. Unfortunately, it's almost unusable on Fibaro as it is. I even disabled about 10 of my Fibaro Plugs, and i can hardly get the latest 20 days or so of power consumption.

I hope they can increase the database for power consumption... or maybe let everyone choose the period they want to keep data for.

If i wouldn't have invested so much in Fibaro modules, i would probably give up on them.

But, i still hope...

[Markus2801A 2]

Hello experiencing same problem over the past years on my HCL!

 

Is there any solution available? Maybe exporting energy values periodically to avoid data loss?

There enough storage capacity but HCL frequently deletes energy data ;-(

 

would it be possible to include HCL HC2 via raspberry an open hab or iobroker to fetch energy data and log them within the SoC?

 

 

As far as I have read, you need  Fibaro to keep your devices up to date, so completely replacing the fiber gateway with a raspy is not possible?!

 

Kind Regards!

Markus

 

Please login or register to see this attachment.

Please login or register to see this attachment.

[MonkY 9]

Things are getting worse, and Fibaro does nothing to fix this issue. My HC2 can hardly keep 2 weeks of consumption data. This is ridiculous! 

Please Fibaro, just increase the amount of storage for consumption data!

[Markus2801A 2]

Very disappointing! Fibaro has not fixed this issue and its up since years! What a shame!

[Al3x 2]

I thought this was just me i had years worth of data and then decided to update and boom its all gone!!?!?!??!?!?!!?!?!??!!

[Markus2801A 2]

Thats absolutely unacceptable! I decided to go for another method to collect energy consumption via an EnergyMeter (ModBus) and RaspberryPi 4. With this setup I "only" can measure and log complete energy consumption of my home and not every connected device. But I think this setup will keep my data is more stable and reliable!

[Guest PeterCogen]

I've decided to transfer the information at the end of every month to an Excel sheet.

 

Takes time, but at least I can analyze trends over a longer period, this way.

 

For what it's worth, I repeat the suggestion that the HC2 should keep "light-weight" aggregated data for two or more years and store the details for a short period only (one or two months).

 

For debugging and/or event monitoring, a user may want the short term details. For power consumption analysis, the aggregated date will be more than enough.

 

That would allow for a reliable AND predictable behaviour.

[ReneNL 48]

See also for example the following post if you want a solid option to capture all your sensor data. It also allows you to make your own dashboards, customize, aggregate, etc.

 

 

[Markus2801A 2]

35 minutes ago, ReneNL said:

See also for example the following post if you want a solid option to capture all your sensor data. It also allows you to make your own dashboards, customize, aggregate, etc.

 

 

Thanks for the hint!

I also use Grafana and influx to log energy data in my gardenhouse and will setup this also for my main house with an ABB/Janitza B23 energy Meter.

The above described solution with the python script only works on HC2, unfortunately i don't have one, I use the HCL.

[MonkY 9]

3 hours ago, ReneNL said:

See also for example the following post if you want a solid option to capture all your sensor data. It also allows you to make your own dashboards, customize, aggregate, etc.

 

 

Great. I will give it a try. But is there any tutorial, and a list with what i need to acomplish that lovely chart? 

Thanks in advance!

[ReneNL 48]

I have been too lazy to write up a proper tutorial... Basically you only need a lightweight server (I use a raspberry pi 3) as hardware. On it, install influxdb (free, see influxdb site for steps on how to set it up), install grafana (free, see grafana site for steps) and regularly run the python script (use crontab) to pull the data from the HC2. In my post above you find further info in the trail.

 

See also post below for a further tutorial/info and an alternative python script.

 

If there is further interest I could see if I can find time for a proper tutorial.

[MonkY 9]

I wonder if the new Home Center 3 will have the same limits. If so, I think I'm done with Fibaro...

[Markus2801A 2]

39 minutes ago, MonkY said:

I wonder if the new Home Center 3 will have the same limits. If so, I think I'm done with Fibaro...

hopefully the limit will be gone and this device is working properly!!!

 

Any news on availability and working experience with HC3?