How to decrease Memory Usage. In my case - from 96% to 55%

[afendikov 37]

Hi All,

 

Last weeks I checked that my system works with high memory usage.

I start to export memory data to Influx and check it with Grafana and results are not good - system mostly use 95-96% and even after reboot it start from 85%

 

FYI - I have 111 Z-Wave devices and 27 QA's, some of them pretty big

 

So, I wrote little scene to optimize my system. 

  - disable all logs to history ( I use export to Influx and usually don't use HC3 history panel )

  - disable all Energy Panels reports

  - disable all devices which are hidden ( but have possibility to have some exclusion list )

  - disable all devices which are in special hidden room, which I call "hidden garbage" - parent devices are here and all unused childs also here

  - delete all history each run

 

Result - memory usage dropped from 96% to 55%. Not bad !

 

Scene here:

Warning - use it at your own risk!

-- ===============================================

-- write all hidden devices here which need to stay enabled. format: room/device name.
local exclusion             = { "desk/relay internet" }

-- room with all "garbage" devices
local hiddenRoom            = "hidden garbage"

-- set true if you don't need Energy Panel
local disableEnergyPanel    = true

-- ===============================================
function isInList(dev, list)
    for key, data in pairs(list) do
        if (tostring(dev) == tostring(data)) then
            return true
        end
    end
    return false
end

 

function setPropertyParam(id, prop, action)

    local data = {
      deviceId = id,
      propertyName = prop,
      value = action      
    }
    api.post("/plugins/updateProperty", data)
end

 

function disableDevice(id)
    api.put("/devices/"..id, { enabled = false } )
end

 

function transform(t)
    local res = {}
    for k, v in pairs(t) do
        local idx = v.id
        res[idx] = v.name
    end
    return res
end

 

-- scene, device, system, user, profile, unknown 
function deleteHistory()
    local timeNow = os.time()
    api.delete("/events/history?timestamp="..timeNow.."&objectType=device")
    api.delete("/events/history?timestamp="..timeNow.."&objectType=scene")
end

 

rooms      = api.get("/rooms")
rooms      = transform(rooms)

devices    = api.get("/devices")

 

for k, dev in pairs(devices) do
    deviceName = string.format("%s/%s", rooms[dev.roomID], dev.name);

    if dev.properties.saveLogs == true then
        print(dev.id.." is saving logs")
        setPropertyParam(dev.id, "saveLogs", false)
    end

 

    if dev.properties.saveToEnergyPanel == true and disableEnergyPanel == true then
        print(dev.id.." is saving to energy panel")
        setPropertyParam(dev.id, "saveToEnergyPanel", false)
    end

 

    if roomName == hiddenRoom and dev.enabled == true then
        print(dev.id.." is marked as hidden garbage")
        disableDevice(dev.id)

 

    elseif dev.visible == false and dev.parentId ~= 0 and dev.enabled == true then
        if isInList( deviceName, exclusion ) then 
            print(dev.id.." is hidden, but needed.. skipping for now")
        else
            print(dev.id.." is not visible, but enabled")
            disableDevice(dev.id)
        end
    end
end
deleteHistory()
print("Done...")
-- =================================================

 

to run it every day at 3:30 put in left ( Conditions ) tab:

 

{

  isTrigger = true,

  operator = "match",

  property = "cron",

  type = "date",

  value = { "30", "03", "*", "*", "*", "*" }

}

 

Please login or register to see this attachment.

[Sjakie 156]

Create backup without history

Restore backup

Memory usage will be much smaller.

Succes

//Sjakie

[RH_Dreambox 189]

And ...

Please login or register to see this link.

[10der 832]

Kostya, if you remove all QAs is memory grow?

another question 

if you export dump

in “process file” we can see CPU for each process and memory for each allocated process

you can see what for each QA fibaro core allocate memory from HC3 memory pool 

1 hour ago, Sjakie said:

Create backup without history

Restore backup

Memory usage will be much smaller.

Succes

//Sjakie

It’s not right to manipulate  with backups 

first of we should api for clearing history 

second we should have option(s) how we should keep history 

and last but not least why Fibaro load all history to memory?

[Sjakie 156]

@10der, this is the way Fibaro works.

backup without history (Cleans perhaps more history as being done in API?)

restore and wallah meme usage back as it was before.

Dont shoot me, hahah

Its helping me to get mem down

//Sjakie

[m.roszak 763]

4 hours ago, 10der said:

and last but not least why Fibaro load all history to memory?

We are not
Some of the things are being loaded to RAM but we are working on it to remove this entirely. 

[afendikov 37]

3 hours ago, Sjakie said:

@10der, this is the way Fibaro works.

backup without history (Cleans perhaps more history as being done in API?)

restore and wallah meme usage back as it was before.

Dont shoot me, hahah

Its helping me to get mem down

//Sjakie

I'm truly believe, that much better way not to generate extra garbage as first priority

It will save some CPU time too, not only memory

 

And scene can work in automatic mode and back-up and restore needs to be processed in manual mode, because some times Fibaro can go to Recovery console after restart.