Nonexisting device sending PUSH notifications - Fibaro devs any help?

[Neo Andersson 230]

Ghost Processes in the System – Unexplained QuickApp Behavior

For quite some time now, I, along with other users, have been experiencing so-called ghost processes in the system. These are processes running in the background, seemingly originating from nonexistent devices, or instances where some QuickApps load twice upon startup.

Initially, we suspected that this issue might be related to ER5 or some of @jgab 's libraries.

Due to these ghost effects, I have developed the habit of logging every action my QuickApp performs. This allows me to trace back events, identify which device triggered an action, and analyze system behavior.

Strange Push Notifications from Nowhere

Recently, I started receiving push notifications that don’t match reality. For instance, I have a QuickApp that monitors my garage door and sends an alert if the door remains open beyond a predefined time. However, I have been receiving alerts about the garage being open even when it is actually closed.

This prompted me to dig deeper...

I checked my system extensively, yet I found nothing that could be responsible for these alerts. My logs eventually revealed that Device 3860 was sending the notifications.

 

Please login or register to see this image.

/monthly_2025_02/image.png.36bb68636afb7945498c5d842497daf4.png" />

 

 

However, this device shouldn’t exist anymore, as it was my old QuickApp for the garage, which I deleted long ago.

 

Investigating the Phantom Device

I retrieved the logs and confirmed that Device 3860 was indeed sending the notifications. Naturally, I rushed to locate this device in the system, only to find that...

🚨 The device does not exist in the system! 🚨

To further investigate, I checked the Fibaro system logs and found that the notification was indeed sent at the reported time. However, the logs did not identify any sender device.

 

Please login or register to see this attachment.

 

Curious to see if I could still fetch information about Device 3860, I ran the following script:

 

Please login or register to see this code.

 

 

The result? NIL – meaning the system itself does not recognize this device!

What Is Going On?

So, my question is: How can a non-existent device still be running in the background and sending notifications?

This is not the first time ghost effects have been reported. I have created support tickets about these issues over a year ago, and yet, these problems persist.

And then we wonder why the system memory is constantly at 99%... 🤔

I assume a simple system restart might solve this issue temporarily, but that’s not the real solution. I would prefer Fibaro developers to investigate this properly rather than having to restart my system just to make the issue disappear.

Looking for Answers

Has anyone else encountered this problem? Any ideas on what could be causing these ghost processes?

I am attaching the logs in case someone is able to analyze them.

Thanks in advance for any help!

 

 

 

Please login or register to see this attachment.

Edited February 1, 2025 by Neo Andersson

[Tim__ 126]

Over the years I've read about people with ghost devices, which come about because a deletion hasn't worked properly.  The advice has been that support can remotely log in and delete them for you. (Would be nice if power users could do that themselves.)

 

Makes me wonder how many ghost devices I have lurking in my system!

[Neo Andersson 230]

Today i was monitoring the situation closely..so it is now for me a fact...

Some devices are not correctly deleted, so they still do their stuff in behind..

Todays logs from device 3860 (which doesnt exist in the system)

 

Please login or register to see this image.

/monthly_2025_02/image.png.5b7aaa7746eb54febf909156a52f5131.png" />

[Tim__ 126]

Good that you could isolate this @Neo Andersson.  Perhaps email fibaro support and get them to take a look in your system?  I'll be interested to know the outcome.

[jgab 2,642]

I don't think it's exactly the deletion that's the problem.

In the past we had ghost QAs appearing when QAs were (re)-saved. The old version of the QA continued to run and showed up in the console log (logging to the same TAG).

So, maybe that is what happened here. You resaved 3860, a doppelgänger was created and then you deleted 3860 making the doppelgänger live on...

 

A QA is defined by a device structure and one or more files stored on disk (or a database) and a Linux process running a Lua virtual machine.

1. When a QA is resaved (we make a change to a file), the file is saved and the process is killed and then started again.

2. When a QA is deleted the process is killed and the stored resources are cleaned up.

I would assume that something goes wrong with 1. and the process is not killed and a new is started. There could be all kinds of reasons depending on how they have chosen to terminate the running lua instance...

 

The interesting part is that there is no repeatable pattern of operations that causes this. When @Neo Andersson had issues in the past, the same QA posted to me did not cause issues on my system (creating doppelgänger when resaved). However, I have twice in the past seen this behavior myself with other QAs - where only a restart of the box got rid of the process.

 

So, it doesn't seem to be an issue with the QA itself, but rather the state of the HC3 - how loaded/resource constraint the box is. 

Maybe combined with a large QA (the QA @Neo Andersson had an issue with, ER5, is pretty large). So maybe the save-kill-restart steps are susceptible to some race condition due to box cpu load and QA size - and we end up with the old "zombie" process continuing to run.

In the case of 3860 where you deleted the QA it would be interesting to see in the system logs what happens when it tries to access or write to it's device (structure) state in the database - that doesn't exist anymore...

 

Edited February 3, 2025 by jgab