auto::cerr – Automatic Capture of Errors on HC3

[Łukasz997 117]

I hope that it sometimes happens to miss an error from a scene or QA in a deluge of messages in the web console. Scripts can run for months and have branches of code that are rarely visited. Such errors can be easily missed – especially because a QA restarts after a failure and “nothing obvious” shows up later. Still, it does not mean that nothing bad happens… And what about errors at night? Or when you are on holidays? Just because is after 1st of May? I read on this forum that user had over 350 (!) QAs & Scenes running in parallel - best chance to miss something important.

 

So, I would like to introduce auto::cerr – Automatic Capture of Errors. auto::cerr captures all error messages from www console, fingerprints each error, throttles repeats, and pushes concise alerts to your phone (plus optionally to Virtual Console, if installed). It also keeps a compact, color-coded history inside the QA UI and sends periodic summaries during “error storms”.

@cag014 advised me to capture errors this way. Thanks.

 

Features summarized:

• Catches every logged error. Tracks errors from QA, setTimeout & setInterval, scenes, explicit error(…) (terminates QA), and self:error(…) (non-terminating). Bases on api.get "/debugMessages".
• Immediate push alert. Sends a normalized message (HTML stripped) to the admin phone (or another chosen user) with shortened error text, time, origin. You will never miss an error again.
• Virtual Console integration (optional). Emits a richer log line to your vconsole QA (full text, emphasis, counters). Works standalone too, but the console gives you better forensics – and is recommended (because auto::cerr prints very little to the web console). Virtual console is similar to www console but works with Yubii. Description and download:

  Please login or register to see this link.

• Compact on-device dashboard. Presents statistics of captured errors in the QA interface, with automatic deletion after a defined time (so the table does not grow infinitely), color representation of error age (fresh -> old), and more. You can view all errors in one place within a customized time window. Also on the phone.
• Summaries of errors. Emits a summary of every error that meets a criterion (user defined, e.g., occurs more than once in 10 min) – to phone and vconsole. Tracks which errors are summarized and does not include them again (until a long time gap occurs).
• Storm detection & smart throttling. Incorporates a sophisticated mechanism to:
  • automatically detect an error storm from any source (many errors one after another due to QA auto-restart)
  • prevent the user’s phone from being flooded with error messages. Only one message is pushed. Configurable rules suppress repeated messages with dynamic (non-linear) relaxation based on the real situation.
  • there are two user defined time-windows: one for fast storm detection and one normal (o.e. 3 errors in 5 min OR 8 in 10 min to detect storm)
• DST-aware operation. As any time-driven app is sensitive to DST time changes. Automatically detects DST flips and, per your custom policy, either:
  • corrects internal data structures to reflect the new time (experimental, not fully tested), or
  • restarts the QA. Other sources of time drift are ignored (time server failure, manual changes).
• Auto-selects push receiver. “admin” is automatically selected as the recipient of info. Others can be chosen manually (by design – only one).
• Blacklist. Manages a list of QA or Scene IDs to be completely ignored (no vconsole, no stats, no push).
• Mute/disable controls. Framework can be muted for 30 min, until the end of the day, or completely disabled.
• Hash token. Generates a short hash token (from text+origin) for each message to easier reference to the same errors (shown with push and vconsole info).
• Works out of the box. No configuration required – defaults work well.

 

QA for download and additional info in next mail.

 

Please login or register to see this image.

/monthly_2025_09/image.png.7f9ef798aaf6832ba366184c7619d194.png" />

[Łukasz997 117]

UI Table Columns explained
1. ID - Numeric identifier of the source (QuickApp ID or Scene ID). Scene IDs are shown in orange.
2. Message - Normalized error text (HTML stripped, white space cleaned). Truncated to configured length (CONF.infor_char_len).
3. Str (Storm) - Indicator of whether this error is currently in a storm mode. ○ = not in storm, ● = in storm.
4. Sm (Summary informed) - Marks whether a summary has already been sent for this error. ○ = none yet, ● = already summarized.
5. # (Total count) - Total number of occurrences of this error since first seen.
6. Sup (Suppress window) 

• Current suppress duration in minutes for this error.
• Controls how soon a repeat error can trigger a new alert (on phone, vconsole).
• Grows adaptively with frequent repeats, relaxes with half-life during silence.

7. LastT - Time of the last occurrence (HH:MM). Annotated with a small superscript:
        ᵀ = today
        ᵞ = yesterday
        ᵞ⁺ = 2 days ago
        ³⁺ = 3+ days ago
    Background color shows how close the entry is to being purged (green → yellow → red).

Please login or register to see this attachment.

Edited September 4, 2025 by Łukasz997

[Łukasz997 117]

FAQ

Q: Error messages are not reflected in auto::cerr immediately, why?

A: It scans for new messages every 60 secs. As additional protection it has a 1 sec step-back not to miss "on-second's-edge" messages. From technical point errors generates an event in HC3, but not all. Scene and error inside setInterval and setTimeout doesn't. That's why periodic scanning is needed.
Q: Does it catch LUA error(...) statement from user code? Or self:error(...)?

A: Yes. error(...) (terminating) and self:error(...) (non-terminating) are both tracked as soon as HC3 logs them.
Q: Can I silence a noisy QA?

A: Add its ID to the blacklist. It will be fully ignored (UI, push, vconsole).
Q: Will I get spammed during after-QA-error infinite-restart loops?

A: No - storm detection + suppress window ensure only the first push per storm, and one summary.
Q: And after one push I will never receive same error?
A: You will, but after a time gap between identical messages. Suppress time will dynamically change - after user-defined time it's divided by two; as long as message suppress > base suppress time (configurable).
Q: What happens after very long silence with UI-listed messages?

A: The UI auto-colors “age” and auto-purges old entries after configurable time (dafault: 24h). When you enlarge this time, consider remap color notification in table CONF.colors.time  - to cover elongated message lifespan. It's clever to leave message uncolored for 5-10 min (set first color time little less than error lifespan [purge_after_sec]); you will see at once which is the newest message.
Q: Where can I set parameters?
A: All parameters of auto:cerr are grouped in table named CONF at the begin of main code. They are verbosely commented there. There’s only limited validation of parameters, use wisely. When something is not clear there, I can explain it fully.

 

Edited September 4, 2025 by Łukasz997

[Sankotronic 2,370]

Hi @Łukasz997,

 

Thanks for this nice Quick app. I testing it and seems OK except that leaving user_id set to "auto" gives following error:

Please login or register to see this code.

 

It seems that if left set to "auto" it doesn't really find any users. After changing this settings I get message.

[Łukasz997 117]

3 godziny temu, Sankotronic napisał:

Please login or register to see this code.

 

Hello @Sankotronic, 

thank you for the message!

In theory user auto-find search for user named "admin" and assign it's ID to the variable used in push, otherwise try if user ID=2 exists - if so push messages to ID=2, then (if both fails) should select first received by api.get user. In your example it all fails and probably remain with value "auto". I tested only first path & only on my system (it works), so please forgive.

I'm preparing new version with advanced filters and better color-marking age of errors. I'm almost done. And investigate & correct also this error.

Give me few days!

[Łukasz997 117]

I checked it at once. It turned out then if you do not have user named "admin" and in CONF.user_id = "auto", no conditions are executed and variable used to send push == "auto".

The solution is very simple: at the beginning of main file in table CONF, change first field to:

Please login or register to see this code.

Makes the user auto-select work as expected. But in your situation (no user named admin) it's better to set user_id to ID you want to receive pushes.

However, if you set CONF.user_id = ID_you_want it will work only if such user exists (no verification,  no try to resolve name from ID). 

This part of the script is a bit poorly written and I correct it in next version.

Edited September 10, 2025 by Łukasz997

[Sankotronic 2,370]

Hi @Łukasz997,

 

I do not have any user with name "admin" and that explains why it gives error. Maybe instead of looking for user name you can look for user type "superuser" if user_id is set to "auto"?

[Łukasz997 117]

1 godzinę temu, Sankotronic napisał:

I do not have any user with name "admin" and that explains why it gives error. Maybe instead of looking for user name you can look for user type "superuser" if user_id is set to "auto"?

The same come to my mind after more carefully examine table which api.get returns. It's already done  that way 

 

[Łukasz997 117]

Good evening,

A few days ago I said the next version would launch very soon, now two weeks have passed - and nothing.

This quickapp has evolved into a different product built on the original idea. There are so many changes that it’s taking longer than I expected, twice the lines of code, a mini DSL with a compiler for building filters, and more. Now finished, in tests. I hope it will be worth the wait. Stay tuned...  

Please login or register to see this attachment.

Edited September 22, 2025 by Łukasz997

[jayrock 10]

Hi,

 

thanks for the useful QA.

 

Since I updated to 5.200.8, I get these errors repeatedly:

 

[19.12.2025] [10:53:31] [ERROR] [QUICKAPP745]: scanLoop exception: ./include/Mobile_comm.lua:157: Wrong parameter type, string required. Provided param 'false' is type of boolean

......

[19.12.2025] [10:55:05] [ERROR] [QUICKAPP745]: scanLoop exception: ./include/Mobile_comm.lua:121: Wrong parameter type, string required. Provided param 'false' is type of boolean

 

I cannot find hub.alert in the API documentation, and the UI does not show the full context help for hub.alert. So I changed to this as a workaround, however I don't get push notifications to my mobile anymore:

 

Please login or register to see this code.

 

Cheers,

 

jayrock

 

EDIT: I'm on [19.12.2025] [10:59:06] [TRACE] [QUICKAPP745]: [INFO] Version: 2.9.9 28.09.2025, by Łukasz Stachura

Edited December 19, 2025 by jayrock
Addition

[Łukasz997 117]

It briefly flashed before my eyes on the Forum that the way hub.alert is called has changed. I think an additional parameter was added - the email subject - which is now a text value rather than true/false. I did not focus on it at the time because I did not really have time, but I am almost certain this is the cause. As soon as I find a moment, I will track down that information and make the fix.

[jayrock 10]

Thanks. And good news, I do receive push notifications, they seem to have arrived with delay.

 

[jayrock 10]

Found it: 

 

[Łukasz997 117]

Thank you - the update is in auto:cerr2 thread (you don't need it - you've already dealt with the problem).

Now , a::c helps me to identify all hub.alert calls in my files!...
 

[Sankotronic 2,370]

Hi @Łukasz997,

 

I found this quick app as a very useful tool to capture fatal errors from my new built quick apps 👍

The only filter I use is the first one from the list "Only errors, all sources".

During usage I noticed that auto:cerr2 is using lots of resources with default settings, RAM and CPU so I have reduced database_limit from 2000 to 500 and RAM usage dropped immediately for about 8% and CPUs does not spike to 100%.

 

BTW - thanks to auto:cerr I have successfully captured and corrected some 🐞 in my quick apps code specially those that show up rarely and are hard to catch. THANKS!  

[Łukasz997 117]

Hi @Sankotronic,

I’m really glad that auto::cerr helped you catch those rare bugs - that was exactly the purpose of this tool.

About RAM/CPU usage: the name cerr may be a little misleading here, because this tool is not only a simple “error viewer”. It keeps captured ALL MESSAGES data (to enable finding with DSL ANY matching message) so with a high database_limit the internal database can become quite large. Additional parameter is purge_after_sec: after how long time the database starts to be purged (default 14 days). It works like "which one occur first".

Also, the internal data is not just a flat list of messages - 4 or 5 tables with different indexing used for fast deduplication, suppressing, counting, filtering and displaying errors. I’m sure there is still room for optimization there. 

I have not seen CPU reaching 100% in my own tests, honesty. If your system generated many unique error entries, reducing database_limit from 2000 to 500 is reasonable setting. For normal daily use, 500 is more than enough. The value 2000 & long purge time is useful for very long observation, by a cost of RAM. CPU may enlarge once a minute - this is the interval of the database operations and retrieving (only new) messages via api.