HCServer[1442]: segfault at c ip b6f4a1d3 sp a902a0e8 error 4 in libstdc++.so.6.0.21[b6eca000+111000]

[midi 7]

Hello, yesterday I have noticed a strange behaviour of my HC2 (4.600):

- can't login to HTTP interface (no response after clicking login button)

- fibaro API returns: HTTP/1.1 503 Service Temporarily Unavailable

 

In /var/log/messages I found this entry

 

Please login or register to see this code.

 

Anyone has seen this before?

Maybe some memory (RAM) error? 

[midi 7]

Another case today:

 

Please login or register to see this code.

 

I have made a further investigation:

 

Please login or register to see this code.

 

So the HCserver deamon seems to die after the segfault.

 

After starting it up with: "/etc/init.d/fibaro/HCServer start" everything back to normal.

 

I think some watchdog script in crontab would be useful here...

[midi 7]

No, this is not a problem with RAM...

MemTest86 v9.0

 

Please login or register to see this attachment.

[midi 7]

For further investigate I decided to enable core dumps in my HC2.

 

The core dumps are disabled for default as you see:

Please login or register to see this code.

 

To change this I just added this line ("ulimit -c unlimited") at the end of /etc/profile:
 

Please login or register to see this code.

 

Remember to restart the Fibaro HC2 (so the HCserver deamon could start with new settings).

 

The problem is that root filesystem (/) is mounted with ro flag - so core files can't be stored their.

To change this just set this kernel parameter:

 

Please login or register to see this code.

 

To make this change persistent create file: /etc/sysctl.conf and put there a line:

 

Please login or register to see this code.

 

and after that run:

 

Please login or register to see this code.

 

 

Now let's wait for another crash of HCserver...

Edited March 13, 2021 by midi

[midi 7]

I created a watchdog script to those who need a simple workaround.

 

vi /opt/fibaro/scripts/HCServerWatchdog.sh

 

Please login or register to see this code.

 

chmod 755 /opt/fibaro/scripts/HCServerWatchdog.sh

 

Then at the and of crontab just add this line:

 

crontab -e

 

Please login or register to see this code.

 

After that the HCServer will be automatically started in less than one minute after any crash.

You still need to observe the logs just to be sure that HCServer is not in some death loop (restart over and over again).

[midi 7]

The problem has been reported in Fibaro Support (by me) with ticket: #45142