AI / Machine learning in home automation

[3JL 7]

Hi all,

As a geek++ I would be really interested to explore possible ways to truly automate my Fibaro-driven home with e.g. deep learning / machine learning. Has anyone tried this? Or seen examples?

Thanks! J

Edited May 23, 2017 by 3JL

[AutoFrank 372]

+1

[fibarojiang 0]

+1

[emielstroeve 22]

ìnstead of +1, could we come up with examples? 

[Sankotronic 2,370]

1 hour ago, emielstroeve said:

ìnstead of +1, could we come up with examples? 

 

Yes, examples will be most welcomed. AI in the HA? Hmmm, I just know that I don't want my house to become too intelligent because I might one day be kicked out  

[Bodyart 231]

Rise of machines...

[gucio1234 79]

One said that the smart home could not be any smarter than his owner.

 

[emielstroeve 22]

6 hours ago, gucio1234 said:

One said that the smart home could not be any smarter than his owner.

 

with machine learning HC2 could use the smartness of multiple HC2's, and therefor multiple users, into individual homes. So our HC2's could learn from each other.

 

The only question left is could our significant other handle this increase in intelligence

Edited May 25, 2017 by emielstroeve

[AutoFrank 372]

For me AI in my HA system would be about learning our behavior and actions and coding itself and having a true natural language interface...

ie - every time we put on the TV in the playroom,,we switch off the music, every time I set night mode, I turn off certain lights, set alarm, shutdown all media, everytime we switch on the TV in the living room I also switch on the Sat Receiver and there are lots more examples

I currently have all these scenarios coded but the ideal would be once all the actors and sensors are installed the system would learn and adapt and create these connections itself.

Regarding a voice interface, Alexa is a god start but not being able to say open/close and other simple commands is a big restriction. If I have an echo dot in the living room and I say 'turn on music', it should know I am in the living room (other sensors) and put on that music.

 

HA is about convenience and safety and AI should make this much much easier to setup and run.

 

[3JL 7]

Autofrank & emielstroeve, those are exactly my thoughts

 

I think it should have several components assuming we should put something ourselves:

- some kind of common data storage format to register actors and sensors (e.g. NAS)

- a smart algorithm that can do supervised learning (where the supervision lies first in old data and later in corrections of the AI's actions by e.g. human actions or an app asking "does the current light setting match your mood, acitivities, etc"?)

- or a smart algorithm that can do unsupervised learning, but I have no clue how that would work

- a way to generalise learnings between systems

 

Sounds easy; let's get started!!

 

On a more serious note: who would be interested to explore this in more detail and just try? I think I can help on the algorithms as I had some AI training as part of my engineering degree, but I have no experience in architecture. 

 

To me a logical start would be the data model as we do need training data before we can start experimenting with algorithms but I am very eager to hear your thoughts!!

 

J

[3JL 7]

So it seems this is not the right place/way to initiate this... Any suggestions?

[Sankotronic 2,370]

On 5/25/2017 at 9:54 AM, AutoFrank said:

For me AI in my HA system would be about learning our behavior and actions and coding itself and having a true natural language interface...

 

On 5/26/2017 at 10:42 PM, 3JL said:

Sounds easy; let's get started!!

 

On 6/7/2017 at 0:12 AM, 3JL said:

So it seems this is not the right place/way to initiate this... Any suggestions?

 

Hi @3JL,

 

You initiated this question and only asked more questions. You stated above that this sounds easy and called everyone to start working on HA AI. Great, so have you done anything yet? Or you are now waiting for others to do the job?

 

AI is everything but easy especially if self learning is involved. It is so broad theme with million questions, except if you were referring to something more realistic level which actually cannot be even called AI? Can you at least make some frame and narrow this to something specific?

 

I can help you with that. I already asked AI related question here:

This is definitely related to learning. HA is learning from motion sensors about our movement, where we stay mostly, what rooms are used seldom and from that it can start predicting. So, not really an AI, but that is beginning. Check how much answers I get there.

 

On 5/26/2017 at 10:42 PM, 3JL said:

On a more serious note: who would be interested to explore this in more detail and just try? I think I can help on the algorithms as I had some AI training as part of my engineering degree, but I have no experience in architecture. 

 

To me a logical start would be the data model as we do need training data before we can start experimenting with algorithms but I am very eager to hear your thoughts!!

 

Since you have some AI training, can you build algorithm that will crunch motion sensors breach/safe data and put it in some matrix that can be later use for at least mimicking lights and blinds while we are not at home? You don't really have to build algorithm in LUA code, you don't need to use any programming code in that matter, you can build it in normal language expressions and mathematical formulas. Programmers will then do the rest.

 

So is this task good enough to start building HA IA or I missed the point?

 

I'm busy at the moment with building some basic automation needed for more sophisticated actions that will at least look like AI. It is always first task to gather information. Then next task is to analyze that data and build some matrices. Last is then building logic that will use that matrices for making decisions and do actions.

[akatar 208]

machine learning?

ha is for me some automatic tasks, windows open/close etc but also giving information:

tomorrow the "green" garbage is being collected etc

who is calling incl the number on my touchscreen and tv screen

etc etc

 

[3JL 7]

Hi @Sankotronic,

I like your thoughts and your push in a first direction.

 

If I understand correctly, you want to identify typical sequences of movement (possibly related to day and time) and (i) mimic those (easiest) and (ii) use those to predict movement (harder).

1. For mimicking we need some sort of pattern recognition: finding typical sequences of actions and replicating those.
2. To predict movement we can add some kind of probabilistic modeling.
3. But that is where it becomes really interesting  Those predicted sequences will not be perfect and your home situation changes over time, that is where AI can come into play. 

Would you have example data that I can play with to start on the pattern recognition part? (I only have one motion sensor ) 

J

[Sankotronic 2,370]

22 hours ago, 3JL said:

Would you have example data that I can play with to start on the pattern recognition part? (I only have one motion sensor ) 

 

Hi @3JL,

 

Since code that I provided link does not save any data I don't have example data yet. But if there is a way to have code that can use data from event panel and segregate motion sensors breach/safe times then that will be a good starting point. Maybe somebody here on forum have some similar code?

[3JL 7]

For all interested:

 

Below my current plan (with increasing complexity); I am eager to hear your thoughts!

- Get log parsing in place

- Start with a very simple mimicking script (when not home, HC2 selects a random typical sequence of lights going on and off, harmonised to day of week and sun rise and set). This essentially does not need sensors (hence is applicable to my home).

- Build supervised pattern recognition and probability maps, trying to predict next steps (with supervised I mean: I will add the intelligence)

- Build AI engine to accommodate for feedback loops (i.e. correction by users)

- Expand logging to network monitoring (devices logging on / off, etc)

- Build AI engine for unsupervised pattern recognition (using clustering techniques without bias)

 

Step 1 is already in place. I now have a Python script on my Synology NAS that fetches the HC2 logs every night. Next step is to make sense of those logs

 

FYI, I use this script:

Please login or register to see this code.

I use MariaDB and Mysql/Python connector packes on DSM 6. DB layout is shown below.

 

Please login or register to see this attachment.

 

If you see things that can be improved and/or simplified, I would be happy to hear!

J

 

[Fiberik 0]

Would love some updates on the progress and what you have achieved.