Jump to content

Welcome to Smart Home Forum by FIBARO

Dear Guest,

 

as you can notice parts of Smart Home Forum by FIBARO is not available for you. You have to register in order to view all content and post in our community. Don't worry! Registration is a simple free process that requires minimal information for you to sign up. Become a part of of Smart Home Forum by FIBARO by creating an account.

 

As a member you can:

  •     Start new topics and reply to others
  •     Follow topics and users to get email updates
  •     Get your own profile page and make new friends
  •     Send personal messages
  •     ... and learn a lot about our system!

 

Regards,

Smart Home Forum by FIBARO Team


  • 0

Does Netatmo API Security Update means BYE BYE to Fibaro integration?


Sankotronic

Question

 

I believe that all current Solutions for Netatmo integration to Fibaro use Client Credentials grand type which Netatmo plans to remove in October 2022. I see this as a big setback for all home automation integrations. Security is always important and any improvements are most welcomed. Maybe they could do better by removing sending username and password and allowing local connection between home automation gateways and Netatmo devices, but it seems that local connection will never be provided.

 

The only way to connect to our Netatmo products after this date is to use Authorization Code grand type which is interactive or requires from uses to login and allow access to the application who's ID is used in request.

 

As we all know, Fibaro HC does not provide such user interaction and no popup windows with entry fields are possible.

 

I was testing step 1 by sending following request:

Please login or register to see this code.

 

first with Postman and response is always HTML page with login to Netatmo.

 

I also try to send same request from browser on which I was already logged in to Netatmo and then I get page which ask me to click on button to authorize or decline app to access my Netatmo weather station. Of course after clicking on "Yes I accept" button I get back JSON response:

Please login or register to see this code.

 

This error is returned probably due to wrong redirect URI?

 

I also try with my URL link that I use for HC2 to receive webhook from Geofency app and then I did get URL with Netatmo generated code, but how to get that code to make step 4 and get access token?

 

Not to mention that step 2 is not possible on Fibaro gateways.

 

At this moment I don't see how to solve this issue and will much appreciate help if anyone has knowledge how to do it!

 

If there is no solution then in October will be BYE BYE to Netatmo. Now I'm happy to have Tempest weather station and I just hope they will not to the same stupid step as Netatmo and make integration more difficult or almost impossible.

 

  • Thanks 1
Link to comment
Share on other sites

Recommended Posts

  • 0
6 hours ago, Sankotronic said:

At this moment I don't see how to solve this issue and will much appreciate help if anyone has knowledge how to do it!

I do not know how to solve it and even worse I think there is no easy solution, I do however know a "auth flow" on Home Assistant, and this might give you some clues as to why the "redirect" url exist. It is indeed the final step to "receive" the token used for oauth

 

Please login or register to see this link.

 

The gist of it is this:

 

Please login or register to see this code.

 

IMHO the key points are

 

1) When you are on the server site (eg Netatmo) you have to enter a valid URL. Rules to what makes a valid redirect URL differ, depending on provider, for example see

Please login or register to see this link.

2) The authentication server uses that redirect URL to "post" the code, by appending "?code=AUTHORIZATION_CODE" to the request

 

I have been toying with this and it all boils down to having a nice "client library" to handle all this and hide the complexities...

 

I think this is one of the better explanations of the flow

 

Please login or register to see this link.

 

BTW, the official Home Connect Integration only handles the initial token flow, and the token will expire after 1 or 3 months IIRC and the integration does not handle token renewal at all. So after X months you have to re-autorise. Just saying that because you might face the same challenge if you get past the first hurdle

 

It is not my field of expertise, I hope someone else chimes in

 

6 hours ago, Sankotronic said:

BYE BYE to Netatmo

Well, they have nice enclosures and I know a guy who did some DIY with "off the shelf" components and now enjoys cloud-free operation...

Edited by petergebruers
  • Like 1
  • Thanks 1
Link to comment
Share on other sites

  • 0
21 hours ago, petergebruers said:

I do not know how to solve it and even worse I think there is no easy solution,

 

install Home Assistant somewhere in internet, and let poll HC3 the data from there, hehe. 

  • Like 1
Link to comment
Share on other sites

  • 0
  • Inquirer
  • On 8/8/2022 at 2:43 PM, petergebruers said:

    Well, they have nice enclosures and I know a guy who did some DIY with "off the shelf" components and now enjoys cloud-free operation...

     

    Hi @petergebruers ,

    did this guy ever published how he did DIY? Is there any link?

     

    On 8/9/2022 at 12:26 PM, tinman said:

     

    install Home Assistant somewhere in internet, and let poll HC3 the data from there, hehe. 

     

    Hi @tinman ,

    do I see some sarcasm in your comment or I'm mistaken? ;-)

     

    For now I see one very simple solution, sell all Netatmo products and look somewhere else.

    Link to comment
    Share on other sites

    • 0
    22 hours ago, Sankotronic said:

    did this guy ever published how he did DIY? Is there any link?

    Oh, we both know him very well but this was discussed in a privater topic, so I did not mention his name here.

     

    I recommend a CO2 MH-Z19 sensor and a "Wemos D1 mini" ESP8266 board. In 2022 I would still recommend the CO2 sensor, for the board you might consider something newer. Though for such a simple sensor, either use the (still available but old) ESP8266 boards or the newer "ESP32 C3" chip - both will work.

     

    For software, there are several projects, but these are the "big" ones.

     

    Please login or register to see this link.

     

    If you have Home Assistant - or even when you do NOT have home assistant:

     

    Please login or register to see this link.

     

    Both can serve data in several ways eg through a web page, htttp post/get or using MQTT.

     

    Once you got the problem of measuring CO2 solved, ... there is a ton of other sensors that can be added. For example,  for temperature (DS18b20 or DHT22), lux (BH1750), PIR, Particulate Matter ...

     

    If you want a DIY Z-Wave solution, that is possible too. Use a Z-Uno 

    Please login or register to see this link.

     

    I have the MH-Z19B on "ESP" boards and also one on a Z-Uno with software I wrate (Arduino platform)

     

    Requires basic soldering skills (2.54 mm pin pitch).

    Edited by petergebruers
    • Like 1
    • Thanks 1
    Link to comment
    Share on other sites

    • 0

    Hi @petergebruers, might be me your are talking about, as we have already talk about a similar subject few time ago.
    @SankotronicI'll be back home by the end of next week, i'll share few pictures i have when i've done the project. 
    As Peter said,it's based on wemos D1 and mhz19. With the netatmo casing i've also made other small projects like particule sensors+dht22.

    There is nothing fancy, i've just throw to the been the netatmo components :) and as said reused the casing.

    I'll post very soon !

    • Like 2
    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Hello @Tony270570 ,

     

    I will be most grateful if you can share this project. Their casing is nice and seems to be soon only usable part of their devices ;-)

    • Like 1
    Link to comment
    Share on other sites

    • 0

    A mechanism for defining http ingress points (per QA) was number 7 on my wishlist. Besides callbacks for oath it's also useful for web hooks etc.

    Number 6 mention more libraries like Oauth. 

    For both these I was mainly thinking about local access by devices in your own network.

    I'm not sure if that's enough for Oauth, but I think it is. Maybe it could be modelled after 

    Please login or register to see this link.

     

     

    • Like 1
    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Well,

     

    I'm not guru for Oauth2 so I just need a bit more understanding to be able to follow.

     

    What to do with Authorization code grant type?

    It is quite clear that Home Center cannot do authorization process since it can not be logged in to my Netatmo account and neither can show prompt to login, not to mention show app authorization confirmation page.

     

    As you all suggested now users need to setup their own server with either Home assistant or Node-red and write code there which will handle authorization and then process devices data which then Home Center can pool. Is this correct?

     

    Since my internet connection is changing IP address every 24h I guess my redirect URL must reflect this changes if I install HA or NR on some ras-PI connected to my local network? Another solution is to place this server somewhere on the internet which usually costs money and it is still security wise questionable.

     

    Still even if I can do that and make such solution with my own server, question is: how to share that solution. Do we really expect that every Fibaro HC and Netatmo user will go through such investment and process? I personally do not think so.

     

    It seems to me that at this moment no one has solution to replace Client Credentials grant type authorization. That is a bit disturbing because all our solutions will stop working soon and there will be no replacement so BYE BYE Netatmo still stands!

     

    This Netatmo decision one more time proofs that we should avoid buying any devices that do not have LAN access and provide only access over the cloud.

    One more recent example:

    Please login or register to see this link.

     

     

     

     

    Link to comment
    Share on other sites

    • 0

    I don't think the problem is not having LAN access. It could still be a local device on the LAN requiring Oath to login.

    I believe it's going to be more and more common to have these kind of permission based authorisation schemas in the flow. All the issues with privacy and security, Oath allows the application to ask for only specific permissions and the user then becomes involved in only granting the permissions it wants the application to have. Google, Twitter, Facebook is going this way and lots of services use them for authorisation. 

    For limited devices that can't receive http calls there is probably the model where the authorisation is done elsewhere (separate app, could it be done in JS in the browser) and then the access+refresh token is handed to the device - but the process would be initiated from the (separate) app and not from the device.

    In theory, refresh tokens would never need to expire - but I guess it's implementation dependent - but in general it's not user friendly to force the user to authorizate again - especially if it involves making a lot of permission decisions...

     

    Of course, Fibaro, could also give us a library that does Oath for us so that developer QAs can leverage it - worst case they implement Oath only for their own plugins....

     

     

     

    • Like 1
    • Thanks 2
    Link to comment
    Share on other sites

    • 0

    I have taken contact with Netatmo team, and they respond quickly. On french forum I publish the discussion.

    The last one is :

     

    Quote

    La création du token et de l'accès va changer, donc non l'ancien système ne continuera pas à fonctionner si un nouveau token est créer pour chaque refresh. 
    L'access token et le refresh token ne vont pas arrêter de fonctionner, ce qui va changer c'est que toutes les trois heures il faudra générer un nouveau refresh token. Ce changement n'a pas encore d'ETA mais j'estimerais que vers la fin de l'année il devrait se mettre en place. 
     
    Nous allons aussi permettre la génération d'un refresh token à partir de postman ou du portail open API. Il sera donc possible d'utiliser ce token pour récupérer les access/refresh token sans avoir à ré-authentifier l'utilisateur. 

     

     

    The last phrase is very important, because it is really what we need to resolve all problem. But the fact is that will come after. I am trying to change deadline, in order to have this before shutdown of the olh authentification service. Normaly next week a call with them, in order to clarify that.

    • Like 1
    Link to comment
    Share on other sites

    • 0

    Small translation:

     

    Quote

    The creation of the token and the access will change, so no, the old system will not continue to function if a new token is created for each refresh.
    The access token and the refresh token will not stop working, what will change is that every three hours a new refresh token will have to be generated. This change does not yet have an ETA but I would estimate that towards the end of the year it should be in place.
     
    We will also allow the generation of a refresh token from postman or the open API portal. It will therefore be possible to use this token to retrieve access/refresh tokens without having to re-authenticate the user.

     

    Link to comment
    Share on other sites

    • 0

    Hello,

     

    Today I received the following from Netatmo:

    __________________________________________________________________________________________________

    Authentication update

     

    Dear Netatmo Developer,

    as of today, the Netatmo servers respond with two tokens when you update an Access Token via the associated endpoint

    Please login or register to see this link.

    an Access Token and a Refresh Token.

    If the previous Access Token is still valid, the newly returned Access Token is identical, but its expiration time is extended by 3 hours.

    The Refresh Token will not be renewed in any case.

    As of 04/17/2023, this behavior will change to comply with the recommendations of the OAuth2 Authorization Framework RFC (Section 10.4) and to improve the security of our users' data.

    When tokens are refreshed, Access Tokens and Refresh Tokens will be automatically renewed and previous tokens will be invalidated.

     

    What does this mean for me?

    If you have already updated the tokens you specified when refreshing your tokens, this change will not affect you.

    If you do not update the refresh token when updating your access token, your users will be disconnected after 3 hours as the previous tokens will be invalidated.

    To fix this, you will need to update the tokens as soon as you receive the newly generated tokens.

     

    Sincerely,

    Legrand - Netatmo - Bticino

    ______________________________________________________________________________________________________

    I am using UHAS with Netatmo Weather and Netatmo Cam's. Will this still work after 17/04/2023?

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • 1 hour ago, speedoxx007 said:

    I am using UHAS with Netatmo Weather and Netatmo Cam's. Will this still work after 17/04/2023?

     

    Hopefully will work. I will check and update code accordingly to use this new API and oAuth2 requirements if possible. We are not going to loose it or let it go.

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Just to add that I hope to provide updated Netatmo weather and Cameras with next UHAS update 1.0.2 planned to be published before next weekend. Delay in publishing is possible since I have to invest few days of preparation for upcoming interviews I will have soon.

     

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Hi @mart ,

     

    I have updated:

    and

    and all use new Netatmo authentication that came with Netatmo security update. All downloads provide Installation and user manuals.

     

    Link to comment
    Share on other sites

    • 0
  • Inquirer
  • Hi @Corceiro ,

     

    To be honest I didn't se a proper integration for Netatmo Weather station, not to mention for Netatmo Aircare modules (former coaches), cameras and thermostat.

    I do not understand Fibaro and their approach to quick apps, see here for details:

    Didn't get any comments from Fibaro guys until now, but I guess there is nothing to add from their side ;-)

    • Like 1
    Link to comment
    Share on other sites

    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.

    Guest
    Answer this question...

    ×   Pasted as rich text.   Paste as plain text instead

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×
    ×
    • Create New...