Does Netatmo API Security Update means BYE BYE to Fibaro integration?

[Sankotronic 1,976]

 

I believe that all current Solutions for Netatmo integration to Fibaro use Client Credentials grand type which Netatmo plans to remove in October 2022. I see this as a big setback for all home automation integrations. Security is always important and any improvements are most welcomed. Maybe they could do better by removing sending username and password and allowing local connection between home automation gateways and Netatmo devices, but it seems that local connection will never be provided.

 

The only way to connect to our Netatmo products after this date is to use Authorization Code grand type which is interactive or requires from uses to login and allow access to the application who's ID is used in request.

 

As we all know, Fibaro HC does not provide such user interaction and no popup windows with entry fields are possible.

 

I was testing step 1 by sending following request:

Please login or register to see this code.

 

first with Postman and response is always HTML page with login to Netatmo.

 

I also try to send same request from browser on which I was already logged in to Netatmo and then I get page which ask me to click on button to authorize or decline app to access my Netatmo weather station. Of course after clicking on "Yes I accept" button I get back JSON response:

Please login or register to see this code.

 

This error is returned probably due to wrong redirect URI?

 

I also try with my URL link that I use for HC2 to receive webhook from Geofency app and then I did get URL with Netatmo generated code, but how to get that code to make step 4 and get access token?

 

Not to mention that step 2 is not possible on Fibaro gateways.

 

At this moment I don't see how to solve this issue and will much appreciate help if anyone has knowledge how to do it!

 

If there is no solution then in October will be BYE BYE to Netatmo. Now I'm happy to have Tempest weather station and I just hope they will not to the same stupid step as Netatmo and make integration more difficult or almost impossible.

 

[petergebruers 1,259]

6 hours ago, Sankotronic said:

At this moment I don't see how to solve this issue and will much appreciate help if anyone has knowledge how to do it!

I do not know how to solve it and even worse I think there is no easy solution, I do however know a "auth flow" on Home Assistant, and this might give you some clues as to why the "redirect" url exist. It is indeed the final step to "receive" the token used for oauth

 

Please login or register to see this link.

 

The gist of it is this:

 

Please login or register to see this code.

 

IMHO the key points are

 

1) When you are on the server site (eg Netatmo) you have to enter a valid URL. Rules to what makes a valid redirect URL differ, depending on provider, for example see

Please login or register to see this link.

2) The authentication server uses that redirect URL to "post" the code, by appending "?code=AUTHORIZATION_CODE" to the request

 

I have been toying with this and it all boils down to having a nice "client library" to handle all this and hide the complexities...

 

I think this is one of the better explanations of the flow

 

Please login or register to see this link.

 

BTW, the official Home Connect Integration only handles the initial token flow, and the token will expire after 1 or 3 months IIRC and the integration does not handle token renewal at all. So after X months you have to re-autorise. Just saying that because you might face the same challenge if you get past the first hurdle

 

It is not my field of expertise, I hope someone else chimes in

 

6 hours ago, Sankotronic said:

BYE BYE to Netatmo

Well, they have nice enclosures and I know a guy who did some DIY with "off the shelf" components and now enjoys cloud-free operation...

Edited August 8, 2022 by petergebruers

[tinman 1,011]

21 hours ago, petergebruers said:

I do not know how to solve it and even worse I think there is no easy solution,

 

install Home Assistant somewhere in internet, and let poll HC3 the data from there, hehe. 

[Sankotronic 1,976]

On 8/8/2022 at 2:43 PM, petergebruers said:

Well, they have nice enclosures and I know a guy who did some DIY with "off the shelf" components and now enjoys cloud-free operation...

 

Hi @petergebruers ,

did this guy ever published how he did DIY? Is there any link?

 

On 8/9/2022 at 12:26 PM, tinman said:

 

install Home Assistant somewhere in internet, and let poll HC3 the data from there, hehe. 

 

Hi @tinman ,

do I see some sarcasm in your comment or I'm mistaken?

 

For now I see one very simple solution, sell all Netatmo products and look somewhere else.

[petergebruers 1,259]

22 hours ago, Sankotronic said:

did this guy ever published how he did DIY? Is there any link?

Oh, we both know him very well but this was discussed in a privater topic, so I did not mention his name here.

 

I recommend a CO2 MH-Z19 sensor and a "Wemos D1 mini" ESP8266 board. In 2022 I would still recommend the CO2 sensor, for the board you might consider something newer. Though for such a simple sensor, either use the (still available but old) ESP8266 boards or the newer "ESP32 C3" chip - both will work.

 

For software, there are several projects, but these are the "big" ones.

 

Please login or register to see this link.

 

If you have Home Assistant - or even when you do NOT have home assistant:

 

Please login or register to see this link.

 

Both can serve data in several ways eg through a web page, htttp post/get or using MQTT.

 

Once you got the problem of measuring CO2 solved, ... there is a ton of other sensors that can be added. For example,  for temperature (DS18b20 or DHT22), lux (BH1750), PIR, Particulate Matter ...

 

If you want a DIY Z-Wave solution, that is possible too. Use a Z-Uno 

Please login or register to see this link.

 

I have the MH-Z19B on "ESP" boards and also one on a Z-Uno with software I wrate (Arduino platform)

 

Requires basic soldering skills (2.54 mm pin pitch).

Edited August 12, 2022 by petergebruers

[Tony270570 92]

Hi @petergebruers, might be me your are talking about, as we have already talk about a similar subject few time ago.
@SankotronicI'll be back home by the end of next week, i'll share few pictures i have when i've done the project. 
As Peter said,it's based on wemos D1 and mhz19. With the netatmo casing i've also made other small projects like particule sensors+dht22.

There is nothing fancy, i've just throw to the been the netatmo components  and as said reused the casing.

I'll post very soon !

[Sankotronic 1,976]

Hello @Tony270570 ,

 

I will be most grateful if you can share this project. Their casing is nice and seems to be soon only usable part of their devices

[Tony270570 92]

Hi @Sankotronic, i'll do as soon i'm home, as you've said it will be soon usefull for spare parts. 

[jgab 1,739]

A mechanism for defining http ingress points (per QA) was number 7 on my wishlist. Besides callbacks for oath it's also useful for web hooks etc.

Number 6 mention more libraries like Oauth. 

For both these I was mainly thinking about local access by devices in your own network.

I'm not sure if that's enough for Oauth, but I think it is. Maybe it could be modelled after 

Please login or register to see this link.

 

 

[Sankotronic 1,976]

Well,

 

I'm not guru for Oauth2 so I just need a bit more understanding to be able to follow.

 

What to do with Authorization code grant type?

It is quite clear that Home Center cannot do authorization process since it can not be logged in to my Netatmo account and neither can show prompt to login, not to mention show app authorization confirmation page.

 

As you all suggested now users need to setup their own server with either Home assistant or Node-red and write code there which will handle authorization and then process devices data which then Home Center can pool. Is this correct?

 

Since my internet connection is changing IP address every 24h I guess my redirect URL must reflect this changes if I install HA or NR on some ras-PI connected to my local network? Another solution is to place this server somewhere on the internet which usually costs money and it is still security wise questionable.

 

Still even if I can do that and make such solution with my own server, question is: how to share that solution. Do we really expect that every Fibaro HC and Netatmo user will go through such investment and process? I personally do not think so.

 

It seems to me that at this moment no one has solution to replace Client Credentials grant type authorization. That is a bit disturbing because all our solutions will stop working soon and there will be no replacement so BYE BYE Netatmo still stands!

 

This Netatmo decision one more time proofs that we should avoid buying any devices that do not have LAN access and provide only access over the cloud.

One more recent example:

Please login or register to see this link.

 

 

 

 

[jgab 1,739]

I don't think the problem is not having LAN access. It could still be a local device on the LAN requiring Oath to login.

I believe it's going to be more and more common to have these kind of permission based authorisation schemas in the flow. All the issues with privacy and security, Oath allows the application to ask for only specific permissions and the user then becomes involved in only granting the permissions it wants the application to have. Google, Twitter, Facebook is going this way and lots of services use them for authorisation. 

For limited devices that can't receive http calls there is probably the model where the authorisation is done elsewhere (separate app, could it be done in JS in the browser) and then the access+refresh token is handed to the device - but the process would be initiated from the (separate) app and not from the device.

In theory, refresh tokens would never need to expire - but I guess it's implementation dependent - but in general it's not user friendly to force the user to authorizate again - especially if it involves making a lot of permission decisions...

 

Of course, Fibaro, could also give us a library that does Oath for us so that developer QAs can leverage it - worst case they implement Oath only for their own plugins....

 

 

 

[nicohi 18]

I have taken contact with Netatmo team, and they respond quickly. On french forum I publish the discussion.

The last one is :

 

Quote

La création du token et de l'accès va changer, donc non l'ancien système ne continuera pas à fonctionner si un nouveau token est créer pour chaque refresh. 
L'access token et le refresh token ne vont pas arrêter de fonctionner, ce qui va changer c'est que toutes les trois heures il faudra générer un nouveau refresh token. Ce changement n'a pas encore d'ETA mais j'estimerais que vers la fin de l'année il devrait se mettre en place. 
 
Nous allons aussi permettre la génération d'un refresh token à partir de postman ou du portail open API. Il sera donc possible d'utiliser ce token pour récupérer les access/refresh token sans avoir à ré-authentifier l'utilisateur. 

 

 

The last phrase is very important, because it is really what we need to resolve all problem. But the fact is that will come after. I am trying to change deadline, in order to have this before shutdown of the olh authentification service. Normaly next week a call with them, in order to clarify that.

[SmartHomeEddy 789]

Small translation:

 

Quote

The creation of the token and the access will change, so no, the old system will not continue to function if a new token is created for each refresh.
The access token and the refresh token will not stop working, what will change is that every three hours a new refresh token will have to be generated. This change does not yet have an ETA but I would estimate that towards the end of the year it should be in place.
 
We will also allow the generation of a refresh token from postman or the open API portal. It will therefore be possible to use this token to retrieve access/refresh tokens without having to re-authenticate the user.

 

[gsmart 29]

Hi,

Please take a look at this post...

 

[speedoxx007 27]

Hello,

 

Today I received the following from Netatmo:

__________________________________________________________________________________________________

Authentication update

 

Dear Netatmo Developer,

as of today, the Netatmo servers respond with two tokens when you update an Access Token via the associated endpoint

Please login or register to see this link.

an Access Token and a Refresh Token.

If the previous Access Token is still valid, the newly returned Access Token is identical, but its expiration time is extended by 3 hours.

The Refresh Token will not be renewed in any case.

As of 04/17/2023, this behavior will change to comply with the recommendations of the OAuth2 Authorization Framework RFC (Section 10.4) and to improve the security of our users' data.

When tokens are refreshed, Access Tokens and Refresh Tokens will be automatically renewed and previous tokens will be invalidated.

 

What does this mean for me?

If you have already updated the tokens you specified when refreshing your tokens, this change will not affect you.

If you do not update the refresh token when updating your access token, your users will be disconnected after 3 hours as the previous tokens will be invalidated.

To fix this, you will need to update the tokens as soon as you receive the newly generated tokens.

 

Sincerely,

Legrand - Netatmo - Bticino

______________________________________________________________________________________________________

I am using UHAS with Netatmo Weather and Netatmo Cam's. Will this still work after 17/04/2023?

[Sankotronic 1,976]

1 hour ago, speedoxx007 said:

I am using UHAS with Netatmo Weather and Netatmo Cam's. Will this still work after 17/04/2023?

 

Hopefully will work. I will check and update code accordingly to use this new API and oAuth2 requirements if possible. We are not going to loose it or let it go.

[Sankotronic 1,976]

Just to add that I hope to provide updated Netatmo weather and Cameras with next UHAS update 1.0.2 planned to be published before next weekend. Delay in publishing is possible since I have to invest few days of preparation for upcoming interviews I will have soon.

 

[mart 1]

Hi @Sankotronic any news regarding this?

[Sankotronic 1,976]

Hi @mart ,

 

I have updated:

and

and all use new Netatmo authentication that came with Netatmo security update. All downloads provide Installation and user manuals.

 

[Corceiro 3]

Still no Netatmo thermostat integration on HC3?

[Sankotronic 1,976]

Hi @Corceiro ,

 

To be honest I didn't se a proper integration for Netatmo Weather station, not to mention for Netatmo Aircare modules (former coaches), cameras and thermostat.

I do not understand Fibaro and their approach to quick apps, see here for details:

Didn't get any comments from Fibaro guys until now, but I guess there is nothing to add from their side