API invalid grant at random intervals

david

4 juin 2024 19:53

So, is the fix to pre-emptively refresh the grant after n seconds? The documentation seems clear that you should only refresh expired tokens.

0

alexander

4 juin 2024 21:12

@david: I checked my token refresh code and I'm NOT setting an "Authorization" header for the "https://api.netatmo.com/oauth2/token" request when refreshing a token and I don't see the problems you are seeing.

So I would try removing the following line in "DownloadRefreshToken" and see if it helps:

httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", expiredToken);

0

sk

5 juin 2024 06:25

It's strange, but sometimes just 7 hours of sleep help: without having changed anything in the code, everything is now working smoothly again.

The problem where authentication on one device cancelled that of the other has disappeared. What I did last night, however, was to delete all third-party links in my netatmo account and then recreate them all. Who knows if this has anything to do with solving the problem? However, everything is currently working perfectly. Perhaps something has been corrected on the netatmo infrastructure side after all, or some caches have been cleared ... ???

0

david

5 juin 2024 13:07

Removing the authorization header from the renewal method per @alexander didn't change the behavior. We're going to try refreshing the token before it expires going forward.

0

Leslie Community moderator

5 juin 2024 13:36

Hello,

@sk : so can we consider the problem solved ? I don't really know why it took time for you (maybe as you said some cache cleaned with automatic operations in the night ?)

@david : you could try this. But normally the refresh_token value never expires. So, even if the access_token reached his 3 hours lifespan you should be able to correctly refresh it and get your new pair of access/refresh_token for the next /token call

Have a good day,

Leslie - Community Manager

0

sk

5 juin 2024 13:49

@Leslie: Yes, problem solved. :o)

1

david

5 juin 2024 13:54

@Leslie: dommage, mais non. After removing the auth header from the refresh call, the grant still expired promptly at 3 hours, 20 minutes after the last refresh.

Is there any reason not to refresh the token before it expires?

0

Leslie Community moderator

5 juin 2024 14:40

@David, Strange ... no problem, you can refresh the access before its expiration if you want

Please keep me informed if it fixed your issue

Have a good day,

Leslie - Community Manager

0

alexander

5 juin 2024 20:05

@David: I've looked at your code again, and you don't seem to take into account the "expires_in" return parameter at all.

This is usually 10800 seconds, so 3 hours.

If the expiration date (timestamp of last token refresh + "expires_in" seconds) is reached, you should refresh the token before sending the next request. You don't seem to do that.

0

david

5 juin 2024 20:08

@alexander: That's because until Monday the documentation said to handle token expiration by refreshing the token. The code posted above worked without interruption for half a year until noon CEST Monday. The API behavior changed; our code didn't.

We've added code today that will simply refresh the token every 2 hours. We'd rather not have the automated process communicate with the scheduling process, but if necessary, that's our next step.

1

alexander

5 juin 2024 20:16

The documentation says (and I don't believe that has changed): "When you request a token (via grant type credentials or authorization code) you retrieve as well the validity timelapse and a refresh token. Once your token has expired, you'll have to request a new one using the grant_type refresh_token."

I always interpreted it like this:

If the number of seconds returned by the "expires_in" return parameter has passed, the access token should be considered as expired and it should be refreshed before using it to make a new API request.

So refreshing after 10800 seconds (or whatever the server returns) should work. That's what I've always done and it still works after the recent server changes.

I believe you that your code didn't change, but it looks like you interpreted the documentation and what "expired" means differently.

0

david

5 juin 2024 20:30

I always interpreted the docs to mean that the API will tell you that the token has expired, and then you go and get a new one. The API behaved that way until this week, so we had no reason to think otherwise.

We're pushing the new bits this evening (Chicago time). If it runs successfully overnight, then refreshing the token before it expires is the solution.

0

david

6 juin 2024 03:03

I've just spent several hours debugging this, and I have new data.

Fundamentally, the invalid_grant response seems to occur when the token has expired. If I navigate to the app setup page (https://dev.netatmo.com/apps/{id}) after the function app gets the invalid_grant response, the app setup page also reports invalid_grant.

I would submit to Netatmo that a bad request (400) response with invalid_grant as the error code doesn't actually follow the documentation at https://dev.netatmo.com/apidocumentation/oauth.

We're going to keep monitoring this. We are refreshing the token every 2 hours now, and logging everything that the API sends back (except the tokens themselves; we're only logging a checksum to compare with previous versions). The next token refresh is an hour from now, at 0401 UTC. The most recently refreshed token expires at 0546 UTC. So one hopes that everything works overnight and the tokens are always refreshed before they expire.

If the token has expired, is there a way to programmatically get a new grant? We know the grant we need (read_station).

0

Leslie Community moderator

6 juin 2024 12:26

Hello David,

Indeed we display "invalid_grant" and not something like "expired_token". I'll see with the teams if it's wanted or if it can be more verbose

If the refresh token process is broken for any reason (i.e : the refresh_token value wasn't stored and so is unknown) you have to redo the /authorize process to get the "code" value at user's redirection and use it in your first /token request

The refresh_token value doesn't have an expiration date. So, if for any reason the refresh token request can't be done in time, this value can still be used after the access_token expiration to get connectivity back

Have a good day,

Leslie - Community Manager

1

david

6 juin 2024 20:38

OK, we owe Netatmo an apology, but in our defense, this was not easy to find.

Without going into too much detail, the problem was that Microsoft seems to have removed some obsolete roles on Azure that we didn't know we were using. We store the tokens in an Azure Key Vault. The function app that interacts with Netatmo needs to write to the Key Vault when it receives a new token. Until Monday, the production function app's roles included the appropriate role for writing secrets. Suddenly Monday afternoon, it didn't.

That should have been easy to find, except that the function app had a bit of old code that tried to write the secret synchronously. This happened on a separate thread, which was not synchronized with the function app's main thread. So when the write failure threw an exception, the exception went nowhere.

After a lot of logging and testing (which always worked because (a) the debugger can see all the threads and (b) any developer testing against Key Vault has the appropriate permissions), we figured out what was going on, and replaced the synchronous call with a proper async await. At the next refresh, the production app threw the proper exception (permission denied writing a secret), which enabled us to put the function app in the correct RBAC role and give it the correct permissions.

The last refresh at 2005 UTC worked fine, and the app is talking to the API without a problem.

We will now eat crow for dinner.

2

lakehatori

7 juin 2024 12:07

https://x.com/HatorikoBot/status/1798967523976839225

It seems that #netatmo has strictly applied OAuth2.0 recently, and the acquisition and display of weather data on our site had stopped for the past 2-3 days. I apologize for the inconvenience.

Previously, I had been running it with a self-made Python code, but I was quite busy with my main job, so I wasn't sure what to do. On a whim, I decided to throw the problem to #chatgpt4o, and to my surprise, it came up with a working code on the first try.

The prompt I used was simply: "I want to fully comply with OAuth2.0 section 10.4 in the following Python code. Please create a new code while keeping as much of the original code as possible."
Even without mentioning Netatmo, it seems to have understood my intention from the crude code I had created.

1

Leslie Community moderator

7 juin 2024 12:22

@David I forgive you :D

Thanks for this feedback, it could be helpful for others

Have a good day,

Leslie - Community Manager

0

thierry.home

7 juin 2024 19:14

Hello,

I used this code : https://github.com/philippelt/netatmo-api-python

I've read @david and @lakehatori but i'm so sorry...

I don't understand what i have to do :(

Someone can explain me ?

Thanks a lot.

0

lakehatori

8 juin 2024 12:34

gpt-4o whisper to me.

---
client_id = 'your_client_id'
client_secret = 'your_client_secret'
initial_refresh_token = 'your_initial_refresh_token'

TOKEN_FILE = 'netatmo_tokens.json'

def load_tokens():
if os.path.exists(TOKEN_FILE):
with open(TOKEN_FILE, 'r') as f:
return json.load(f)
return None

def save_tokens(tokens):
with open(TOKEN_FILE, 'w') as f:
json.dump(tokens, f)

tokens = load_tokens()
if not tokens:
tokens = {'refresh_token': initial_refresh_token}
elif 'refresh_token' not in tokens:
tokens['refresh_token'] = initial_refresh_token

refresh_token = tokens['refresh_token']

headers = {'Content-Type': 'application/x-www-form-urlencoded', 'charset': 'UTF-8'}
url = 'https://api.netatmo.com/oauth2/token'
data = {
'grant_type': 'refresh_token',
'refresh_token': refresh_token,
'client_id': client_id,
'client_secret': client_secret,
'scope': 'read_station',
}

try:
refresh_response = requests.post(url=url, headers=headers, data=data)
refresh_response.raise_for_status()
tokens = refresh_response.json()
access_token = tokens["access_token"]
new_refresh_token = tokens["refresh_token"]
print(f'access_token : {access_token}')
print(f'refresh_token : {new_refresh_token}')

save_tokens(tokens)
except requests.exceptions.RequestException as e:
print(f"Error fetching access token: {e}")
print(f"Response content: {e.response.content}")
exit(1)
---

Arigato.

0

thierry.home

9 juin 2024 10:20

Hello all,

specialy thanks to lakehatori.

The script did the job ! wonderfull IA :-)

but finally, the same issue :

Error fetching access token: 400 Client Error: for url: https://api.netatmo.com/oauth2/token
Response content: b'{"error":"invalid_grant"}'

0

antonio.martini

10 juin 2024 12:47

Hello, It's been about 10 days since my php scripts that access my netatmo and publish the data on the web (for example the external temperature) no longer work. The problem is that to obtain an access_token/refresh_token you always go to the page where with manual intervention you have to enter user/passwd of my station for authorization. But I don't think something like that makes sense, should I give this data to anyone on the web? Is it possible to have an example php script that gets the Json with the station data? I'm wasting a lot of time understanding how to do it in one script but without success. Thanks in advance for any help

0

antonio.martini

10 juin 2024 15:29

I would simply like to put the outside temperature on a web page

0

daniel.uhlin

11 juin 2024 17:56

@antonio.martini

I solved my issues with the following code as a plugin on my WordPress site.

<?php
/*
Plugin Name: Netatmo Weather
Description: Hämtar och visar väderdata från Netatmo.
Version: 2.5
Author: Daniel Uhlin
*/

// Konfigurationsinställningar
$client_id = '';
$client_secret = '';
$initial_refresh_token = '';
$token_file = plugin_dir_path(__FILE__) . 'netatmo_tokens.json';

// Funktion för att ladda tokens från fil
function loadTokens($token_file) {
    if (file_exists($token_file)) {
        $json_data = file_get_contents($token_file);
        return json_decode($json_data, true);
    }
    return null;
}

// Funktion för att spara tokens till fil
function saveTokens($token_file, $tokens) {
    $json_data = json_encode($tokens);
    file_put_contents($token_file, $json_data);
}

// Förnya åtkomsttoken
function refreshAccessToken() {
    global $client_id, $client_secret, $token_file, $initial_refresh_token;

    $tokens = loadTokens($token_file);
    if (!$tokens) {
        $tokens = array('refresh_token' => $initial_refresh_token);
    } elseif (!isset($tokens['refresh_token'])) {
        $tokens['refresh_token'] = $initial_refresh_token;
    }

    $refresh_token = $tokens['refresh_token'];

    $headers = array('Content-Type: application/x-www-form-urlencoded', 'charset: UTF-8');
    $url = 'https://api.netatmo.com/oauth2/token';
    $data = array(
        'grant_type' => 'refresh_token',
        'refresh_token' => $refresh_token,
        'client_id' => $client_id,
        'client_secret' => $client_secret,
        'scope' => 'read_station'
    );

    $options = array(
        'http' => array(
            'header'  => implode("\r\n", $headers),
            'method'  => 'POST',
            'content' => http_build_query($data),
        ),
    );

    $context = stream_context_create($options);
    $result = file_get_contents($url, false, $context);

    if ($result === FALSE) {
        die('Error fetching access token');
    }

    $response = json_decode($result, true);
    $access_token = $response["access_token"];
    $new_refresh_token = $response["refresh_token"];

    // Spara nya tokens
    $response['refresh_token'] = $new_refresh_token;
    saveTokens($token_file, $response);

    return $access_token;
}

// Funktion för att hämta data från Netatmo API
function getWeatherData($access_token) {
    $url = 'https://api.netatmo.com/api/getstationsdata';

    $options = array(
        'http' => array(
            'header' => "Authorization: Bearer $access_token\r\n",
            'method' => 'GET',
        ),
    );

    $context  = stream_context_create($options);
    $result = file_get_contents($url, false, $context);
    if ($result === FALSE) {
        die('Error fetching data');
    }

    return json_decode($result, true);
}

// Funktion för att omvandla vindvinkel till vädersträckning
function wind_angle_to_direction($angle) {
    $directions = array('N', 'NNO', 'NO', 'ÖNO', 'O', 'OSO', 'SO', 'SSO', 'S', 'SSV', 'SV', 'VSV', 'V', 'VNV', 'NV', 'NNV');
    $index = round($angle / 22.5) % 16;
    return $directions[$index];
}

// Kortkodsfunktion för att visa väderdata
function netatmo_weather_shortcode() {
    $access_token = refreshAccessToken();
    $weather_data = getWeatherData($access_token);

    // Processa och formatera väderdata
    ob_start();
    ?>
    <h4>Väder Blombergshamn</h4>
    <?php if ($weather_data && isset($weather_data['body']['devices'][0]['modules'])) : ?>
        <?php foreach ($weather_data['body']['devices'][0]['modules'] as $module) : ?>
            <?php if ($module['type'] === 'NAModule1') : ?>
                <?php
                $max_temp_time = isset($module['dashboard_data']['date_max_temp']) ? date('H:i', $module['dashboard_data']['date_max_temp'] + (2 * 3600)) : 'Ingen data';
                $min_temp_time = isset($module['dashboard_data']['date_min_temp']) ? date('H:i', $module['dashboard_data']['date_min_temp'] + (2 * 3600)) : 'Ingen data';
                ?>
                <b><?php echo esc_html($module['module_name']); ?></b>
                <ul>
                    <li>Temperatur: <?php echo esc_html($module['dashboard_data']['Temperature']); ?> °C</li>
                    <li>Max: <?php echo esc_html($module['dashboard_data']['max_temp']); ?> °C (<?php echo esc_html($max_temp_time); ?>)</li>
                    <li>Min: <?php echo esc_html($module['dashboard_data']['min_temp']); ?> °C (<?php echo esc_html($min_temp_time); ?>)</li>
                    <li>Uppdaterad: <?php echo esc_html(date('j M H:i', $module['dashboard_data']['time_utc'] + (2 * 3600))); ?></li>
                </ul>
            <?php elseif ($module['type'] === 'NAModule2') : ?>
                <?php
                    $wind_strength_kmh = $module['dashboard_data']['WindStrength'];
                    $wind_strength_ms = number_format($wind_strength_kmh * 1000 / 3600, 1);
                    $wind_angle = $module['dashboard_data']['WindAngle'];
                    $wind_direction = wind_angle_to_direction($wind_angle);
                    $gust_strength_kmh = $module['dashboard_data']['GustStrength'];
                    $gust_strength_ms = number_format($gust_strength_kmh * 1000 / 3600, 1);
                    $gust_angle = isset($module['dashboard_data']['GustAngle']) ? $module['dashboard_data']['GustAngle'] : 0;
                    $gust_direction = wind_angle_to_direction($gust_angle);
                    $max_gust_speed_kmh = $module['dashboard_data']['max_wind_str'];
                    $max_gust_speed_ms = number_format($max_gust_speed_kmh * 1000 / 3600, 1);
                    $max_wind_angle = isset($module['dashboard_data']['max_wind_angle']) ? $module['dashboard_data']['max_wind_angle'] : 0;
                    $max_wind_direction = wind_angle_to_direction($max_wind_angle);
                    $date_max_wind_str = isset($module['dashboard_data']['date_max_wind_str']) ? date('H:i', $module['dashboard_data']['date_max_wind_str'] + (2 * 3600)) : 'Ingen data';
                ?>
                <b><?php echo esc_html($module['module_name']); ?></b>
                <ul>
                    <li>Vindhastighet: <?php echo esc_html($wind_strength_ms); ?> m/s (<?php echo esc_html($wind_direction); ?>)</li>
                    <li>Byvind: <?php echo esc_html($gust_strength_ms); ?> m/s (<?php echo esc_html($gust_direction); ?>)</li>
                    <li>Maxvind idag: <span style="white-space: nowrap;"><?php echo esc_html($max_gust_speed_ms); ?> m/s (<?php echo esc_html($max_wind_direction); ?>) kl. <?php echo esc_html($date_max_wind_str); ?></span></li>
                    <li>Uppdaterad: <?php echo esc_html(date('j M H:i', $module['dashboard_data']['time_utc'] + (2 * 3600))); ?></li>
                </ul>
            <?php elseif ($module['type'] === 'NAModule3') : ?>
                <b><?php echo esc_html($module['module_name']); ?></b>
                <ul>
                    <li>Regn senaste timmen: <?php echo esc_html($module['dashboard_data']['sum_rain_1']); ?> mm</li>
                    <li>Regn totalt idag: <?php echo esc_html($module['dashboard_data']['sum_rain_24']); ?> mm</li>
                    <li>Uppdaterad: <?php echo esc_html(date('j M H:i', $module['dashboard_data']['time_utc'] + (2 * 3600))); ?></li>
                </ul>
            <?php endif; ?>
        <?php endforeach; ?>
    <?php else : ?>
        <p>Ingen data tillgänglig för närvarande.</p>
    <?php endif;
    return ob_get_clean();
}
add_shortcode('netatmo_weather_data', 'netatmo_weather_shortcode');
?>

0

antonio.martini

Modifié 12 juin 2024 06:50

@daniel.uhlin

Thank you! It works but I think each web call generates a new refresh token. However it works! Very kind

0

daniel.uhlin

12 juin 2024 07:22

@antonio.martini

Great! Here is the complete plugin for wp that store the credential i wp database and have a setting page in the adminpanel for the tokens, this version should just update the tokens when it has expired . change it to the way you want.

Link to the zip file https://blombergsbatsallskap.se/download/netatmo_plugin_WP.zip: https://blombergsbatsallskap.se/download/netatmo_plugin_WP.zip

/Daniel

0

antonio.martini

12 juin 2024 08:14

@daniel.uhlin

Thank you very much for the script. For the moment I leave the old version. When I have some time I'll try to switch to the new one. In the meantime I downloaded the zip file. Thanks again and have a nice day

0

nicolas

24 juin 2024 05:46

Same problem here, I generated a new token on website wednesday, back to works, but yesterday (Sunday) invalid grant again.

I have to check my own code, but can Netatmo confirm that each app token are isolated?

I mean I have 2 apps declared with theirs own credentials (appId, appSecret) on the same Netatmo account. Each app handle its own refresh token.
Question is: when app1 refresh its token that will not invalidate app2 refresh token?

0

Leslie Community moderator

25 juin 2024 14:23

Hello Nicolas,

I confirm that there is no correlation between developer apps coming from a same Netatmo account : app1 will not invalidate tokens from app2 and vice versa

Have a good day,

Leslie - Community Manager

0

lars

5 juillet 2024 17:03

I am still having this issue and it is almost never working or only first time.
Is there a solution i have missed? I am using inetatmo.py on RPI.

0

ak_booer

5 juillet 2024 19:18

Yes, me too, but running Lua code on a variety of machines.

1

API invalid grant at random intervals

Commentaires

<%= block.name %>

Vous avez encore besoin d'aide ?

<% if (block.html_url) { %> <%= block.name %> <% } else { %> <%= block.name %> <% } %>

Catégories

Ouvrir/Fermer le menu de navigation

<%= category.name %>