Unrecognized client exception when calling the GetTableMetadata operation: The security token included in the request is invalid.

[NatalieLow]

Issue overview

When attempting to use the buildstock-query notebooks, I'm getting a Client / Operational Error.

Current Behavior

Expected Behavior

Steps to Reproduce

from buildstock_query import BuildStockQuery
(works fine)
2.
aws configuration
(enter keys, region, etc.)
Account has:

• AdministratorAccess
• AmazonAthenaFullAccess
• AmazonS3FullAccess

my_run = BuildStockQuery(workgroup='rescore', db_name='euss-final', table_name='euss_res_final_2018_550k_20220901')
This and various other table names and working groups produce Client / Operational errors:
OperationalError: An error occurred (UnrecognizedClientException) when calling the GetTableMetadata operation: The security token included in the request is invalid.

Possible Solution

Details

Environment

Some additional details about your environment for this issue (if relevant):

• Platform (Operating system, version): MacOS Sonoma 14.4.1, Apple M2
• Version of OpenStudio (if using an intermediate build, include SHA): OS: 3.7.0, 3.6.1, 3.5.1, 3.4.0
• Python versions attempted: 3.10.13, 3.12.0

Context

We're able to query and access the Data Lake via AWS Athena interface, but would also like to use the python querying tools to more easily interface with our analysis of residential data.

[rajeee]

Are you sure you have access to that particular database and table in Athena with your user account?

[NatalieLow]

Hi!

With this user account I am able to use AWS glue and set up a crawler to create tables from the oedi data lake, then run queries in Athena. I'm assuming this means I have access?

[rajeee]

You should be able to go to Athena console and see those specific db and table name. If you can't, then you will need to first create those. Please follow the instructions here: https://www.youtube.com/watch?v=qSR1MFpSiro&t=0s

[NatalieLow]

I still seem to be missing something.

In Athena, everything seems to work well.

I have two tables: one for the metadata and one for the time series data.

I've also tried specifying the db_schema = resstock_oedi as it maps to my table suffixes.

Do you have any other ideas?

Thank you!

[rajeee]

Thanks for the screenshot. That's very helpful.

It seems like somehow the credentials being pulled is not granting you the required permission.
Could you maybe try by adding this in the beginning.

import os
os.environ['AWS_ACCESS_KEY_ID'] = '<access_key>' 
os.environ['AWS_SECRET_ACCESS_KEY']  = '<secret_key>'

[NatalieLow]

It worked. Thank you!

I needed a couple of additional steps:

1. Added my session token in addition to the key ID and access key:
   os.environ['AWS_SESSION_TOKEN'] = '<session token>'

2. Specified a location for the query results in the workgroup