Selection of variables

[markkvdb]

How do you select which variables will be stored for each weather model? It seems that a subset of variables is made available via open-data and I was wondering if there is some kind of policy regarding this selection.

I started running a local open-meteo API server, which was really easy to setup (so thank you!) and I was trying to collect a few variables for different models. More concretely, I'm interested in serving the cloud coverage, radiation and wind speed (between 100m and 200m) for AROME (HD), ARPEGE, GFS and the different ICON models. I noticed, however, that some models that these variables are not available via this program. For example, it doesn't contain wind speed data from the GFS 13 model.

Is there a reason why these variables are not available? Would it be possible to add these variables for each model? Finally, if it's not possible or it's not a priority right now, could you give some help in how I could for example download this data myself and use a mix of the publicly available data and data that I can download myself?

Thank you for your work and looking forward to your answer!

[markkvdb]

I noticed that the sync command has an option to set the server. If I'd host some of the data on a private S3 bucket, how could I point the sync command to this private S3 bucket? Or should I proceed in another way?

[patrick-zippenfenig]

Wind forecasts at different elevations are not available for all models. GFS013 only offers wind forecasts at 10 meters. Other levels are only available in GFS025. I try to include as many level as possible, but it is limited by the available weather model data.

Yes, you could create your own S3 bucket with data. However, there is no support for S3 authentication. Your bucket needs to publicly accessible. Open-Meteo uses the same sync command internally to distribute data across nodes, but there is a very simple API-Key based authentication. There is limited documentation available to run your own sync-server endpoint here.

You can also use AWS S3 replication or AWS CLI S3 sync command to selectively mirror data.

[markkvdb]

Ah that's my bad. I was using the GFS025 model and assumed that the GFS013 would offer the same data... I might try to get some additional data using a publicly available S3 bucket in that case.

If I understand it well, the data node is a regular API node but with the option to serve data to other nodes enabled. So I could either sync data from this data node with the API key system OR copy the data of the data node to a publicly available S3 bucket and use the sync command with this S3 bucket?

[patrick-zippenfenig]

Yes, a sync-server is just an API node with the sync-endpoint enabled. In practice I am using dedicated servers that only download and process model data, expose a sync-endpoint and pull data via each API node.