Wind speed derivation from 'uas' and 'vas'

[malininae]

Hi,

I might have missed something, but I was wondering if there's interest in a sfcwind variable derivation from uas and vas? It's may be not super important for the CMIP data, but it is for ERA5 (and may be other reanalysis or observation). In couple of weeks, I might come to that, if others are supportive.

[valeriupredoi]

cheers, Liza! I support anything that scientists may need - I can assist with the PR 👍 How do you think @ESMValGroup/scientific-lead-development-team ?

[malininae]

@valeriupredoi the other option is to create sfcWind directly in ERA5 cmorizer, but I'm not sure it's such a good idea. As said, I'll come to that hopefully next week. I hope to join this month's meeting, can briefly discuss that issue there.

[katjaweigel]

Thanks for the issue @malininae. I would be also interested (thought about it before but never calculated it so far). I think one issue we must be aware of, is the nonlinearity of the calculation. It should be fine to calculate hourly sfcWind from hourly uas and vas, but I'm not sure, how the daily/monthly sfcWind is defined, i.e. if it needs to be calculated as mean of the hourly sfcWind or if we can use the monthly mean uas and vas directly?

[malininae]

Thanks for the issue @malininae. I would be also interested (thought about it before but never calculated it so far). I think one issue we must be aware of, is the nonlinearity of the calculation. It should be fine to calculate hourly sfcWind from hourly uas and vas, but I'm not sure, how the daily/monthly sfcWind is defined, i.e. if it needs to be calculated as mean of the hourly sfcWind or if we can use the monthly mean uas and vas directly?

@katjaweigel, a very good point! I did a brief analysis of the daily sfcWind for one extratropical storm comparing ERA5 and HighResMIP models, but what I did there, I processed ERA5 in the diagnostic (again, naughty me) and calculated hourly sfcWind and then took a daily mean from there. It was a quick analysis, just to get a rough estimate, but I wanted to look into it further. I'll need to double check the script, but isn't ERA5 cmorized as daily uas and vas? So may be the cmorizer should be done for the hourly tables, or indeed we might want to add the sfcWind calculation into the ERA5 cmorizer.

[katjaweigel]

@malininae yes, for hourly to daily that could be probably included in the cmorizer, because there are no daily raw ERA5 data in any way. I'm more concerned about monthly data: There is monthly uas and vas. I'm not 100% sure if monthly sfcWind is defined as mean hourly sfcWind or can be calculated from monthly uas and vas (the second would be much more convenient due to data amount but we should not do it, if it would be not defined that way).

[malininae]

@malininae yes, for hourly to daily that could be probably included in the cmorizer, because there are no daily raw ERA5 data in any way. I'm more concerned about monthly data: There is monthly uas and vas. I'm not 100% sure if monthly sfcWind is defined as mean hourly sfcWind or can be calculated from monthly uas and vas (the second would be much more convenient due to data amount but we should not do it, if it would be not defined that way).

@katjaweigel oh yeah, those might be problematic. As an option, if one is bored, we can look at how do the monthly means from monthly mean uas and vas look in comparison to the monthly mean sfcWind derived from the hourly data.

[malininae]

@valeriupredoi @katjaweigel Sorry, two weeks turned into eight months :-) But, I sort of created the function, but would need an advice: we derive sfcWind, which is an existing CMOR variable. For my current application, we need the sfcWind with 3hr frequency, so I ended up using uas and vas from 3hr table, but the sfcWind is in CF3hr. I ended up creating a custom CMOR table, which fixed that issue, but there is another problem, namely in the list of all available derivable datasets

ESMValCore/esmvalcore/preprocessor/_derive/__init__.py

Line 93 in bb66bb3

DerivedVariable = ALL_DERIVED_VARIABLES[short_name.lower()] # noqa: N806

there is a reference to the ALL_DERIVED_VARIABLES[short_name.lower()] and in

ESMValCore/esmvalcore/preprocessor/_derive/__init__.py

Line 59 in bb66bb3

DerivedVariable = ALL_DERIVED_VARIABLES[short_name] # noqa: N806

there is a reference to the ALL_DERIVED_VARIABLES[short_name]. So what ends up happening for the variable named sfcWind, if the script filename is sfcWind.py there is an error in line 93, but if it is called sfcwind.py then it fails in line 59. Since there is a comment # noqa: N806, I suspect the issue was already raised. There are two ways to deal with it, first to make the call of the ALL_DERIVED_VARIABLES more consistent, or to rename the variable as let's say dersfcwind as derived_near_surface_wind_speed referring that it's derived from uas and vas. What do you think would be the best way?

[valeriupredoi]

@malininae two points: that noqa is there to tell pep8 (standards) to go for a walk ie standard N806 (variable in function should be lowercase) is ignored. About the derived var issue - could you point me to a PR or branch and give me the errors you get please?

[malininae]

@valeriupredoi sorry, we had Canadian Thanksgiving, so didn't answer right away :-) The branch is https://github.com/ESMValGroup/ESMValCore/tree/dev_derive_sfcWind I am attaching the recipe and the log file. Since there is no specific reason why ALL_DERIVED_VARIABLES should be lower case, I propose to remove .lower() in line 93. If you approve, I can do it once I will make a PR.

recipe_test_sfcWind.yml.txt
main_log.txt

[malininae]

@valeriupredoi sorry, we had Canadian Thanksgiving, so didn't answer right away :-) The branch is https://github.com/ESMValGroup/ESMValCore/tree/dev_derive_sfcWind I am attaching the recipe and the log file. Since there is no specific reason why ALL_DERIVED_VARIABLES should be lower case, I propose to remove .lower() in line 93. If you approve, I can do it once I will make a PR.

recipe_test_sfcWind.yml.txt main_log.txt

@valeriupredoi it looks like getting rid of .lower() in line 93 fixes the issue. I will write a test tomorrow and see if the other tests fail, and if not submit a PR shortly.

[bouweandela]

The .lower() is there because the convention is that Python modules are always named all lowercase (so the file would need to be called esmvalcore/preprocessor/_derive/sfcwind.py). So instead of removing it on line 93, it needs to be added here

ESMValCore/esmvalcore/preprocessor/_derive/__init__.py

Line 55 in bb66bb3

if short_name not in ALL_DERIVED_VARIABLES:

and here

ESMValCore/esmvalcore/preprocessor/_derive/__init__.py

Line 59 in bb66bb3

DerivedVariable = ALL_DERIVED_VARIABLES[short_name] # noqa: N806

and that should solve the issue.

To compute the 3-hourly data from the hourly data, you can use the hourly_statistics preprocessor function with the argument hour: 3

[bouweandela]

Is this done now that #2242 has been merged?