Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Issue/164: Upgrade harmony-py to avoid 413 request entity too large issue #165

Merged
merged 2 commits into from
Mar 14, 2024

Conversation

skorper
Copy link
Contributor

@skorper skorper commented Mar 14, 2024

See #164

The Harmony team released a new version which uses a POST request instead of a GET request when submitting Harmony jobs, which fixes our "413 request entity too large" error.

Testing:

poetry run python3 subscriber/podaac_data_subscriber.py -c SWOT_L2_LR_SSH_BASIC_2.0 -d ./data --start-date 2023-05-01T00:00:00Z --end-date 2024-12-21T00:00:00Z --subset -b="120,-30,160,20"
[2024-03-14 10:41:58,354] {podaac_data_subscriber.py:183} WARNING - No .update__SWOT_L2_LR_SSH_BASIC_2.0 in the data directory. (Is this the first run?)
[2024-03-14 10:42:07,065] {subsetting.py:102} INFO - Waiting for Harmony subsetting job to complete...
./data/SWOT_L2_LR_SSH_Basic_006_547_20231122T012915_20231122T022044_PIC0_01_subsetted.nc4
...
./data/SWOT_L2_LR_SSH_Basic_012_019_20240307T091643_20240307T100711_PIC0_01_subsetted.nc4
[2024-03-14 10:57:00,828] {podaac_data_subscriber.py:298} INFO - END

(removed lines from output for brevity)

@skorper skorper merged commit e5f8205 into release/1.15.1 Mar 14, 2024
skorper added a commit that referenced this pull request Mar 20, 2024
* initial implementaiton of subset code

* added tests, more updates

* removed skip_errors (not implemented yet)

* Upgrade to harmony-py 0.4.9

* subsetting capability initial commit

* update spacing

* Update docs

* update poetry lock

* upgrade to poetry 1.5.1 (latest)

* fix failing checksum test

* fixed regression test issues

* bump poetry version in second gh action file

* Update poetry version to fix build issue

* bump poetry version

* bump poetry version

* bump version to 1.15.0

* bump version to 1.15.0

* add unreleased section to changelog

* issue/161: Fix bug where --subset flag fails with data subscriber (#162)

* fix bug where the subset flag fails for subscriber

* move arg validation to correct spot

* bump version

* bump version

* Issue/164: Upgrade harmony-py to avoid 413 request entity too large issue (#165)

* bump harmony-py version to 0.4.12

* /version 1.15.1-alpha.2

---------

Co-authored-by: mike-gangl <michael.e.gangl@jpl.nasa.gov>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants