Test database and API operations that support new collection versions in SIT/UAT

[nikki-t]

Once version D data is available, we need to test the proposed database and API operations to make sure they function as expected.

[nikki-t]

Also test track ingest operations on new collection version including eventbridge schedule input parameters.

[nikki-t]

Test work from the following PRs/issues:

• Create new database tables for Version D collections #229
• Create new API parameter for user to specify data version #233
• Modify track ingest operations to support new collection versions #267

[nikki-t]

Database test results

PR: #229

Confirmed creation of version D database tables in UAT:

• hydrocron-SWOT_L2_HR_LakeSP_D-prior-lake-table
• hydrocron-SWOT_L2_HR_LakeSP_D-node-table
• hydrocron-SWOT_L2_HR_LakeSP_D-reach-table
• hydrocron-SWOT_L2_HR_LakeSP_D-prior-lake-track-ingest
• hydrocron-SWOT_L2_HR_LakeSP_D-node-track-ingest
• hydrocron-SWOT_L2_HR_LakeSP_D-reach-track-ingest

API test results

PR: #233

Confirmed API operations default to version 2.0 when collection_name query parameter is not provided:

Version 2.0 query: https://soto.podaac.uat.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?feature=Reach&feature_id=45244300191&start_time=2020-01-01T00:00:00%2b00:00&end_time=2025-10-30T00:00:00%2b00:00&output=csv&fields=reach_id,time_str,wse

Version D query: https://soto.podaac.uat.earthdatacloud.nasa.gov/hydrocron/v1/timeseries?collection_name=SWOT_L2_HR_RiverSP_D&feature=Reach&feature_id=45244300191&start_time=2020-01-01T00:00:00%2b00:00&end_time=2025-10-30T00:00:00%2b00:00&output=csv&fields=reach_id,time_str,wse

Track ingest test results

PR: #267

Ran track ingest operations for reaches and nodes.:

Confirmed reaches were reconciled, relevant logs:

2025-01-29T09:21:14.017-05:00 [INFO] 2025-01-29T14:21:14.017Z  Collection shortname: SWOT_L2_HR_RiverSP_reach_D
2025-01-29T09:21:14.017-05:00 [INFO] 2025-01-29T14:21:14.017Z  Hydrocron table: hydrocron-SWOT_L2_HR_RiverSP_D-reach-table
2025-01-29T09:21:14.018-05:00 [INFO] 2025-01-29T14:21:14.017Z  Hydrocron track ingest table: hydrocron-SWOT_L2_HR_RiverSP_D-reach-track-ingest
2025-01-29T09:21:14.018-05:00 [INFO] 2025-01-29T14:21:14.018Z  Temporal indicator for revision dates: True
2025-01-29T09:21:14.018-05:00 [INFO] 2025-01-29T14:21:14.018Z  Temporal start date: 2023-07-27 00:00:00+00:00
2025-01-29T09:21:14.018-05:00 [INFO] 2025-01-29T14:21:14.018Z  Temporal end date: 2024-10-30 23:59:59+00:00
2025-01-29T09:21:14.018-05:00 [INFO] 2025-01-29T14:21:14.018Z  Reprocessed CRID: PGC0
2025-01-29T09:21:14.018-05:00 [INFO] 2025-01-29T14:21:14.018Z  Environment: UAT
2025-01-29T09:21:14.018-05:00 [INFO] 2025-01-29T14:21:14.018Z  Querying CMR temporal range: 2023-07-27 00:00:00+00:00 to 2024-10-30 23:59:59+00:00.
2025-01-29T09:21:15.738-05:00 [INFO] 2025-01-29T14:21:15.738Z  Located 44 granules in CMR.
2025-01-29T09:21:18.313-05:00 [INFO] 2025-01-29T14:21:18.313Z  Located 20 granules NOT in Hydrocron.
2025-01-29T09:21:18.313-05:00 [INFO] 2025-01-29T14:21:18.313Z  Located 18 unique CRID granules NOT in Hydrocron after removing duplicates.
2025-01-29T09:21:29.914-05:00 [INFO] 2025-01-29T14:21:29.914Z  Located 17 granules that require ingestion.
2025-01-29T09:21:29.914-05:00 [INFO] 2025-01-29T14:21:29.914Z  Located 24 granules that are already ingested.
2025-01-29T09:21:52.821-05:00 [INFO] 2025-01-29T14:21:52.821Z  Updated hydrocron-SWOT_L2_HR_RiverSP_D-reach-track-ingest with 41 items.

Confirmed nodes were reconciled, relevant logs:

2025-01-29T14:35:35.066Z [INFO] 2025-01-29T14:35:35.066Z  Collection shortname: SWOT_L2_HR_RiverSP_node_D
2025-01-29T14:35:35.066Z [INFO] 2025-01-29T14:35:35.066Z  Hydrocron table: hydrocron-SWOT_L2_HR_RiverSP_D-node-table
2025-01-29T14:35:35.066Z [INFO] 2025-01-29T14:35:35.066Z  Hydrocron track ingest table: hydrocron-SWOT_L2_HR_RiverSP_D-node-track-ingest
2025-01-29T14:35:35.066Z [INFO] 2025-01-29T14:35:35.066Z  Temporal indicator for revision dates: True
2025-01-29T14:35:35.067Z [INFO] 2025-01-29T14:35:35.066Z  Temporal start date: 2023-07-27 00:00:00+00:00
2025-01-29T14:35:35.067Z [INFO] 2025-01-29T14:35:35.067Z  Temporal end date: 2024-10-30 23:59:59+00:00
2025-01-29T14:35:35.067Z [INFO] 2025-01-29T14:35:35.067Z  Reprocessed CRID: TID3
2025-01-29T14:35:35.067Z [INFO] 2025-01-29T14:35:35.067Z  Environment: UAT
2025-01-29T14:35:35.067Z [INFO] 2025-01-29T14:35:35.067Z  Querying CMR temporal range: 2023-07-27 00:00:00+00:00 to 2024-10-30 23:59:59+00:00.
2025-01-29T14:35:36.543Z [INFO] 2025-01-29T14:35:36.543Z  Located 44 granules in CMR.
2025-01-29T14:35:37.985Z [INFO] 2025-01-29T14:35:37.985Z  Located 20 granules NOT in Hydrocron.
2025-01-29T14:35:37.985Z [INFO] 2025-01-29T14:35:37.985Z  Located 18 unique CRID granules NOT in Hydrocron after removing duplicates.
2025-01-29T14:35:39.430Z [INFO] 2025-01-29T14:35:39.429Z  Located incremented product counter: SWOT_L2_HR_RiverSP_Node_002_565_NA_20230831T062904_20230831T063715_TID3_02.zip.
2025-01-29T14:35:42.097Z [INFO] 2025-01-29T14:35:42.096Z  Located 17 final granules NOT in Hydrocron after product counter filter.
2025-01-29T14:35:42.110Z [INFO] 2025-01-29T14:35:42.110Z  Located 25 granules with 'to_ingest' status.
2025-01-29T14:35:42.111Z [INFO] 2025-01-29T14:35:42.111Z  Located 24 unique granules with 'to_ingest' status. 
2025-01-29T14:36:31.883Z [INFO] 2025-01-29T14:36:31.883Z  Located 17 granules that require ingestion.
2025-01-29T14:36:31.883Z [INFO] 2025-01-29T14:36:31.883Z  Located 24 granules that are already ingested.
2025-01-29T14:36:47.909Z [INFO] 2025-01-29T14:36:47.909Z  Updated hydrocron-SWOT_L2_HR_RiverSP_D-node-track-ingest with 41 items.

[frankinspace]

This is awesome, thank you @nikki-t