Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Invoke SNAP on master/slave pairs store to readable format (Zarr, Geotiff) #1034

Closed
EmileSonneveld opened this issue Feb 4, 2025 · 2 comments
Assignees

Comments

@EmileSonneveld
Copy link
Contributor

EmileSonneveld commented Feb 4, 2025

Duplicate of cloudinsar/s1-workflows#3

@EmileSonneveld
Copy link
Contributor Author

EmileSonneveld commented Feb 6, 2025

Not all PVCs need to be s3 mounted in local tests. This might give better performance on CDSE staging:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: calrissian-input-data
  namespace: calrissian-demo-project
spec:
  accessModes:
    - ReadWriteOnce
    - ReadOnlyMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: calrissian-tmpout
  namespace: calrissian-demo-project
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: calrissian-output-data
  namespace: calrissian-demo-project
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: csi-s3-calrissian

This might avoid errors like: cwltool.errors.WorkflowException: ../calrissian/input-data/r-25020610512247ea8b-cal-inp-46968e7e.cwl:18:7: Did not find output file with glob pattern: '['output.txt']' (for openEO request r-25020610512247ea8b4c994a5e78a650 )

@EmileSonneveld
Copy link
Contributor Author

Example script on how to select bursts and preprocess with calrissian.

import openeo
import openeo.processes
import openeo.internal

url = "https://openeo-staging.dataspace.copernicus.eu/"
# url = "https://openeo.dev.warsaw.openeo.dataspace.copernicus.eu/"  # needs VPN
# url = "http://127.0.0.1:8080"
connection = openeo.connect(url).authenticate_oidc()

datacube = openeo.rest.datacube.DataCube(
    openeo.rest.datacube.PGNode(
        "_cwl_insar",
        arguments={
            "spatial_extent": {
                "west": 10.751,
                "south": 46.741,
                "east": 10.759,
                "north": 46.749
            },
            "temporal_extent": [
                "2024-08-14",
                "2024-08-26"
            ]
        },
    ),
    connection=connection,
)

datacube.download("cwl_result.json")  # TODO: Export as datacube

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant