S3 Support

[matcor-bodc]

Hello,

We've recently looked into using our ERRDAP instance with a Minio S3 object store. Unfortunately it looks like ERDDAP currently only works with AWS S3 locations. Is this something that could be added in? It certainly would be a big help for us and hopefully could be useful for others

[BobSimons]

Yes. Someone else mentioned this.

When I wrote the S3 support, it only existed at Amazon. I didn't anticipate that would change, but I suspect there is a straightforward way to deal with this.

Currently, if ERDDAP sees that a file's path matches String2.AWS_S3_REGEX (which is Amazon-specific), it knows to deal with the file via the AWS S3 Java API. Since files in your S3 installation don't match that regex, ERDDAP just sees them as goofy/invalid file names. Perhaps there could be a new tag (in datasets.xml?) to specify additional regexes for other S3 installations. I'll leave it to Chris to consider this.

[JessyBarrette]

@ChrisJohnNOAA and whoever it concerns sorry for the delay. We've generated a test public bucket from a none Amazon s3 bucket here for testing purposes.
https://s3.valeria.science/ul-val-prj-amundsen-erddap/

Our hope is to be able to serve that data via ERDDAP. Here's the dataset.xml section associated with this dataset when served from a local copy:
https://git.valeria.science/amundsen/erddap-amundsen-science/-/blob/main/datasets.d/amundsen11943.xml