Changing GDAL version in default docker image or AWS lambda runtime

[scottyhq]

Hi! I was just trying out the SAM deployment with some customizations and was surprised that the GDAL version is a bit older (3.6.4 vs 3.8). Is there a reason for keeping that pinned to an older version? Some environment variable config options for example are only available for >=3.7 for example (https://gdal.org/user/configoptions.html#global-configuration-options).

docker run --rm -it ghcr.io/developmentseed/titiler:0.15.6 rio --show-versions

rasterio info:
  rasterio: 1.3.9
      GDAL: 3.6.4
      PROJ: 9.0.1
      GEOS: 3.11.1
 PROJ DATA: /opt/bitnami/python/lib/python3.11/site-packages/rasterio/proj_data
 GDAL DATA: /opt/bitnami/python/lib/python3.11/site-packages/rasterio/gdal_data

System:
    python: 3.11.0 (main, Nov 28 2022, 06:24:00) [GCC 10.2.1 20210110]
executable: /opt/bitnami/python/bin/python
   machine: Linux-6.4.16-linuxkit-x86_64-with-glibc2.31

Python deps:
/opt/bitnami/python/lib/python3.11/site-packages/setuptools-63.4.3-py3.11.egg/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/opt/bitnami/python/lib/python3.11/site-packages/setuptools-63.4.3-py3.11.egg/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
    affine: 2.4.0
     attrs: 23.1.0
   certifi: 2023.07.22
     click: 8.1.3
     cligj: 0.7.2
    cython: None
     numpy: 1.26.2
    snuggs: 1.4.7
click-plugins: None
setuptools: 63.4.3

[vincentsarago]

Hi @scottyhq

The version in the Lambda layer is determined by rasterio wheels https://github.com/rasterio/rasterio-wheels/blob/main/env_vars.sh#L11

You can create your own layer either compiling GDAL or using https://github.com/lambgeo/docker-lambda (which was just updated with 3.8 yesterday 😅 )

[scottyhq]

Thanks for the pointers! This wouldn't be the first time I was confused by the pypi wheels dependencies versus conda-forge dependencies (conda search --info rasterio=1.3.9 | grep gdal. --> libgdal >=3.7.2,<3.8.0a0 I'll give customizing the lambda layer a try

[scottyhq]

@vincentsarago I just ran into this again while experimenting with new GDAL 3.10 GTI abilities 😅. I was thinking it could be nice to include the output of rasterio.show_versions() somewhere. Maybe on the landing page? Or alongside the /healthz endpoint? If you think that'd be a useful thing I could take a stab at it with a bit of guidance :)

[vincentsarago]

Or alongside the /healthz endpoint? If you think that'd be a useful thing I could take a stab at it with a bit of guidance :)

That's a good idea 🙏

we just need to update https://github.com/developmentseed/titiler/blob/main/src/titiler/application/titiler/application/main.py#L209-L218

FYI: I'm head down on project work until the end of the year and I'll pickup open-source work next year only 🙏