Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

RasterSource attempts to read hadoop scheme with gdal reader #314

Open
vpipkt opened this issue Aug 23, 2019 · 0 comments
Open

RasterSource attempts to read hadoop scheme with gdal reader #314

vpipkt opened this issue Aug 23, 2019 · 0 comments
Labels
bug When it really isn't a "feature". good first issue Good for newcomers
Milestone

Comments

@vpipkt
Copy link
Member

vpipkt commented Aug 23, 2019

Trying to read a hadoop scheme with gdal installed such as s3a://${bucket}/${prefix}.TIF produces an error such as below message/stack trace.

This seems consistent with documented use of HDFS versus GDAL schemes and such: which is the s3n / s3a schemes are not compatibal with GDAL read and default to hadoop read.

However in the stack trace we see that the behavior is to read the s3a scheme URI with the GDAL reader (note GDALRasterSource). Roughly here needs a check for schemes perhaps: https://github.com/locationtech/rasterframes/blob/0.8.0/core/src/main/scala/org/locationtech/rasterframes/ref/RasterSource.scala#L104-L138

stack trace

ERROR 4: landsat-pds/c1/L8/190/028/LC08_L1TP_190028_20190705_20190719_01_T1/LC08_L1TP_190028_20190705_20190719_01_T1_B8.TIF: No such file or directory
2019-08-23 17:39:45 ERROR RasterSourceToRasterRefs:77 - Error fetching data for one of: GDALRasterSource(s3a://landsat-pds/c1/L8/190/028/LC08_L1TP_190028_20190705_20190719_01_T1/LC08_L1TP_190028_20190705_20190719_01_T1_B8.TIF)
java.lang.Exception: get_transform or get_widht_height
	at geotrellis.contrib.vlm.gdal.GDALDataset$.rasterExtent$extension1(GDALDataset.scala:45)
	at geotrellis.contrib.vlm.gdal.GDALDataset$.rasterExtent$extension0(GDALDataset.scala:37)
	at geotrellis.contrib.vlm.gdal.GDALBaseRasterSource$class.rasterExtent(GDALBaseRasterSource.scala:69)
	at geotrellis.contrib.vlm.gdal.GDALRasterSource.rasterExtent$lzycompute(GDALRasterSource.scala:22)
	at geotrellis.contrib.vlm.gdal.GDALRasterSource.rasterExtent(GDALRasterSource.scala:22)
	at geotrellis.contrib.vlm.RasterSource$class.cols(RasterSource.scala:76)
...
...
...

Originally posted by @vpipkt in #38 (comment)

@vpipkt vpipkt added this to the 0.8.2 milestone Aug 23, 2019
@metasim metasim added the bug When it really isn't a "feature". label Aug 23, 2019
@metasim metasim modified the milestones: 0.8.2, 0.8.3 Sep 23, 2019
@metasim metasim modified the milestones: 0.8.3, 0.8.4 Oct 9, 2019
@metasim metasim added the good first issue Good for newcomers label Sep 10, 2020
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug When it really isn't a "feature". good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants