Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Can't send 'SpatExtent' object in parallel #1430

Closed
BastienFR opened this issue Feb 16, 2024 · 2 comments
Closed

Can't send 'SpatExtent' object in parallel #1430

BastienFR opened this issue Feb 16, 2024 · 2 comments

Comments

@BastienFR
Copy link

This is a minor issue and with an easy work around, however I still wanted to document the behaviour here so the issue is known.

Sending SpatExtent object to parallel computation fails with a unhelpful error message. I know already that SpatRaster and SpatVector needed to be wrap to allow usage on a cluster, however it seems it's also the case for SpatExtent and there is no method in wrap to do so (it may not be needed either).

RE:

library(terra)
library(parallel)

# prepare raster
f <- system.file("ex/elev.tif", package="terra")
r <- rast(f)

# create a list of extent
ls_ext <- lapply(list(r,r), ext)

# the function to be parallized
my_fc <- function(the_extent){
  library(terra)
  the_extent[c(1,2)]
}

# calling it normally
lapply(ls_ext, my_fc)
[[1]]
    xmin     xmax 
5.741667 6.533333 

[[2]]
    xmin     xmax 
5.741667 6.533333

# calling it in parallel
cl <- makeCluster(2)
parLapply(cl, ls_ext, my_fc)
Error in checkForRemoteErrors(val) : 
  2 nodes produced errors; first error: NULL value passed as symbol address
stopCluster(cl)

It's not a major problem and it can be easily fixed by sending vector instead of SpatExtent and converting them back with ext:

# fixing the function so I do not export a S4 object for the extent
my_fc_fix <- function(the_extent){
  library(terra)
  ext(the_extent)[c(1,2)]
}

cl <- makeCluster(2)
parLapply(cl, lapply(ls_ext, "["), my_fc_fix)
[[1]]
    xmin     xmax 
5.741667 6.533333 

[[2]]
    xmin     xmax 
5.741667 6.533333 
stopCluster(cl)

If you think of an easy way to fix this problem it would be nice, but considering the easy work around, feel free to close this issue right away.

@rhijmans
Copy link
Member

I could add a wrap/unwrap for SpatExtent. It seems unnecessary since as.vector(ext) and similar as you show is so easy. But perhaps it is good for completeness sake; and to make it clear that SpatExtent cannot be passed as-is.

@rhijmans
Copy link
Member

I have added these methods.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jan 5, 2025
# version 1.8-5

## bug fixes

- `Spatsample(method='stratified', ext=e)` returned the wrong sampling
  coordinates [#1628](rspatial/terra#1628)
  by Barnabas Harris

- `spatSample(method='stratified')` could fail with small sample sizes
  [#1503](rspatial/terra#1503) by karluf

- transparency (alpha) did not work with RGB
  plotting. [#1642](rspatial/terra#1642) by
  Timothée Giraud

- rasterization failed on very large rasters
  [#1636](rspatial/terra#1636) by Mary
  Fisher, [#1463](rspatial/terra#1463) by
  Nic Spono and [#1281](rspatial/terra#1281)
  by Sebastian Dunnett

- `tmpFiles` only looked in the default temp files folder
  [#1630](rspatial/terra#1630) by
  smckenzie1986

- `where.min` did not work well if there were negative values
  [#1634](rspatial/terra#1634) by Michael
  Sumner

- `plet<SpatRaster>` now works for RGB rasters and rasters with a
  color table [#1596](rspatial/terra#1596)
  by Agustin Lobo

- `vect<MULTIPOINT WKT>` did not work properly
  [#1376](rspatial/terra#1376) by
  silasprincipe

- `compareGeom<SpatVector>` did not work
  [#1654](rspatial/terra#1654) by Jason
  Flower

- `buffer<SpatVector>` is now more accurate buffers for lonlat
  polygons [#1616](rspatial/terra#1616) by
  Roberto Amaral-Santos

- `terra:interpNear` used square windows, not circles, beyond 100
  points [#1509](rspatial/terra#1509) by
  Jean-Luc Dupouey

- `vect` read INT64 fields as integers, sometimes leading to
  overflows. [#1666](rspatial/terra#1666) by
  bengannon-fc

- `plot` showed a legend title even if none was requestd if title
  parameters were specified
  . [#1664](rspatial/terra#1664) by Márcia
  Barbosa



## enhancements

n- improved documentation of `writeVector` overwrite when using
  layers. [#1573](rspatial/terra#1573) by
  Todd West

- improved treatment of (supposedly) flipped rasters by Timothée
  Giraud [#1627](rspatial/terra#1627) and
  fchianucci [#1646](rspatial/terra#1646)

- added `map.pal("random")`
  [#1631](rspatial/terra#1631) by Agustin
  Lobo

- expressions can now be used in legend titles
  [#1626](rspatial/terra#1626) by Noah
  Goodkind

- `app` and `tapp` now emit a warning when factors are coerced to
  numeric [#1566](rspatial/terra#1566) by
  shuysman

- `plet<SpatRaster>` now has argument "stretch" for RGB rasters
  [#1596](rspatial/terra#1596) by Agustin

- `%%` and `%/%` now behave the same for SpatRaster as for (base R)
  numbers [#1661](rspatial/terra#1661) by
  Klaus Huebert

## new

- `patches` with option `valus=TRUE` can now distinguish regions based
  on their cell values (instead of only NA vs not-NA)
  [#495](rspatial/terra#495) by Jakub
  Nowosad and [#1632](rspatial/terra#1632)
  by Agustin Lobo

- `rowSums`, `rowMeans`, `colSums` and `colMeans` for SpatRaster

- `metags` for SpatRasterDataset
  [#1624](rspatial/terra#1624) by Andrea
  Manica

- `metags` for layers (bands) of SpatRaster are now saved to and read
  from GTiff files
  [#1071](rspatial/terra#1071) by Mike
  Koontz

- `global` has new effcient functions "anyNA" and "anynotNA"
  [#1540](rspatial/terra#1540) by Kevin J
  Wolz

- `wrap`, `saveRDS` and `serialize` for
  SpatExtent. [#1430](rspatial/terra#1430)
  by BastienFR

- `vect<SpatGraticule>` method suggested in relation to [tidyterra
  #155](dieghernan/tidyterra#155) by Diego
  Hernangómez

- `toMemory<SpatRaster>` and `<SpatRasterDataset>` methods
  [#1660](rspatial/terra#1660) by Derek Friend
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants