tmpFiles() does not recognized orphaned temporary files if terra option tempdir is not the default tempdir()

[smckenzie1986]

Hi Robert,
I ran into a little snag trying to do parallel processing with large raster tiles and many geoprocessing steps. To avoid overloading my memory, I write all intermediate steps as temporary files to a folder on an external hard drive. I have been trying to delete the temporary files between each iteration of a foreach() loop, but I got hung up because I couldn't get tmpFiles() to recognize just orphan files after I had removed their corresponding spatRasters. I dug into the source code, and found out that the reason I ran into this problem was that the internal function terra:::.orphanTmpFiles() only searches the default temporary directory generated by tempdir(). Below is a reproducible example showing the behavior I described, and the proposed solution I came up with. Could this potentially be integrated into a future version of terra? Thanks as always for such a powerful and user-friendly package!

##Example of behavior##
library(terra)

othertmpdir<-"C:/Example"

if(!dir.exists(othertmpdir)){
  dir.create(othertmpdir)
}

terraOptions(tempdir="C:/Example", todisk=TRUE)

r<-rast(xmin= -122, xmax=-121, ymin=45, ymax=45.5, nrows=1000, ncols=1000, crs="epsg:4326")
values(r)<-rnorm(100000, 500, 50)
r_6557<-project(r, "epsg:6557") *100

tmpFiles()

rm(r)

tmpFiles(current=FALSE, orphan=TRUE)


####Proposed fix####
orphan_files<-function () 
{
  objects <- ls(envir = globalenv())
  ftmp <- list()
  for (i in seq_along(objects)) {
    x <- get(objects[i], envir = globalenv())
    if (inherits(x, "SpatRaster")) {
      ftmp[[i]] <- sources(x)
    }
  }
  ftmp <- unique(unlist(ftmp))
  ftmp <- ftmp[ftmp != ""]
  pattrn <- "^spat_.*tif$"
  i <- grep(pattrn, basename(ftmp))
  ftmp <- ftmp[i]
  ff <- list.files(terraOptions()$tempdir, pattern = pattrn, full.names = TRUE) #This is the only line that I changed from your source code
  i <- !(basename(ff) %in% basename(ftmp))
  ff[i]
}

orphan_files()


new_tmpFiles<-function (current = TRUE, orphan = FALSE, old = FALSE, remove = FALSE) 
{
  if (!(old | current | orphan)) {
    error("tmpFiles", "at least one of 'orphan', 'current' and 'old' must be set to TRUE")
  }
  opt <- terra:::spatOptions()
  d <- opt$tempdir
  f <- NULL
  if (old) {
    if (normalizePath(tempdir()) != normalizePath(d)) {
      warn("tmpFiles", "old files can only be found if terra uses the R tempdir")
    }
    else {
      f <- list.files(dirname(d), recursive = TRUE, pattern = "^spat_", 
                      full.names = TRUE)
      f <- grep("Rtmp", f, value = TRUE)
      if ((length(f) > 0) && (!current)) {
        i <- grep(d, f)
        if (length(i) > 0) {
          f <- f[-i]
        }
      }
    }
  }
  if (current) {
    ff <- list.files(d, pattern = "^spat", full.names = TRUE)
    f <- c(f, ff)
  }
  else if (orphan) {
    fo <- orphan_files()
    f <- c(f, fo)
  }
  if (remove) {
    file.remove(f)
    return(invisible(f))
  }
  else {
    return(f)
  }
}

new_tmpFiles(current=FALSE, orphan=TRUE)

[rhijmans]

Thanks, that was indeed an oversight. I think this commit fixes it.