configure sen2r to use Google Cloud SDK launching the function",
+ "check_gcloud().
",
+ "For details, see this post."
+ )),
easyClose = TRUE,
footer = NULL
))
})
- observeEvent(input$help_server, {
+ observeEvent(TRUE, { # TEMPORARY show this when the GUI is open
showModal(modalDialog(
title = "Input SAFE servers",
p(HTML(
- "Input Sentinel-2 SAFE archives can be searched and retrieved from",
- "two sources:",
- "
ESA Hub, the official data source, is",
- "the default and legacy option; to use it, SciHub credentials",
- "must be provided using the button \"Login to SciHub\";
",
- "
Google Cloud is an optional third-party",
- "data source; to use it, Google Cloud SDK must be installed",
+ "target='_blank'>Google Cloud, a third-party",
+ "data source; to use it, Google Cloud SDK must be installed",
"and configured following the official instructions.
",
- "The two data sources can be also used in combination (in this case,",
- "archives are searched on Google Cloud first, and on ESA Hub subsequently)."
+ "href='https://cloud.google.com/sdk/docs/install'",
+ "target='_blank'>official instructions.",
)),
p(HTML(
- "By default, SAFE archives are searched on ESA Hub.",
- "After the reduction of the retention time to 30 days,",
- "it is highly probable that products older then 30 days will not be",
- "found online (see this issue).",
- "Moreover, after ordering them from Long Term Archive (LTA),",
- "in several cases corrupted archives are obtained (see this issue), and refer to this reference",
- "for details about the ESA LTA policy).",
- "To avoid this problems, the research and download from Google Cloud",
- "was implemented."
+ "This mode is not optimal (a modern STAC API approach should be used),",
+ "and searching products could be a very slow process.",
+ "Nevertheless, the old official data source, ESA Hub, is no",
+ "longer available since November 2023 due to it discontinuation;",
+ "since sen2r is not actively maintained, the replacement with the new",
+ "Copernicus Data Space",
+ "is not planned (anyone who wants to help can contribute on GitHub)."
)),
p(HTML(
- "Notice that Google Cloud is an experimental data source;",
- "in case of problems, please report them opening a",
- "GitHub Issue."
+ "Notice that the Online mode is available only if",
+ "Google Cloud SDK is correctly installed and configured; to do it:",
+ "
configure sen2r to use Google Cloud SDK launching the function",
+ "check_gcloud().
",
+ "For details, see this post."
)),
easyClose = TRUE,
footer = NULL
@@ -3136,25 +3149,25 @@ s2_gui <- function(param_list = NULL,
))
})
- observeEvent(input$help_register_scihub, {
- showModal(modalDialog(
- title = "New/edit SciHub credentials",
- size = "s",
- p(HTML(
- "Notice that SciHub credentials are recognised by API Hub",
- "(used by sen2r) with a delay of one week (see",
- "this alert);",
- "for this reason, newly created credentials and password edits",
- "are generally not immediately recognised."
- )),
- a("Register new account", href="https://scihub.copernicus.eu/dhus/#/self-registration", target="_blank"),
- "\u2000\u2014\u2000",
- a("Forgot password?", href="https://scihub.copernicus.eu/dhus/#/forgot-password", target="_blank"),
- easyClose = TRUE,
- footer = NULL
- ))
- })
+ # observeEvent(input$help_register_scihub, {
+ # showModal(modalDialog(
+ # title = "New/edit SciHub credentials",
+ # size = "s",
+ # p(HTML(
+ # "Notice that SciHub credentials are recognised by API Hub",
+ # "(used by sen2r) with a delay of one week (see",
+ # "this alert);",
+ # "for this reason, newly created credentials and password edits",
+ # "are generally not immediately recognised."
+ # )),
+ # a("Register new account", href="https://scihub.copernicus.eu/dhus/#/self-registration", target="_blank"),
+ # "\u2000\u2014\u2000",
+ # a("Forgot password?", href="https://scihub.copernicus.eu/dhus/#/forgot-password", target="_blank"),
+ # easyClose = TRUE,
+ # footer = NULL
+ # ))
+ # })
# observeEvent(input$fix_online, {
# showModal(modalDialog(
diff --git a/R/s2_list.R b/R/s2_list.R
index c30333c9..e6958066 100644
--- a/R/s2_list.R
+++ b/R/s2_list.R
@@ -1,30 +1,11 @@
#' @title Retrieve list of available S2 products.
#' @description The function retrieves the list of available Sentinel-2
#' products satisfying given search criteria.
-#' @details By default, SAFE archives are searched on ESA Sentinel Hub
-#' (argument `server = "scihub"`), which is the faster and stable option.
-#' After the reduction of the retention time to 30 days,
-#' it is highly probable that products older then 30 days will not be found
-#' online (see \url{https://github.com/ranghetti/sen2r/issues/408}).
-#' Moreover, after ordering them from Long Term Archive (LTA), in several cases
-#' corrupted archives are obtained
-#' (see \url{https://github.com/ranghetti/sen2r/issues/406}, and refer to this
-#' reference for details about the ESA LTA policy).
-#'
-#' To avoid this problems, the research and download from Google Cloud was
-#' recently implemented.
-#' Users can set `server = "gcloud"` to use this data exclusively, or
-#' `server = c("scihub", "gcloud"), availability = "check"`
-#' to search for products available on SciHub first, or on Google Cloud
-#' subsequently.
-#' **Important**: to search and download from Google Cloud, Google Cloud SDK
+#' @details **Important**: to search and download from Google Cloud, Google Cloud SDK
#' must be installed and configured following the indications in
#' \url{https://cloud.google.com/sdk/docs/install}.
#' Notice also that querying Google Cloud can be very slow (while downloading
#' them is generally faster than from SciHub).
-#'
-#' Searching and downloading from Google Cloud is an experimental feature;
-#' in case of bugs, report them at \url{https://github.com/ranghetti/sen2r/issues}.
#' @param spatial_extent A valid spatial object object of class `sf`,
#' `sfc` or `sfg`
#' @param tile `string array` Sentinel-2 Tiles to be considered string (5-length character)
@@ -44,26 +25,22 @@
#' product
#' - "L1C": list available level-1C products
#' - "L2A": list available level-2A products
-#' @param server The servers where archives are searched.
-#' Available options are `"scihub"` (ESA Sentinel Hub) and `"gcloud"`
-#' (Google Cloud).
-#' Default is `"scihub"`, meaning that only ESA Sentinel Hub is considered.
-#' In case of multiple values, they are used in order of priority.
-#' If `availability = "check"`, products on LTA are always left as last choice.
+#' @param server (deprecate) Character vector of length 1, with the names of
+#' the servers on which SAFE archives are searched.
+#' Currently, only `"gcloud"` (Google Cloud) is supported.
+#' Old `"scihub"` (ESA Sentinel Hub) can no more be used, since November 2023,
+#' when the Copernicus Sentinel Data is no longer available and has been
+#' replaced by the Copernicus Data Space Ecosystem.
#' See also the section "Details".
-#' @param apihub Path of the `apihub.txt` file containing credentials
-#' of SciHub account.
-#' If NA (default), the default location inside the package will be used.
-#' @param service Character: it can be `"dhus"` or `"apihub"` (default),
-#' in which cases the required service is forced instead that the one present
-#' in the URLs passed through argument `s2_prodlist`.
+#' @param apihub _deprecated_
+#' @param service _deprecated_
#' @param max_cloud Integer number (0-100) containing the maximum cloud
#' level of the tiles to be listed (default: no filter).
#' @param availability Character argument, determining which products have
#' to be returned:
#' - `"online"` : only archive names already available for download are returned;
#' - `"lta"`: only archive names stored in the
-#' [Long Term Archive](https://scihub.copernicus.eu/userguide/LongTermArchive)
+#' Long Term Archive
#' are returned;
#' - `"check"`: all archive names are returned, checking if they are
#' available or not for download (see "Value" to know
@@ -105,10 +82,10 @@
#' \donttest{
#'
#' pos <- sf::st_sfc(sf::st_point(c(9.85,45.81)), crs = 4326)
-#' time_window <- as.Date(c("2016-05-01", "2017-07-30"))
+#' time_window <- as.Date(c("2020-08-01", "2020-08-11"))
#'
#' # Full-period list
-#' if (is_scihub_configured()) {
+#' if (is_gcloud_configured()) {
#' example_s2_list <- s2_list(
#' spatial_extent = pos,
#' tile = "32TNR",
@@ -123,7 +100,7 @@
#' safe_getMetadata(example_s2_list, "sensing_datetime")
#'
#' # Seasonal-period list
-#' if (is_scihub_configured()) {
+#' if (is_gcloud_configured()) {
#' example_s2_list <- s2_list(
#' spatial_extent = pos,
#' tile = "32TNR",
@@ -145,7 +122,7 @@ s2_list <- function(spatial_extent = NULL,
time_interval = c(Sys.Date() - 10, Sys.Date()),
time_period = "full", # temporal parameters
level = "auto",
- server = "scihub",
+ server = "gcloud",
apihub = NA,
service = "apihub",
max_cloud = 100,
@@ -202,10 +179,10 @@ s2_list <- function(spatial_extent = NULL,
time_interval <- as.Date(time_interval)
}
- if (anyNA(match(server, c("scihub", "gcloud", "s2cogs")))) {
+ if (anyNA(match(server, c("gcloud", "s2cogs")))) {
print_message(
type = "error",
- "`server` must be \"gcloud\" or \"scihub\""
+ "`server` must be \"gcloud\" (\"apihub\" is no longer available)."
)
}
diff --git a/R/s2_list_scihub.R b/R/s2_list_scihub.R
index 0a6b96ce..7dc8da05 100644
--- a/R/s2_list_scihub.R
+++ b/R/s2_list_scihub.R
@@ -1,227 +1,6 @@
.s2_list_scihub <- function(spatial_extent, time_intervals, tile, orbit, max_cloud, apihub, service, availability, .s2tiles) {
-
- n_entries <- 1
- out_list <- list()
-
- # Check connection
- if (!check_scihub_connection()) {
print_message(
type = "error",
- "Impossible to reach the SciHub server ",
- "(internet connection or SciHub may be down)."
+ "The SciHub server can no longer be accessed."
)
- }
-
- # Get credentials
- creds <- read_scihub_login(apihub)
- if (!check_scihub_login(creds[1,1], creds[1,2])) {
- print_message(
- type = "error",
- "SciHub credentials are not correct, ",
- "please check them."
- )
- }
-
- # S2 tiles
- if (missing(.s2tiles)) {.s2tiles <- s2_tiles()}
-
- # # If spatial_extent is not point, simplify polygon if needed / convert to bbox
- spatial_extent_or <- spatial_extent
- if (length(st_cast(spatial_extent, "POINT")) >= 50) {
- spatial_extent <- st_convex_hull(spatial_extent_or)
- }
- if (length(st_cast(spatial_extent, "POINT")) >= 50) {
- spatial_extent <- st_as_sfc(sf::st_bbox(spatial_extent_or))
- print_message(
- type = "warning",
- "Input extent contains too many nodes, so its bounding box was used ",
- "(a larger number of Sentinel-2 tiles could have been used); ",
- "consider simplifying the extent manually."
- )
- }
-
- # Prepare footprint
- foot <- ifelse(
- inherits(spatial_extent, "sfc_POINT"),
- paste0('footprint:%22Intersects(', paste(as.numeric(sf::st_coordinates(spatial_extent)[c(2,1)]), collapse = ",%20"),')%22'),
- paste0('footprint:%22Intersects(', sf::st_as_text(sf::st_geometry(spatial_extent)),')%22')
- )
-
- for (t_int in seq_len(nrow(time_intervals))) {
-
- rows <- 100
- start <- 0
- end_query <- FALSE
-
- while (!end_query) {
-
- query_string <- paste0(
- 'https://',ifelse(service=='dhus','scihub','apihub'),
- '.copernicus.eu/',service,'/search?',
- 'start=', start,
- '&rows=', rows,
- '&q=', foot,
- ' AND platformname:Sentinel-2',
- ' AND beginposition:[', time_intervals[t_int,1], 'T00:00:00.000Z',
- ' TO ', time_intervals[t_int,2], 'T23:59:59.000Z]',
- ' AND cloudcoverpercentage:[0 TO ', max_cloud,']'
- )
- query_string <- gsub(" ", "%20",query_string)
- query_string <- gsub("\\[", "%5b",query_string)
- query_string <- gsub("\\]", "%5d",query_string)
-
- times_429 <- 10 # if 429 "too many requests", retry up to 10 times
- while (times_429 > 0) {
- out_query <- RETRY(
- verb = "GET",
- url = query_string,
- config = authenticate(creds[1,1], creds[1,2])
- )
- times_429 <-if (out_query$status_code != 429) {0} else {times_429 - 1}
- }
-
- out_xml <- content(out_query, as = "parsed", encoding = "UTF-8")
- out_xml_list <- xmlRoot(htmlTreeParse(out_xml, useInternalNodes = TRUE))
- out_xml_list <- out_xml_list[["body"]][["feed"]]
-
-
- for (ll in which(names(out_xml_list)=="entry")) {
-
- in_entry <- strsplit(saveXML(out_xml_list[[ll]]), "\n")
-
- if (length(which(grepl("([^<]+).*$", "\\1",
- in_entry[which(grepl("", in_entry))]
- )
-
- url <- gsub(
- "^.*.*$", "\\1",
- in_entry[which(grepl("([^<]+).*$", "\\1",
- in_entry[which(grepl("\\\"relativeorbitnumber\\\"", in_entry))]
- )
- ))
-
- footprint <- tryCatch(
- st_as_sfc(
- gsub(
- "^.*([^<]+).*$", "\\1",
- in_entry[which(grepl("\\\"footprint\\\"", in_entry))]
- ),
- crs = 4326
- ),
- error = function(e) {st_polygon()}
- )
-
- clouds <- as.numeric(gsub(
- "^.*([^<]+).*$", "\\1",
- in_entry[which(grepl("\\\"cloudcoverpercentage\\\"", in_entry))]
- ))
-
- proc_level <- gsub(
- "^.*Level\\-([^<]+).*$", "\\1",
- in_entry[which(grepl("\\\"processinglevel\\\"", in_entry))]
- )
-
- mission <- gsub(
- "^.*Sentinel\\-([^<]+).*$", "\\1",
- in_entry[which(grepl("\\\"platformserialidentifier\\\"", in_entry))]
- )
-
- id_tile <- gsub("^.+_T([0-9]{2}[A-Z]{3})_.+$", "\\1", title)
-
- sensing_datetime <- as.POSIXct(
- gsub(
- "^S2[AB]\\_MSIL[12][AC]\\_([0-9]{8}T[0-9]{6})\\_N[0-9]{4}\\_R[0-9]{3}\\_T[A-Z0-9]{5}\\_[0-9]{8}T[0-9]{6}$",
- "\\1",
- title
- ),
- format = "%Y%m%dT%H%M%S", tz = "UTC"
- )
-
- creation_datetime <- as.POSIXct(
- gsub(
- "^S2[AB]\\_MSIL[12][AC]\\_[0-9]{8}T[0-9]{6}\\_N[0-9]{4}\\_R[0-9]{3}\\_T[A-Z0-9]{5}\\_([0-9]{8}T[0-9]{6})$",
- "\\1",
- title
- ),
- format = "%Y%m%dT%H%M%S", tz = "UTC"
- )
-
- ingestion_datetime <- as.POSIXct(
- gsub(
- "^.*([0-9\\-]+)T([0-9\\:\\.]+)Z.*$",
- "\\1 \\2",
- in_entry[which(grepl("name=\"ingestiondate\"", in_entry))]
- ),
- tz = "UTC"
- )
-
- uuid <- gsub(
- "^.*([^<]+).*$", "\\1",
- in_entry[which(grepl("\\\"uuid\\\"", in_entry))]
- )
-
-
- # print(paste0(title, ".SAFE"))
- out_list[[n_entries]] <- data.frame(
- name = paste0(title, ".SAFE"),
- url = url,
- mission = mission,
- level = proc_level,
- id_tile = id_tile,
- id_orbit = id_orbit,
- sensing_datetime = sensing_datetime,
- ingestion_datetime = ingestion_datetime,
- clouds = clouds,
- footprint = st_as_text(footprint),
- uuid = uuid,
- stringsAsFactors = FALSE
- )
- n_entries <- n_entries + 1
- }
- }
-
- if (sum(names(out_xml_list)=="entry") != rows) {
- end_query <- TRUE
- } else {
- start <- start + rows
- }
- }
-
- }
- out_dt <- rbindlist(out_list)
-
- if (nrow(out_dt) == 0) {return(data.table())}
-
- # remove "wrong" tiles and orbits if needed
- if (!is.null(tile)) {
- out_dt <- out_dt[id_tile %in% tile,]
- } else {
- sel_s2tiles <- suppressMessages(suppressWarnings(
- sf::st_intersection(.s2tiles, spatial_extent_or)))
- out_dt <- out_dt[id_tile %in% unique(sel_s2tiles$tile_id),]
- }
-
- if (!is.null(orbit)) {
- out_dt <- out_dt[id_orbit %in% sprintf("%03i", as.numeric(orbit)),]
- }
-
- # check online availability
- out_dt$online <- if (availability == "ignore") {
- NA
- } else {
- as.logical(safe_is_online(out_dt, verbose = FALSE, apihub = apihub))
- }
-
- out_dt
-
}
diff --git a/R/s2_order.R b/R/s2_order.R
index b41a9bf4..24eee18f 100644
--- a/R/s2_order.R
+++ b/R/s2_order.R
@@ -1,73 +1,13 @@
-#' @title Order S2 products.
-#' @description The function orders S2 products from
-#' [Long Term Archive](https://scihub.copernicus.eu/userguide/LongTermArchive).
-#' @param s2_prodlist Named character: list of the products to be ordered,
-#' in the format `safelist` (see [safelist-class]).
-#' Alternatively, it can be the path of a JSON file exported by a previous
-#' execution of [s2_order], in case the user wants, for any reason, to
-#' resubmit the order.
-#' @param export_prodlist Logical or character: if TRUE (default),
-#' the list of ordered products is saved in a JSON text file, so to be easily
-#' retrievable at a later stage with [safe_is_online] or [s2_download];
-#' if FALSE, no output files are generated.
-#' It is also possible to pass the path of an existing folder in which the
-#' JSON file will be saved (otherwise, a default path is used).
-#' @param delay Numeric: time frame (in seconds) to leave between two
-#' consecutive orders. Default is 0.5 seconds: use a higher value if you
-#' encountered errors (i.e. not all the products were correctly ordered).
-#' @param apihub Path of the `apihub.txt` file containing credentials
-#' of SciHub account.
-#' If NA (default), the default location inside the package will be used.
-#' @param service Character: it can be `"dhus"` or `"apihub"`, in which cases
-#' the required service is forced instead that the one present in the URLs
-#' passed through argument `s2_prodlist`.
-#' If NA (default), the service present in the URLs is maintained.
-#' @param reorder Logical: If TRUE, and a json file exported by s2_order
-#' is passed as argument to the function, try to order again also
-#' the `"available"` and `"ordered"` S2 datasets.
-#' Otherwise, only order the `"notordered"` ones.
-#' @return A named vector, containing the subset of `s2_prodlist` elements
-#' which were ordered.
-#' Moreover, the vector includes the following attributes:
-#' - `"available"` with the elements of `s2_prodlist` which were already
-#' available for download,
-#' - `"notordered"` with the elements of `s2_prodlist` which were not ordered
-#' for any reasons,
-#' - `"path"` (only if argument `export_prodlist` is not FALSE) with the path
-#' of the json file in which the list of the products (ordered, available
-#' and not ordered) was saved (if `export_prodlist = TRUE`).
-#'
-#' @author Luigi Ranghetti, phD (2019)
-#' @author Lorenzo Busetto, phD (2020)
-#' @references L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-#' "sen2r": An R toolbox for automatically downloading and preprocessing
-#' Sentinel-2 satellite data. _Computers & Geosciences_, 139, 104473.
-#' \doi{10.1016/j.cageo.2020.104473}, URL: \url{https://sen2r.ranghetti.info/}.
-#' @note License: GPL 3.0
-#' @importFrom httr RETRY authenticate
-#' @importFrom foreach foreach "%do%"
-#' @importFrom jsonlite toJSON
+#' @title Order S2 products (deprecated).
+#' @description This function is deprecated and will be removed.
+#' @param s2_prodlist deprecated
+#' @param export_prodlist deprecated
+#' @param delay deprecated
+#' @param apihub deprecated
+#' @param service deprecated
+#' @param reorder deprecated
+#' @return deprecated
#' @export
-#'
-#' @examples
-#' \dontrun{
-#'
-#' # Generate the lists of products
-#' pos <- sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs = 4326)
-#' time_window <- as.Date(c("2019-10-21", "2019-11-20"))
-#' list_safe <- s2_list(spatial_extent = pos, time_interval = time_window)
-#' print(list_safe)
-#' # (at the time the documentation was updated, this list was containing 6
-#' # archives already available online and 3 stored in the Long Term Archive)
-#'
-#' # Order the products
-#' ordered_prods <- s2_order(list_safe)
-#'
-#' # Check in a second time if the product was made available
-#' (order_path <- attr(ordered_prods, "path"))
-#' safe_is_online(order_path)
-#'
-#' }
s2_order <- function(
s2_prodlist = NULL,
@@ -77,15 +17,7 @@ s2_order <- function(
service = NA,
reorder = TRUE
) {
- .s2_order(
- s2_prodlist = s2_prodlist,
- export_prodlist = export_prodlist,
- delay = delay,
- apihub = apihub,
- service = service,
- reorder = reorder,
- .s2_availability = NULL
- )
+ return(invisible(NULL))
}
# internal function, used internally in order not to repeat the check
@@ -100,286 +32,5 @@ s2_order <- function(
.s2_availability = NULL,
.log_path = TRUE # TRUE to log all, FALSE to skip the path of the json
) {
-
- # to avoid NOTE on check
- i <- NULL
-
- # convert input NA arguments in NULL
- for (a in c("s2_prodlist", "export_prodlist", "apihub")) {
- if (suppressWarnings(all(is.na(get(a))))) {
- assign(a,NULL)
- }
- }
-
- # exit if empty
- if (length(nn(s2_prodlist)) == 0) {
- return(invisible(NULL))
- }
-
- # check export_prodlist
- if (all(is.character(export_prodlist), length(export_prodlist) > 0)) {
- if (!dir.exists(export_prodlist)) {
- print_message(
- type = "error",
- "Argument 'export_prodlist' must be TRUE, FALSE or the path of an existing folder."
- )
- }
- }
-
- # check delay to be numeric
- if (any(length(delay) == 0, !is.numeric(delay))) {
- print_message(
- type = "error",
- "Argument 'delay' must be numeric"
- )
- }
-
- # check input format
- s2_prodlist <- as(s2_prodlist, "safelist")
- # TODO add input checks
-
- # check the used service
- if (!service %in% c("apihub", "dhus", NA)) {
- print_message(
- type = "error",
- "Argument 'service' can be only \"apihub\" or \"dhus\"; ",
- "leaving the input URLs as are."
- )
- } else if (!is.na(service)) {
- s2_prodlist <- gsub(
- "^https://((scihub)|(apihub)).copernicus.eu/((apihub)|(dhus))/odata",
- paste0("https://",ifelse(service=="dhus","scihub","apihub"),
- ".copernicus.eu/",service,"/odata"),
- s2_prodlist
- )
- }
-
- # read credentials
- s2_scihub <- s2_prodlist[grepl("^http.+Products\\(.+\\)/\\$value$", s2_prodlist)]
- if (length(s2_scihub) > 0) {
- creds <- read_scihub_login(apihub)
- }
-
- # Split products to be downloaded from products to be ordered
-
- # Exclude products not from SciHub
- s2_scihub <- s2_prodlist[grepl("^http.+Products\\(.+\\)/\\$value$", s2_prodlist)]
- if (length(s2_scihub) < length(s2_prodlist)) {
- print_message(
- type = "message",
- date = TRUE,
- length(s2_prodlist) - length(s2_scihub),
- " products are not from SciHub and will not considered."
- )
- s2_prodlist <- s2_scihub
- }
-
- # Check availability (order is never attempted for already online products)
- s2_availability <- if (is.null(.s2_availability)) {
- print_message(
- type = "message",
- date = TRUE,
- "Check if products are already available for download..."
- )
- safe_is_online(s2_prodlist, verbose = FALSE, apihub = apihub)
- } else {
- .s2_availability
- }
-
- # If some products are already available, print a message
- if (sum(s2_availability, na.rm = TRUE) > 0) {
- print_message(
- type = "message",
- date = TRUE,
- sum(s2_availability, na.rm = TRUE)," Sentinel-2 images are already online."
- )
- }
-
- ## Order products stored in Long Term Archive
- if (sum(!nn(s2_availability), na.rm = TRUE) > 0) {
- print_message(
- type = "message",
- date = TRUE,
- "Ordering ",sum(!nn(s2_availability), na.rm = TRUE)," Sentinel-2 images ",
- "stored in the Long Term Archive..."
- )
- }
- if (!is.null(attr(s2_prodlist, "order_status")) & reorder == FALSE) {
- # if the list includes the "order_status" attribute (meaning it was
- # derived from a json file saved by s2_order), and reorder is FALSE,
- # create a "old_order" variable contatining indices of already succesfull orders
- # that in the meantime did not change to "available",
- # and an "to_order" variable containing indices of datasets yet to be ordered
- # (notordered and not available - could happen that another user requested the
- # dataset in the meantime)
- old_order <- which(!s2_availability & attr(s2_prodlist, "order_status") == "ordered")
- to_order <- which(!s2_availability & attr(s2_prodlist, "order_status") != "ordered")
-
- } else {
- to_order <- which(!nn(s2_availability))
- old_order <- NULL
- }
-
- false_invalid_safe <- FALSE # initialise variables
- quota_exceeded <- rep(TRUE, length(to_order))
- status_codes <- c()
- i_cred <- 1 # creds record (in case of multiple SciHub credentials)
-
- # cycle along "to_order" to create a TRUE/FALSE array of succesfull orders
- ordered_products <- foreach(i = seq_along(to_order), .combine = c) %do% {
- # delay after previous order
- if (i != 1) {
- Sys.sleep(delay)
- }
-
- # order products
- # in case of user quota exceeded, repeat the order with different credentials
- while (all(quota_exceeded[i], i_cred <= nrow(creds))) {
-
- times_429 <- 10 # if 429 "too many requests", retry up to 10 times
- while (times_429 > 0) {
- make_order <- RETRY(
- verb = "GET",
- url = as.character(s2_prodlist[i]),
- config = authenticate(creds[i_cred,1], creds[i_cred,2])
- )
- times_429 <-if (make_order$status_code != 429) {0} else {times_429 - 1}
- }
-
- # check if the order was successful
- sel_ordered <- if (inherits(make_order, "response")) {
- # save status code
- status_codes[i] <- make_order$status_code
- # check that user quota did not exceed
- quota_exceeded[i] <- any(grepl(
- "retrieval quota exceeded",
- make_order$headers$`cause-message`
- ))
- if (quota_exceeded[i]) {
- i_cred <- i_cred + 1
- if (i_cred <= nrow(creds)) {
- print_message(
- type = "message",
- "Switching to SciHub record ",i_cred," at product ",i,".",
- date = TRUE
- )
- }
- }
-
- # check that an invalid SAFE was not downloaded (#381)
- if (make_order$status_code == 200) {
- false_invalid_safe <- TRUE
- make_order$content <- NULL; gc()
- }
- make_order$status_code == 202
- } else FALSE
-
- } # end of user quota WHILE cycle
-
- sel_ordered
-
- }
-
- # create a temporary array as long as s2_prodlist and "fill" it with logical
- # depending on status: TRUE, if successfull order, FALSE for not
- # succesfull order OR online dataset.
- # In case we are working on resubmitting an order, and reorder is FALSE, recreate
- # the "ordered_products" array, appending results of current run with "old_order"
-
- tempordered <- rep(FALSE, length(s2_prodlist))
- if (!is.null(old_order)) {
- tempordered[sort(unique(c(to_order[ordered_products], old_order)))] <- TRUE
- } else {
- tempordered[to_order[ordered_products]] <- TRUE
- }
- ordered_products <- tempordered
- notordered_products <- !ordered_products & !nn(s2_availability)
-
- out_list <- s2_prodlist[ordered_products]
- attr(out_list, "available") <- s2_prodlist[s2_availability]
- attr(out_list, "notordered") <- s2_prodlist[notordered_products]
-
- # remove order_status if present
- attr(out_list, "order_status") <- NULL
-
- # export list_towrite, unless export_prodlist == FALSE
- list_towrite <- list(
- ordered = as.list(out_list),
- available = as.list(attr(out_list, "available")),
- notordered = as.list(attr(out_list, "notordered"))
- )
- if (any(export_prodlist != FALSE) & length(list_towrite) > 0) {
- order_time <- Sys.time()
-
- prodlist_dir <- if (is.logical(export_prodlist)) {
- file.path(dirname(attr(load_binpaths(), "path")), "lta_orders")
- } else {
- export_prodlist
- }
- dir.create(prodlist_dir, showWarnings = FALSE)
- prodlist_path <- file.path(
- prodlist_dir,
- strftime(order_time, format = "lta_%Y%m%d_%H%M%S.json")
- )
- writeLines(
- toJSON(as.list(list_towrite), pretty = TRUE),
- prodlist_path
- )
- attr(out_list, "path") <- prodlist_path
- }
-
- # Issue processing messages / warnings
- if (sum(ordered_products) > 0) {
- print_message(
- type = "message",
- date = TRUE,
- sum(ordered_products)," of ",sum(!nn(s2_availability), na.rm = TRUE)," Sentinel-2 images ",
- "were correctly ordered. ",
- if (.log_path == TRUE) {paste0(
- "You can check at a later time if the ordered products are available online ",
- "using the command:\n",
- if (is.null(attr(out_list, "path"))) {paste0(
- '\u00A0\u00A0safe_is_online(c(\n "',paste(out_list, collapse = '",\n "'),'"\n))'
- )} else {paste0(
- '\u00A0\u00A0safe_is_online("',attr(out_list, "path"),'")'
- )},
- "\n"
- )}
- )
- }
-
- if (sum(notordered_products) > 0) {
-
- print_message(
- type = "message",
- date = TRUE,
- sum(notordered_products)," of ",sum(!nn(s2_availability), na.rm = TRUE)," Sentinel-2 images ",
- "were not correctly ordered ",
- "(HTML status code: ",unique(paste(status_codes[status_codes!=202]), collapse = ", "),")",
- if (any(quota_exceeded)) {paste0(
- " because user '",creds[1,1],"' offline products retrieval quota exceeded. ",
- "Please retry later, otherwise use different SciHub credentials ",
- "(see ?write_scihub_login or set a specific value for argument \"apihub\")."
- )} else if (false_invalid_safe) {paste0(
- " because some invalid SAFE products were stored on the ESA API Hub. ",
- "Please retry ordering them on DHUS ",
- "(set argument 'service = \"dhus\"' in function s2_order())."
- )} else {
- "."#," Try using a higher value for the argument \"delay\"."
- },
- if (.log_path == TRUE) {paste0(
- " You can try ordering them at a later time ",
- "using the command:\n",
- if (is.null(attr(out_list, "path"))) {paste0(
- '\u00A0\u00A0s2_order(c(\n "',paste(out_list, collapse = '",\n "'),'"\n))'
- )} else {paste0(
- '\u00A0\u00A0s2_order("',attr(out_list, "path"),'")'
- )},
- "\n"
- )}
- )
- }
-
- return(out_list)
-
+ return(invisible(NULL))
}
diff --git a/R/safe_is_online.R b/R/safe_is_online.R
index b01ed6bf..d9cf6ce6 100644
--- a/R/safe_is_online.R
+++ b/R/safe_is_online.R
@@ -1,47 +1,10 @@
-#' @title Check if SAFE is available for download
-#' @description The function checks if the required SAFE archives are
-#' available for download, or if they have to be ordered from the Long Term
-#' Archive.
-#' @param s2_prodlist Named character: list of the products to be checked,
-#' in the format `safelist` (see [safelist-class]).
-#' Alternatively, it can be the path of a JSON file exported by [s2_order].
-#' @param apihub Path of the "apihub.txt" file containing credentials
-#' of SciHub account.
-#' If NA (default), the default location inside the package will be used.
-#' @param verbose Logical: if TRUE, provide processing messages summarising
-#' how many of the SAFE archives in `s2_prodlist` are available online.
-#' @return A logical vector of the same length and names of the SAFE products
-#' passed with `s2_prodlist`,
-#' in which each element is TRUE if the corresponding SAFE archive is
-#' available for download, FALSE if it is not or NA in case of errors with
-#' the SAFE url.
-#' @author Luigi Ranghetti, phD (2019)
-#' @author Lorenzo Busetto, phD (2020)
-#' @references L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-#' "sen2r": An R toolbox for automatically downloading and preprocessing
-#' Sentinel-2 satellite data. _Computers & Geosciences_, 139, 104473.
-#' \doi{10.1016/j.cageo.2020.104473}, URL: \url{https://sen2r.ranghetti.info/}.
-#' @note License: GPL 3.0
-#' @importFrom httr RETRY authenticate content
-#' @importFrom jsonlite fromJSON
+#' @title Check if SAFE is available for download (deprecated)
+#' @description This function is deprecated and will be removed.
+#' @param s2_prodlist deprecated
+#' @param apihub deprecated
+#' @param verbose deprecated
+#' @return deprecated
#' @export
-#'
-#' @examples
-#' \donttest{
-#' if (is_scihub_configured()) {
-#'
-#' # Generate the lists of products
-#' pos <- sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs = 4326)
-#' time_window <- as.Date(c("2018-02-21", "2018-03-20"))
-#' list_safe <- s2_list(spatial_extent = pos, time_interval = time_window)
-#' # (at the time the documentation was written, this list was containing 5
-#' # archives already available online and 2 stored in the Long Term Archive)
-#'
-#' # Check for availability
-#' safe_is_online(list_safe)
-#'
-#' }
-#' }
safe_is_online <- function(s2_prodlist = NULL, apihub = NA, verbose = TRUE) {
@@ -68,51 +31,13 @@ safe_is_online <- function(s2_prodlist = NULL, apihub = NA, verbose = TRUE) {
s2_prodlist
)
- # read credentials
- s2_scihub <- s2_prodlist[grepl("^http.+Products\\(.+\\)/\\$value$", s2_prodlist)]
- if (length(s2_scihub) > 0) {
- creds <- read_scihub_login(apihub)
- }
-
# check for availability
s2_availability <- sapply(s2_prodlist, function(p) {
- if (grepl("^http.+Products\\(.+\\)/\\$value$", p)) {
- tryCatch(
- as.logical(content(RETRY(
- verb = "GET",
- url = gsub("\\$value$", "Online/$value", p),
- config = httr::authenticate(creds[1,1], creds[1,2])
- ), as = "parsed", encoding = "UTF-8")),
- error = function(e) {
- print_message(
- type = "error",
- "Impossible to reach the SciHub server ",
- "(internet connection or SciHub may be down)."
- )
- NA
- }
- )
- } else if (grepl("^gs://gcp-public-data-sentinel-2", p)) {TRUE} else {NA}
+ TRUE
})
names(s2_availability) <- names(s2_prodlist)
- if (verbose == TRUE) {
- if (all(s2_availability)) {
- print_message(
- type = "message",
- date = FALSE,
- "All ", length(s2_availability), " products are online."
- )
- } else {
- print_message(
- type = "message",
- date = FALSE,
- length(which(s2_availability)), " out of ",
- length(s2_availability), " products are online."
- )
- }
- }
s2_availability
}
diff --git a/R/scihub_login.R b/R/scihub_login.R
index f2ae8aee..7998bd5e 100644
--- a/R/scihub_login.R
+++ b/R/scihub_login.R
@@ -1,208 +1,53 @@
-#' @title Import / export / check SciHub username and password
-#' @description
-#' - `read_scihub_login()` reads the SciHub login information;
-#' - `write_scihub_login()` saves new username and password;
-#' - `check_scihub_login()` checks their validity;
-#' - `is_scihub_configured()` check if SciHub credentials are configured.
-#'
-#' Login information is stored in a file `apihub.txt` inside the
-#' ".sen2r" subfolder of the home directory. These functions allow reading
-#' or writing this file, and editing it from the GUI.
-#' In case file `apihub.txt` is missing, `read_scihub_login()` searches inside
-#' the environmental variables `SCIHUB_USER` and `SCIHUB_PASSWORD`.
-#' @param apihub_path Path of the file in which login information is saved.
-#' If NA (default) it is automatically read from the package default location.
-#' @param username SciHub username.
-#' @param password SciHub password.
-#' @param service Character: it can be `"dhus"` or `"apihub"` (default).
-#' @return `read_scihub_login()` returns a matrix of credentials,
-#' in which `username` is in the first column, `password` in the second.
-#' @details Notice that new/recently updated SciHub credentials are recognised by API Hub
-#' with a delay of about one week (see \url{https://scihub.copernicus.eu/twiki/do/view/SciHubWebPortal/APIHubDescription} for details).
-#'
-#' For this reason, newly created credentials can not immediately be used by `sen2r`,
-#' and password edits on old credentials are not immediately recognised.
-#' @author Luigi Ranghetti, phD (2019)
-#' @references L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-#' "sen2r": An R toolbox for automatically downloading and preprocessing
-#' Sentinel-2 satellite data. _Computers & Geosciences_, 139, 104473.
-#' \doi{10.1016/j.cageo.2020.104473}, URL: \url{https://sen2r.ranghetti.info/}.
-#' @note License: GPL 3.0
-#'
-#' @examples
-#' \donttest{
-#' is_scihub_configured()
-#' }
-#' \dontrun{
-#' check_scihub_connection()
-#' check_scihub_login("username", "password")
-#' write_scihub_login("username", "password")
-#' read_scihub_login()
-#' }
+#' @title Import / export / check SciHub username and password (deprecated)
+#' @description These functions are deprecated and will be removed.
+#' @return deprecated
#' @name read_scihub_login
#' @rdname scihub_login
+#' @param apihub_path deprecated
+#' @param username deprecated
+#' @param password deprecated
+#' @param service deprecated
#' @export
-
read_scihub_login <- function(apihub_path = NA) {
-
- # if apihub_path is not specified,
- # retrieve from the current installation
- if (any(c(is.na(apihub_path), length(nn(apihub_path))==0))) {
- apihub_path <- file.path(dirname(attr(load_binpaths(), "path")), "apihub.txt")
- attr(apihub_path, "default") <- TRUE
- } else {
- attr(apihub_path, "default") <- FALSE
- }
-
- # return user and password
- if (file.exists(apihub_path)) {
- t(sapply(strsplit(readLines(apihub_path), " "), c))
- } else if (all(Sys.getenv("SCIHUB_USER") != "", Sys.getenv("SCIHUB_PASSWORD") != "")) {
- matrix(c(Sys.getenv("SCIHUB_USER"), Sys.getenv("SCIHUB_PASSWORD")), ncol = 2)
- } else {
- # if apihub does not exists, return an error
- print_message(
- type="error",
- "File apihub.txt with the SciHub credentials is missing. ",
- if (attr(apihub_path, "default")) {
- "Launch function write_scihub_login('', '') to create it."
- }
- )
- }
-
+ matrix(c("",""), ncol = 2)
}
#' @name is_scihub_configured
#' @rdname scihub_login
-#' @return `is_scihub_configured()` returns TRUE if credentials can be
-#' accessed, FALSE if not.
#' @export
-
is_scihub_configured <- function() {
- !inherits(try(read_scihub_login(), silent = TRUE), "try-error")
+ FALSE
}
-
#' @name check_scihub_login
-#' @return `check_scihub_login()` returns TRUE if credentials are valid,
-#' FALSE elsewhere.
-#' @importFrom httr RETRY authenticate handle
-#' @author Lorenzo Busetto, phD (2019)
#' @rdname scihub_login
#' @export
-
check_scihub_login <- function(username, password, service = "apihub") {
- if (!check_scihub_connection(service = service)) {
- print_message(
- type = "error",
- "Impossible to reach the SciHub server ",
- "(internet connection or SciHub may be down)."
- )
- }
- check_creds <- RETRY(
- verb = "GET",
- url = paste0("https://",ifelse(service=="dhus","scihub","apihub"),
- ".copernicus.eu/",service,"/odata/v1"),
- handle = handle(""),
- config = authenticate(username, password)
- )
- if (check_creds$status == "401") {
- FALSE
- } else {
- TRUE
- }
+ FALSE
}
#' @name check_scihub_connection
-#' @return `check_scihub_connection()` returns TRUE if internet connection
-#' is available and SciHub is accessible, FALSE otherwise.
-#' @importFrom httr RETRY handle
#' @rdname scihub_login
#' @export
check_scihub_connection <- function(service = "apihub") {
- check_online <- try(
- RETRY(
- "GET",
- url = paste0("https://",ifelse(service=="dhus","scihub","apihub"),
- ".copernicus.eu/",service,"/"),
- handle = handle("")
- )
- )
- !inherits(check_online, "try-error")
+ FALSE
}
-
#' @name write_scihub_login
-#' @param check Logical: if TRUE (default), new credentials are checked
-#' before writing them on `apihub_path` (if they are invalid, an error
-#' is provided);
-#' if FALSE, they are directly written.
-#' @param append Logical: if TRUE, new credentials are added
-#' to the ones existing within `apihub_path`;
-#' if FALSE (default), `apihub_path` is replaced with the new ones.
-#' @return `write_scihub_login()` returns NULL.
#' @rdname scihub_login
+#' @param check deprecated
+#' @param append deprecated
#' @export
write_scihub_login <- function(username, password,
apihub_path = NA,
check = TRUE,
append = FALSE) {
-
- # check credentials (if required)
- if (check == TRUE) {
- if (!check_scihub_login(username, password, service = "apihub")) {
- if (!check_scihub_login(username, password, service = "dhus")) {
- print_message(
- type = "error",
- "The provided credentials are not valid, ",
- "so they will not be saved. "
- )
- } else {
- print_message(
- type = "error",
- "The provided credentials are not yet recognised by API Hub, ",
- "although being valid on ESA SciHub; ",
- "probably they were created recently. ",
- "Please notice that new/recently updated SciHub credentials are recognised by API Hub ",
- "with a delay of about one week. ",
- "For this reason, newly created SciHub credentials can not immediately be used by sen2r",
- "and password edits on old credentials are not immediately recognised."
- )
- }
- }
- }
-
- # if apihub_path is not specified,
- # retrieve from the current installation
- if (any(c(is.na(apihub_path), length(nn(apihub_path))==0))) {
- apihub_path <- file.path(dirname(attr(load_binpaths(), "path")), "apihub.txt")
- dir.create(dirname(apihub_path), showWarnings = FALSE)
- }
-
- # if append is required, read the old file
- # in order to exclude duplicated entries and
- # add the new credentials to the top instead that to the bottom
- apihub <- matrix(c(username, password), nrow = 1)
- if (append) {
- apihub <- rbind(apihub, read_scihub_login(apihub_path))
- apihub <- apihub[!duplicated(apihub[,1]),]
- apihub <- matrix(apihub, ncol = 2)
- }
-
- # write credentials
- writeLines(apply(apihub, 1, paste, collapse=" "), apihub_path)
-
+ invisible(NULL)
}
-
-
-
-# #' @name scihub_modal
-# #' @rdname scihub_login
-
# write dialog content
.scihub_modal <- function() { # nocov start
diff --git a/R/sen2r.R b/R/sen2r.R
index cd716696..cd58c52f 100644
--- a/R/sen2r.R
+++ b/R/sen2r.R
@@ -35,22 +35,18 @@
#' @param online (optional) Logical: TRUE (default) to search for available
#' products on SciHub and/or Google Cloud (and download if needed);
#' FALSE to work only with already downloaded SAFE products.
-#' @param server (optional) Character vector of length 1 or 2, with the names of
+#' @param server (deprecate) Character vector of length 1, with the names of
#' the servers on which SAFE archives are searched.
-#' Available options are `"scihub"` (ESA Sentinel Hub) and `"gcloud"`
-#' (Google Cloud).
-#' Default is `"scihub"`, meaning that only ESA Sentinel Hub is considered.
-#' In case of multiple values, they are used in order of priority and
-#' products on LTA are always left as last choice.
+#' Currently, only `"gcloud"` (Google Cloud) is supported.
+#' Old `"scihub"` (ESA Sentinel Hub) can no more be used, since November 2023,
+#' when the Copernicus Sentinel Data is no longer available and has been
+#' replaced by the Copernicus Data Space Ecosystem.
#' See also the section "Details" of `s2_list()`.
#' @param order_lta (optional) Logical: TRUE (default) to order products from
#' the Long Term Archive if unavailable for direct download; FALSE to simply
#' skip them (this option has effect only in online mode).
#' It takes effect only if argument `server` includes `"scihub"`.
-#' @param apihub Path of the text file containing credentials
-#' of SciHub account.
-#' If NA (default), the default location inside the package will be used.
-#' It takes effect only if argument `server` includes `"scihub"`.
+#' @param apihub _deprecated_
#' @param downloader (optional) Character value corresponding to the executable
#' which should be used to download SAFE products. It could be one among
#' `"builtin"` (default) and `"aria2"`.
@@ -435,7 +431,7 @@ sen2r <- function(param_list = NULL,
s2_levels = "l2a",
sel_sensor = c("s2a","s2b"),
online = TRUE,
- server = "scihub",
+ server = "gcloud",
order_lta = TRUE,
apihub = NA,
downloader = "builtin",
diff --git a/R/zzz.R b/R/zzz.R
index 1bee0c80..7cd16a97 100644
--- a/R/zzz.R
+++ b/R/zzz.R
@@ -8,6 +8,12 @@
"Welcome to sen2r. To use the package from a GUI, launch",
" > sen2r()",
"Documentation: https://sen2r.ranghetti.info",
+ "\nIMPORTANT: since November 2023, SAFE archives can no longer",
+ "be downloaded from the ESA Hub, so Google Cloud is the new default server.",
+ "To be able to search and download inputs: ",
+ "- install Google Cloud SDK (https://cloud.google.com/sdk/docs/install);",
+ "- configure sen2r to use Google Cloud SDK launching the function:",
+ " > check_gcloud()",
# if (all(
# Sys.info()["sysname"] %in% c("Windows", "Darwin"),
# length(load_binpaths()) == 0
diff --git a/README.Rmd b/README.Rmd
index 4a07eeb2..c95891f2 100644
--- a/README.Rmd
+++ b/README.Rmd
@@ -52,12 +52,18 @@ allows to:
* mask cloudy pixels (using the Surface Classification Map as masking layer);
* computing spectral indices and RGB images.
-**New feature**: starting from version 1.5.0,
-Sentinel-2 SAFE archives can be downloaded both from official
-[Copernicus Open Access Hub](https://scihub.copernicus.eu/) or from
-[Google Cloud Sentinel-2 bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2)
-(this option can be useful to bypass limitations of ESA Long Term Archive
-retention time; refer to [this post](https://luigi.ranghetti.info/post/safe-gcloud/) for details).
+**Important**: since November 2023,
+Sentinel-2 SAFE archives can be searched and retrieved only from [Google Cloud Sentinel-2 bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2);
+to use it, **Google Cloud SDK must be installed** and configured following the
+[official instructions](https://cloud.google.com/sdk/docs/install).
+Refer to [this post](https://luigi.ranghetti.info/post/safe-gcloud/) for details
+about how to install configure the required external dependency Google Cloud SDK.
+Notice that this mode is not optimal (a modern STAC API approach should be used),
+and searching products could be a very slow process.
+Nevertheless, the replacement of the old official data source (ESA Hub) with the
+new [Copernicus Data Space](https://dataspace.copernicus.eu/analyse/apis) is not
+planned (this package is no longer actively maintained due to a career change of the developer).
+Anyone who wants to help [can contribute](https://github.com/ranghetti/sen2r/issues/464)).
Setting the execution of this processing chain is particularly easy using the
sen2r
diff --git a/README.md b/README.md
index 50db140e..4cbce452 100644
--- a/README.md
+++ b/README.md
@@ -53,13 +53,18 @@ allows to:
layer);
- computing spectral indices and RGB images.
-**New feature**: starting from version 1.5.0, Sentinel-2 SAFE archives
-can be downloaded both from official [Copernicus Open Access
-Hub](https://scihub.copernicus.eu/) or from [Google Cloud Sentinel-2
-bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2)
-(this option can be useful to bypass limitations of ESA Long Term
-Archive retention time; refer to [this
-post](https://luigi.ranghetti.info/post/safe-gcloud/) for details).
+**Important**: since November 2023,
+Sentinel-2 SAFE archives can be searched and retrieved only from [Google Cloud Sentinel-2 bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2);
+to use it, **Google Cloud SDK must be installed** and configured following the
+[official instructions](https://cloud.google.com/sdk/docs/install).
+Refer to [this post](https://luigi.ranghetti.info/post/safe-gcloud/) for details
+about how to install configure the required external dependency Google Cloud SDK.
+Notice that this mode is not optimal (a modern STAC API approach should be used),
+and searching products could be a very slow process.
+Nevertheless, the replacement of the old official data source (ESA Hub) with the
+new [Copernicus Data Space](https://dataspace.copernicus.eu/analyse/apis) is not
+planned (this package is no longer actively maintained due to a career change of the developer).
+Anyone who wants to help [can contribute](https://github.com/ranghetti/sen2r/issues/464)).
Setting the execution of this processing chain is particularly easy
using the ${out} 2>&1
-
- ERROR
-Running the tests in ‘tests/testthat.R’ failed.
-```
-This submission should have managed this issue (I cannot be sure about that
-because I was not able to reproduce this issue on any MacOS machines, see e.g.
-`https://mac.r-project.org/macbuilder/results/1686841396-983eb296818f2468/`
-and previous
-`https://mac.r-project.org/macbuilder/results/1686565664-5f7f28746adca176/`,
-which are not showing any errors.)
diff --git a/docs/404.html b/docs/404.html
index 854d942d..21263216 100644
--- a/docs/404.html
+++ b/docs/404.html
@@ -32,7 +32,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/CODE-OF-CONDUCT.html b/docs/CODE-OF-CONDUCT.html
index 2b5c8fef..aed84c05 100644
--- a/docs/CODE-OF-CONDUCT.html
+++ b/docs/CODE-OF-CONDUCT.html
@@ -17,7 +17,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/LICENSE.html b/docs/LICENSE.html
index 3200a5c0..2d44a0b9 100644
--- a/docs/LICENSE.html
+++ b/docs/LICENSE.html
@@ -17,7 +17,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/articles/docker.html b/docs/articles/docker.html
index 29464e06..71df4093 100644
--- a/docs/articles/docker.html
+++ b/docs/articles/docker.html
@@ -33,7 +33,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/articles/index.html b/docs/articles/index.html
index 08b89cad..c74e0360 100644
--- a/docs/articles/index.html
+++ b/docs/articles/index.html
@@ -17,7 +17,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/articles/installation.html b/docs/articles/installation.html
index 5d31aa69..488deeab 100644
--- a/docs/articles/installation.html
+++ b/docs/articles/installation.html
@@ -33,7 +33,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/articles/outstructure.html b/docs/articles/outstructure.html
index a97f7c91..4a8932e9 100644
--- a/docs/articles/outstructure.html
+++ b/docs/articles/outstructure.html
@@ -33,7 +33,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/articles/sen2r_cmd.html b/docs/articles/sen2r_cmd.html
index 75f93501..ca26e3b3 100644
--- a/docs/articles/sen2r_cmd.html
+++ b/docs/articles/sen2r_cmd.html
@@ -33,7 +33,7 @@
sen2r
- 1.5.6
+ 1.6.0
@@ -286,7 +286,7 @@
Load proce
The processing chain can be re-launched with the command:
sen2r("/home/lranghetti/.sen2r/proc_par/s2proc_20201127_172523.json")
mask cloudy pixels (using the Surface Classification Map as masking layer);
computing spectral indices and RGB images.
-
New feature: starting from version 1.5.0, Sentinel-2 SAFE archives can be downloaded both from official Copernicus Open Access Hub or from Google Cloud Sentinel-2 bucket (this option can be useful to bypass limitations of ESA Long Term Archive retention time; refer to this post for details).
+
Important: since November 2023, Sentinel-2 SAFE archives can be searched and retrieved only from Google Cloud Sentinel-2 bucket; to use it, Google Cloud SDK must be installed and configured following the official instructions. Refer to this post for details about how to install configure the required external dependency Google Cloud SDK. Notice that this mode is not optimal (a modern STAC API approach should be used), and searching products could be a very slow process. Nevertheless, the replacement of the old official data source (ESA Hub) with the new Copernicus Data Space is not planned (this package is no longer actively maintained due to a career change of the developer). Anyone who wants to help can contribute).
Setting the execution of this processing chain is particularly easy using the sen2r GUI, which allows to set the parameters, to directly launch the main function or to save them in a JSON file which can be used to launch the processing at a later stage.
The possibility to launch the processing with a set of parameters saved in a JSON file (or directly passed as function arguments) makes easy to build scripts to automatically update an archive of Sentinel-2 products. Specific processing operations (i.e. applying Sen2Cor on Level-1c SAFE products, merging adjacent tiles, computing spectral indices from existing products) can also be performed using intermediate functions (see usage).
link_sen2cor() can be used to link an existing Sen2Cor installation to sen2r;
-as() methods can be used to convert safelist from/to character, data.frame and data.table (as well as as.character()',as.data.frame()andas.data.table()` functions).
+as() methods can be used to convert safelist from/to character, data.frame and data.table (as well as as.character()',as.data.frame()andas.data.table()` functions).
# the reference path(ref_path<-system.file(package="sf"))
-#> [1] "C:/Users/luigi.ranghetti/AppData/Local/Programs/R/R-4.3.1/library/sf"
+#> [1] "/home/lranghetti/R/x86_64-pc-linux-gnu-library/4.3/sf"# a path with a common parent with ref_path(in_path_1<-system.file(package="stars"))
-#> [1] "C:/Users/luigi.ranghetti/AppData/Local/Programs/R/R-4.3.1/library/stars"
+#> [1] "/home/lranghetti/R/x86_64-pc-linux-gnu-library/4.3/stars"# a path included in ref_path(in_path_2<-system.file("DESCRIPTION", package="sf"))
-#> [1] "C:/Users/luigi.ranghetti/AppData/Local/Programs/R/R-4.3.1/library/sf/DESCRIPTION"
+#> [1] "/home/lranghetti/R/x86_64-pc-linux-gnu-library/4.3/sf/DESCRIPTION"# a path external to ref_path (in Linux)(in_path_3<-system.file(package="base"))
-#> [1] "C:/Users/luigi.ranghetti/AppData/Local/Programs/R/R-4.3.1/library/base"
+#> [1] "/usr/lib/R/library/base"# an unexisting path(in_path_4<-gsub("sf$","unexistingpackage",ref_path))
-#> [1] "C:/Users/luigi.ranghetti/AppData/Local/Programs/R/R-4.3.1/library/unexistingpackage"
+#> [1] "/home/lranghetti/R/x86_64-pc-linux-gnu-library/4.3/unexistingpackage"abs2rel(in_path_1, ref_path)#> [1] "../stars"
@@ -175,7 +175,7 @@
#> Loading required package: sp#> Loading required package: abind#> Loading required package: sf
-#> Linking to GEOS 3.11.2, GDAL 3.6.2, PROJ 9.2.0; sf_use_s2() is TRUE
-#> [2023-10-19 16:28:01] Computing index EVI on date 2019-07-23...
+#> Linking to GEOS 3.10.2, GDAL 3.4.3, PROJ 8.2.1; sf_use_s2() is TRUE
+#> [2023-11-09 22:26:19] Computing index EVI on date 2019-07-23...
+#> Please note that rgdal will be retired during October 2023,
+#> plan transition to sf/stars/terra functions using GDAL and PROJ
+#> at your earliest convenience.
+#> See https://r-spatial.org/r/2023/05/15/evolution4.html and https://github.com/r-spatial/evolution
+#> rgdal: version: 1.6-7, (SVN revision 1203)
+#> Geospatial Data Abstraction Library extensions to R successfully loaded
+#> Loaded GDAL runtime: GDAL 3.4.3, released 2022/04/22
+#> Path to GDAL shared files: /usr/share/gdal
+#> GDAL binary built with GEOS: TRUE
+#> Loaded PROJ runtime: Rel. 8.2.1, January 1st, 2022, [PJ_VERSION: 821]
+#> Path to PROJ shared files: /home/lranghetti/.local/share/proj:/usr/share/proj
+#> PROJ CDN enabled: FALSE
+#> Linking to sp version:1.6-1
+#> To mute warnings of possible GDAL/OSR exportToProj4() degradation,
+#> use options("rgdal_show_exportToProj4_warnings"="none") before loading sp or rgdal.ex_out
-#> [1] "C:\\Users\\luigi.ranghetti\\AppData\\Local\\Temp\\RtmpkXRmAu/S2A2A_20190723_022_Barbellino_EVI_10.tif"
+#> [1] "/tmp/RtmpFvTjNK/S2A2A_20190723_022_Barbellino_EVI_10.tif"# Show outputoldpar<-par(mfrow =c(1,2), mar =rep(0,4))
diff --git a/docs/reference/s2_defNA.html b/docs/reference/s2_defNA.html
index 0e5c40e9..d5d15515 100644
--- a/docs/reference/s2_defNA.html
+++ b/docs/reference/s2_defNA.html
@@ -19,7 +19,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/reference/s2_dop.html b/docs/reference/s2_dop.html
index 3cc2b0b0..b84c64d0 100644
--- a/docs/reference/s2_dop.html
+++ b/docs/reference/s2_dop.html
@@ -26,7 +26,7 @@
sen2r
- 1.5.6
+ 1.6.0
@@ -173,46 +173,46 @@
Examples
# All the passages in a cycle of 10 days over all the orbitss2_dop()#> date mission orbit
-#> 1: 2023-10-19 2A 002
-#> 2: 2023-10-19 2A 003
-#> 3: 2023-10-19 2A 004
-#> 4: 2023-10-19 2A 005
-#> 5: 2023-10-19 2A 006
+#> 1: 2023-11-09 2A 016
+#> 2: 2023-11-09 2A 017
+#> 3: 2023-11-09 2A 018
+#> 4: 2023-11-09 2A 019
+#> 5: 2023-11-09 2A 020#> ---
-#> 282: 2023-10-28 2B 068
-#> 283: 2023-10-28 2B 069
-#> 284: 2023-10-28 2B 070
-#> 285: 2023-10-28 2B 071
-#> 286: 2023-10-28 2B 072
+#> 282: 2023-11-18 2B 083
+#> 283: 2023-11-18 2B 084
+#> 284: 2023-11-18 2B 085
+#> 285: 2023-11-18 2B 086
+#> 286: 2023-11-18 2B 087# The passages in the current month over two orbitss2_dop(c("022", "065"), "this month")#> date mission orbit
-#> 1: 2023-10-03 2A 065
-#> 2: 2023-10-05 2B 022
-#> 3: 2023-10-08 2B 065
-#> 4: 2023-10-10 2A 022
-#> 5: 2023-10-13 2A 065
-#> 6: 2023-10-15 2B 022
-#> 7: 2023-10-18 2B 065
-#> 8: 2023-10-20 2A 022
-#> 9: 2023-10-23 2A 065
-#> 10: 2023-10-25 2B 022
-#> 11: 2023-10-28 2B 065
-#> 12: 2023-10-30 2A 022
+#> 1: 2023-11-02 2A 065
+#> 2: 2023-11-04 2B 022
+#> 3: 2023-11-07 2B 065
+#> 4: 2023-11-09 2A 022
+#> 5: 2023-11-12 2A 065
+#> 6: 2023-11-14 2B 022
+#> 7: 2023-11-17 2B 065
+#> 8: 2023-11-19 2A 022
+#> 9: 2023-11-22 2A 065
+#> 10: 2023-11-24 2B 022
+#> 11: 2023-11-27 2B 065
+#> 12: 2023-11-29 2A 022# The dates in which Sentinel-2A will pass in next six weeks over one orbits2_dop("022", "6 weeks", mission ="2A")$date
-#> [1] "2023-10-20" "2023-10-30" "2023-11-09" "2023-11-19"
+#> [1] "2023-11-09" "2023-11-19" "2023-11-29" "2023-12-09"# The date in which Sentinel-2A would be passed in the last 10 days over one orbits2_dop("022", "-10 days", mission ="2A")$date
-#> [1] "2023-10-10"
+#> [1] "2023-11-09"# All the orbits covered todays2_dop(timewindow =Sys.Date(), mission ="2B")$orbit
-#> [1] "073" "074" "075" "076" "077" "078" "079" "080" "081" "082" "083" "084"
-#> [13] "085" "086" "087"
+#> [1] "088" "089" "090" "091" "092" "093" "094" "095" "096" "097" "098" "099"
+#> [13] "100" "101"# The passages in a fixed time window for one orbits2_dop(65, as.Date(c("2018-08-01", "2018-08-31")))
diff --git a/docs/reference/s2_download.html b/docs/reference/s2_download.html
index 7f074f7c..005f9003 100644
--- a/docs/reference/s2_download.html
+++ b/docs/reference/s2_download.html
@@ -20,7 +20,7 @@
sen2r
- 1.5.6
+ 1.6.0
@@ -197,9 +197,9 @@
Author
Examples
if(FALSE){
-single_s2<-paste0("https://apihub.copernicus.eu/apihub/odata/v1/",
-"Products(\'c7142722-42bf-4f93-b8c5-59fd1792c430\')/$value")
-names(single_s2)<-"S2A_MSIL1C_20170613T101031_N0205_R022_T32TQQ_20170613T101608.SAFE"
+single_s2<-paste0("gs://gcp-public-data-sentinel-2/L2/tiles/32/T/NR/",
+"S2B_MSIL2A_20200804T101559_N0214_R065_T32TNR_20200804T145113.SAFE/")
+names(single_s2)<-"S2B_MSIL2A_20200804T101559_N0214_R065_T32TNR_20200804T145113.SAFE"# (this is equivalent to:# single_s2 <- example_s2_list[1]# where example_s2_list is the output of the example of the
@@ -207,15 +207,7 @@
Examples
# Download the whole products2_download(single_s2, outdir=tempdir())
-
-#' # Download the whole product - using aria2
-s2_download(single_s2, outdir=tempdir(), downloader ="aria2")
-
-# Download more products, ordering the ones stored in the Long Term Archive
-pos<-sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs =4326)
-time_window<-as.Date(c("2018-02-21", "2018-03-20"))
-list_safe<-s2_list(spatial_extent =pos, time_interval =time_window)
-s2_download(list_safe, outdir=tempdir())
+file.info(file.path(tempdir(),names(single_s2)))}
time_interval =c(Sys.Date()-10, Sys.Date()), time_period ="full", level ="auto",
- server ="scihub",
+ server ="gcloud", apihub =NA, service ="apihub", max_cloud =100,
@@ -156,25 +156,21 @@
Arguments
server
-
The servers where archives are searched.
-Available options are "scihub" (ESA Sentinel Hub) and "gcloud"
-(Google Cloud).
-Default is "scihub", meaning that only ESA Sentinel Hub is considered.
-In case of multiple values, they are used in order of priority.
-If availability = "check", products on LTA are always left as last choice.
+
(deprecate) Character vector of length 1, with the names of
+the servers on which SAFE archives are searched.
+Currently, only "gcloud" (Google Cloud) is supported.
+Old "scihub" (ESA Sentinel Hub) can no more be used, since November 2023,
+when the Copernicus Sentinel Data is no longer available and has been
+replaced by the Copernicus Data Space Ecosystem.
See also the section "Details".
apihub
-
Path of the apihub.txt file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.
+
deprecated
service
-
Character: it can be "dhus" or "apihub" (default),
-in which cases the required service is forced instead that the one present
-in the URLs passed through argument s2_prodlist.
+
deprecated
max_cloud
@@ -186,7 +182,7 @@
Arguments
Character argument, determining which products have
to be returned:
"online" : only archive names already available for download are returned;
"lta": only archive names stored in the
-Long Term Archive
+Long Term Archive
are returned;
"check": all archive names are returned, checking if they are
available or not for download (see "Value" to know
@@ -221,28 +217,11 @@
Value
Details
-
By default, SAFE archives are searched on ESA Sentinel Hub
-(argument server = "scihub"), which is the faster and stable option.
-After the reduction of the retention time to 30 days,
-it is highly probable that products older then 30 days will not be found
-online (see https://github.com/ranghetti/sen2r/issues/408).
-Moreover, after ordering them from Long Term Archive (LTA), in several cases
-corrupted archives are obtained
-(see https://github.com/ranghetti/sen2r/issues/406, and refer to this
-reference for details about the ESA LTA policy).
-
To avoid this problems, the research and download from Google Cloud was
-recently implemented.
-Users can set server = "gcloud" to use this data exclusively, or
-server = c("scihub", "gcloud"), availability = "check"
-to search for products available on SciHub first, or on Google Cloud
-subsequently.
-Important: to search and download from Google Cloud, Google Cloud SDK
+
Important: to search and download from Google Cloud, Google Cloud SDK
must be installed and configured following the indications in
https://cloud.google.com/sdk/docs/install.
Notice also that querying Google Cloud can be very slow (while downloading
them is generally faster than from SciHub).
Named character: list of the products to be ordered,
-in the format safelist (see safelist).
-Alternatively, it can be the path of a JSON file exported by a previous
-execution of s2_order, in case the user wants, for any reason, to
-resubmit the order.
-
-
export_prodlist
-
Logical or character: if TRUE (default),
-the list of ordered products is saved in a JSON text file, so to be easily
-retrievable at a later stage with safe_is_online or s2_download;
-if FALSE, no output files are generated.
-It is also possible to pass the path of an existing folder in which the
-JSON file will be saved (otherwise, a default path is used).
-
-
-
delay
-
Numeric: time frame (in seconds) to leave between two
-consecutive orders. Default is 0.5 seconds: use a higher value if you
-encountered errors (i.e. not all the products were correctly ordered).
-
-
-
apihub
-
Path of the apihub.txt file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.
-
-
-
service
-
Character: it can be "dhus" or "apihub", in which cases
-the required service is forced instead that the one present in the URLs
-passed through argument s2_prodlist.
-If NA (default), the service present in the URLs is maintained.
-
-
-
reorder
-
Logical: If TRUE, and a json file exported by s2_order
-is passed as argument to the function, try to order again also
-the "available" and "ordered" S2 datasets.
-Otherwise, only order the "notordered" ones.
-
-
-
-
Value
-
-
-
A named vector, containing the subset of s2_prodlist elements
-which were ordered.
-Moreover, the vector includes the following attributes:
"available" with the elements of s2_prodlist which were already
-available for download,
-
"notordered" with the elements of s2_prodlist which were not ordered
-for any reasons,
-
"path" (only if argument export_prodlist is not FALSE) with the path
-of the json file in which the list of the products (ordered, available
-and not ordered) was saved (if export_prodlist = TRUE).
-
-
-
Note
-
License: GPL 3.0
-
-
-
References
-
L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-"sen2r": An R toolbox for automatically downloading and preprocessing
-Sentinel-2 satellite data. Computers & Geosciences, 139, 104473.
-doi:10.1016/j.cageo.2020.104473
-, URL: https://sen2r.ranghetti.info/.
-
-
-
Author
-
Luigi Ranghetti, phD (2019)
-
Lorenzo Busetto, phD (2020)
-
-
-
-
Examples
-
if(FALSE){
-
-# Generate the lists of products
-pos<-sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs =4326)
-time_window<-as.Date(c("2019-10-21", "2019-11-20"))
-list_safe<-s2_list(spatial_extent =pos, time_interval =time_window)
-print(list_safe)
-# (at the time the documentation was updated, this list was containing 6
-# archives already available online and 3 stored in the Long Term Archive)
-
-# Order the products
-ordered_prods<-s2_order(list_safe)
-
-# Check in a second time if the product was made available
-(order_path<-attr(ordered_prods, "path"))
-safe_is_online(order_path)
-
-}
-
Named character: list of the products to be checked,
-in the format safelist (see safelist).
-Alternatively, it can be the path of a JSON file exported by s2_order.
-
-
apihub
-
Path of the "apihub.txt" file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.
-
-
-
verbose
-
Logical: if TRUE, provide processing messages summarising
-how many of the SAFE archives in s2_prodlist are available online.
-
-
-
-
Value
-
-
-
A logical vector of the same length and names of the SAFE products
-passed with s2_prodlist,
-in which each element is TRUE if the corresponding SAFE archive is
-available for download, FALSE if it is not or NA in case of errors with
-the SAFE url.
-
-
-
Note
-
License: GPL 3.0
-
-
-
References
-
L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-"sen2r": An R toolbox for automatically downloading and preprocessing
-Sentinel-2 satellite data. Computers & Geosciences, 139, 104473.
-doi:10.1016/j.cageo.2020.104473
-, URL: https://sen2r.ranghetti.info/.
-
-
-
Author
-
Luigi Ranghetti, phD (2019)
-
Lorenzo Busetto, phD (2020)
-
-
-
-
Examples
-
# \donttest{
-if(is_scihub_configured()){
-
-# Generate the lists of products
-pos<-sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs =4326)
-time_window<-as.Date(c("2018-02-21", "2018-03-20"))
-list_safe<-s2_list(spatial_extent =pos, time_interval =time_window)
-# (at the time the documentation was written, this list was containing 5
-# archives already available online and 2 stored in the Long Term Archive)
-
-# Check for availability
-safe_is_online(list_safe)
-
-}
-#> Spherical geometry (s2) switched off
-#> old-style crs object detected; please recreate object with a recent sf::st_crs()
-#> old-style crs object detected; please recreate object with a recent sf::st_crs()
-#> old-style crs object detected; please recreate object with a recent sf::st_crs()
-#> Spherical geometry (s2) switched on
-#> 0 out of 7 products are online.
-#> S2B_MSIL1C_20180224T133629_N0206_R038_T21FVC_20180224T194845.SAFE
-#> FALSE
-#> S2B_MSIL1C_20180227T134629_N0206_R081_T21FVC_20180227T200327.SAFE
-#> FALSE
-#> S2A_MSIL1C_20180301T133641_N0206_R038_T21FVC_20180301T145524.SAFE
-#> FALSE
-#> S2B_MSIL1C_20180306T133629_N0206_R038_T21FVC_20180306T183210.SAFE
-#> FALSE
-#> S2A_MSIL1C_20180311T133641_N0206_R038_T21FVC_20180311T163528.SAFE
-#> FALSE
-#> S2A_MSIL1C_20180314T134631_N0206_R081_T21FVC_20180314T150724.SAFE
-#> FALSE
-#> S2B_MSIL1C_20180316T133639_N0206_R038_T21FVC_20180316T145348.SAFE
-#> FALSE
-# }
-
read_scihub_login() reads the SciHub login information;
-
write_scihub_login() saves new username and password;
-
check_scihub_login() checks their validity;
-
is_scihub_configured() check if SciHub credentials are configured.
-
Login information is stored in a file apihub.txt inside the
-".sen2r" subfolder of the home directory. These functions allow reading
-or writing this file, and editing it from the GUI.
-In case file apihub.txt is missing, read_scihub_login() searches inside
-the environmental variables SCIHUB_USER and SCIHUB_PASSWORD.
+
These functions are deprecated and will be removed.
@@ -135,99 +114,7 @@
Import / export / check SciHub username and password
)
-
-
Arguments
-
apihub_path
-
Path of the file in which login information is saved.
-If NA (default) it is automatically read from the package default location.
-
-
-
username
-
SciHub username.
-
-
-
password
-
SciHub password.
-
-
-
service
-
Character: it can be "dhus" or "apihub" (default).
-
-
-
check
-
Logical: if TRUE (default), new credentials are checked
-before writing them on apihub_path (if they are invalid, an error
-is provided);
-if FALSE, they are directly written.
-
-
-
append
-
Logical: if TRUE, new credentials are added
-to the ones existing within apihub_path;
-if FALSE (default), apihub_path is replaced with the new ones.
-
-
-
-
Value
-
-
read_scihub_login() returns a matrix of credentials,
-in which username is in the first column, password in the second.
-
-
-
is_scihub_configured() returns TRUE if credentials can be
-accessed, FALSE if not.
-
-
-
check_scihub_login() returns TRUE if credentials are valid,
-FALSE elsewhere.
-
-
-
check_scihub_connection() returns TRUE if internet connection
-is available and SciHub is accessible, FALSE otherwise.
For this reason, newly created credentials can not immediately be used by sen2r,
-and password edits on old credentials are not immediately recognised.
-
-
-
Note
-
License: GPL 3.0
-
-
-
References
-
L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-"sen2r": An R toolbox for automatically downloading and preprocessing
-Sentinel-2 satellite data. Computers & Geosciences, 139, 104473.
-doi:10.1016/j.cageo.2020.104473
-, URL: https://sen2r.ranghetti.info/.
(optional) Character vector of length 1 or 2, with the names of
+
(deprecate) Character vector of length 1, with the names of
the servers on which SAFE archives are searched.
-Available options are "scihub" (ESA Sentinel Hub) and "gcloud"
-(Google Cloud).
-Default is "scihub", meaning that only ESA Sentinel Hub is considered.
-In case of multiple values, they are used in order of priority and
-products on LTA are always left as last choice.
+Currently, only "gcloud" (Google Cloud) is supported.
+Old "scihub" (ESA Sentinel Hub) can no more be used, since November 2023,
+when the Copernicus Sentinel Data is no longer available and has been
+replaced by the Copernicus Data Space Ecosystem.
See also the section "Details" of s2_list().
@@ -240,10 +239,7 @@
Arguments
apihub
-
Path of the text file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.
-It takes effect only if argument server includes "scihub".
+
deprecated
downloader
@@ -714,45 +710,43 @@
Examples
}else{out_paths_2<-character(0)}
-#> [2023-10-19 16:28:48] #### Starting sen2r execution. ####
+#> [2023-11-09 22:27:28] #### Starting sen2r execution. #####> Spherical geometry (s2) switched off#> Loading required namespace: stringi
-#> [2023-10-19 16:28:49] Searching for available SAFE products...
+#> [2023-11-09 22:27:30] Searching for available SAFE products...#> old-style crs object detected; please recreate object with a recent sf::st_crs()#> old-style crs object detected; please recreate object with a recent sf::st_crs()
-#> [2023-10-19 16:28:50] Querying Google Cloud (this can take a very long
+#> [2023-11-09 22:27:30] Querying Google Cloud (this can take a very long#> time)...
-#> [2023-10-19 16:46:33] Computing output names...
-#> [2023-10-19 16:46:34] Starting to download the required level-2A SAFE
+#> [2023-11-09 22:27:39] Computing output names...
+#> [2023-11-09 22:27:40] Starting to download the required level-2A SAFE#> products.
-#> Images
-#> S2B_MSIL2A_20200801T100559_N0214_R022_T32TNS_20200801T135302.SAFE are
-#> already on your system and will be skipped. Set "overwrite_safe" to
-#> TRUE to re-download them.
-#> No L2A images are needed.
-#> [2023-10-19 16:46:34] Download of level-2A SAFE products terminated.
-#> [2023-10-19 16:46:34] Starting to download the required level-1C SAFE
+#> [2023-11-09 22:27:40] Check if products are available for download...
+#> [2023-11-09 22:27:40] Downloading Sentinel-2 image 1 of 1
+#> (S2B_MSIL2A_20200801T100559_N0214_R022_T32TNS_20200801T135302.SAFE)...
+#> [2023-11-09 22:27:44] Download of level-2A SAFE products terminated.
+#> [2023-11-09 22:27:44] Starting to download the required level-1C SAFE#> products.#> No L1C images are needed.
-#> [2023-10-19 16:46:34] Download of level-1C SAFE products terminated.
-#> [2023-10-19 16:46:34] Updating output names...
-#> [2023-10-19 16:46:35] Starting to translate SAFE products in custom
+#> [2023-11-09 22:27:44] Download of level-1C SAFE products terminated.
+#> [2023-11-09 22:27:44] Updating output names...
+#> [2023-11-09 22:27:44] Starting to translate SAFE products in custom#> format.#> Using UTM zone 32N.#> 2 output files were correctly created.
-#> [2023-10-19 16:46:35] Starting to merge tiles by orbit.
-#> [2023-10-19 16:46:36] Starting to edit geometry (clip, reproject,
+#> [2023-11-09 22:27:45] Starting to merge tiles by orbit.
+#> [2023-11-09 22:27:45] Starting to edit geometry (clip, reproject,#> rescale).
-#> [2023-10-19 16:46:37] Producing required RGB images.
-#> [2023-10-19 16:46:37] Generating image
+#> [2023-11-09 22:27:45] Producing required RGB images.
+#> [2023-11-09 22:27:45] Generating image#> S2B2A_20200801_022_sen2r_RGB432B_10.tif...
-#> [2023-10-19 16:46:38] Generating image
+#> [2023-11-09 22:27:46] Generating image#> S2B2A_20200801_022_sen2r_RGB843B_10.tif...#> 2 output RGB files were correctly created.
-#> [2023-10-19 16:46:41] Computing required spectral indices.
-#> [2023-10-19 16:46:41] Computing index MSAVI2 on date 2020-08-01...
-#> [2023-10-19 16:46:42] Computing index NDVI on date 2020-08-01...
-#> [2023-10-19 16:46:43] Generating thumbnails.
+#> [2023-11-09 22:27:49] Computing required spectral indices.
+#> [2023-11-09 22:27:49] Computing index MSAVI2 on date 2020-08-01...
+#> [2023-11-09 22:27:50] Computing index NDVI on date 2020-08-01...
+#> [2023-11-09 22:27:51] Generating thumbnails.#> 6 output files were correctly created.#> ╔══════════════════════════════════════════════════════════════════════#> ║ sen2r Processing Report
@@ -760,9 +754,9 @@
Examples
#> ║ Dates to be processed based on processing parameters: 1#> ║ Processing completed for: all expected dates.#> ╚══════════════════════════════════════════════════════════════════════
-#> [2023-10-19 16:46:45] #### sen2r session terminated. ####
+#> [2023-11-09 22:27:53] #### sen2r session terminated. #####> The processing chain can be re-launched with the command:
-#> sen2r("C:/Users/LUIGI~1.RAN/DOCUME~1/SEN2R~1/proc_par/s2proc_20231019_162849.json")
+#> sen2r("/home/lranghetti/.sen2r/proc_par/s2proc_20231109_222728.json")#> Spherical geometry (s2) switched on# Notice that passing the path of a JSON file results in launching# a session without opening the gui, unless gui = TRUE is passed.
@@ -790,63 +784,61 @@
Examples
}else{out_paths_3<-character(0)}
-#> [2023-10-19 16:46:46] #### Starting sen2r execution. ####
+#> [2023-11-09 22:27:54] #### Starting sen2r execution. #####> Spherical geometry (s2) switched off
-#> [2023-10-19 16:46:46] Searching for available SAFE products...
+#> [2023-11-09 22:27:55] Searching for available SAFE products...#> old-style crs object detected; please recreate object with a recent sf::st_crs()#> old-style crs object detected; please recreate object with a recent sf::st_crs()
-#> [2023-10-19 16:46:47] Querying Google Cloud (this can take a very long
+#> [2023-11-09 22:27:55] Querying Google Cloud (this can take a very long#> time)...#> old-style crs object detected; please recreate object with a recent sf::st_crs()#> old-style crs object detected; please recreate object with a recent sf::st_crs()
-#> [2023-10-19 17:04:23] Querying Google Cloud (this can take a very long
+#> [2023-11-09 22:28:06] Querying Google Cloud (this can take a very long#> time)...
-#> [2023-10-19 17:21:55] Computing output names...
-#> [2023-10-19 17:21:56] Starting to download the required level-2A SAFE
+#> [2023-11-09 22:28:14] Computing output names...
+#> [2023-11-09 22:28:16] Starting to download the required level-2A SAFE#> products.#> Images#> S2B_MSIL2A_20200801T100559_N0214_R022_T32TNS_20200801T135302.SAFE are#> already on your system and will be skipped. Set "overwrite_safe" to#> TRUE to re-download them.#> No L2A images are needed.
-#> [2023-10-19 17:21:56] Download of level-2A SAFE products terminated.
-#> [2023-10-19 17:21:56] Starting to download the required level-1C SAFE
+#> [2023-11-09 22:28:16] Download of level-2A SAFE products terminated.
+#> [2023-11-09 22:28:16] Starting to download the required level-1C SAFE#> products.
-#> Images
-#> S2B_MSIL1C_20200801T100559_N0209_R022_T32TNS_20200801T130136.SAFE are
-#> already on your system and will be skipped. Set "overwrite_safe" to
-#> TRUE to re-download them.
-#> No L1C images are needed.
-#> [2023-10-19 17:21:56] Download of level-1C SAFE products terminated.
-#> [2023-10-19 17:21:56] Updating output names...
-#> [2023-10-19 17:21:56] Starting to translate SAFE products in custom
+#> [2023-11-09 22:28:16] Check if products are available for download...
+#> [2023-11-09 22:28:16] Downloading Sentinel-2 image 1 of 1
+#> (S2B_MSIL1C_20200801T100559_N0209_R022_T32TNS_20200801T130136.SAFE)...
+#> [2023-11-09 22:28:19] Download of level-1C SAFE products terminated.
+#> [2023-11-09 22:28:19] Updating output names...
+#> [2023-11-09 22:28:20] Starting to translate SAFE products in custom#> format.#> Using UTM zone 32N.#> 1 output files were correctly created.#> Using UTM zone 32N.#> 1 output angle files were correctly created.#> 3 output files were correctly created.
-#> [2023-10-19 17:21:57] Starting to merge tiles by orbit.
-#> [2023-10-19 17:21:58] Starting to edit geometry (clip, reproject,
+#> [2023-11-09 22:28:21] Starting to merge tiles by orbit.
+#> [2023-11-09 22:28:21] Starting to edit geometry (clip, reproject,#> rescale).
-#> [2023-10-19 17:21:59] Starting to apply cloud masks.
-#> [2023-10-19 17:21:59] Masking file
+#> [2023-11-09 22:28:21] Starting to apply cloud masks.
+#> [2023-11-09 22:28:21] Masking file#> S2B1C_20200801_022_Barbellino_TOA_10.tif...
-#> [2023-10-19 17:22:02] Masking file
+#> [2023-11-09 22:28:28] Masking file#> S2B2A_20200801_022_Barbellino_BOA_10.tif...
-#> [2023-10-19 17:22:05] Producing required RGB images.
-#> [2023-10-19 17:22:05] Generating image
+#> [2023-11-09 22:28:35] Producing required RGB images.
+#> [2023-11-09 22:28:35] Generating image#> S2B1C_20200801_022_Barbellino_RGB432T_10.tif...#> 1 output RGB files were correctly created.
-#> [2023-10-19 17:22:07] Generating image
+#> [2023-11-09 22:28:36] Generating image#> S2B2A_20200801_022_Barbellino_RGB432B_10.tif...
-#> [2023-10-19 17:22:08] Generating image
+#> [2023-11-09 22:28:37] Generating image#> S2B2A_20200801_022_Barbellino_RGB843B_10.tif...#> 2 output RGB files were correctly created.
-#> [2023-10-19 17:22:11] Computing required spectral indices.
-#> [2023-10-19 17:22:11] Computing index NDVI on date 2020-08-01...
-#> [2023-10-19 17:22:11] Computing index MSAVI2 on date 2020-08-01...
-#> [2023-10-19 17:22:11] Generating thumbnails.
+#> [2023-11-09 22:28:40] Computing required spectral indices.
+#> [2023-11-09 22:28:40] Computing index NDVI on date 2020-08-01...
+#> [2023-11-09 22:28:40] Computing index MSAVI2 on date 2020-08-01...
+#> [2023-11-09 22:28:40] Generating thumbnails.#> 9 output files were correctly created.#> ╔══════════════════════════════════════════════════════════════════════#> ║ sen2r Processing Report
@@ -854,9 +846,9 @@
#> ORDER[2],#> LENGTHUNIT["metre",1]],#> USAGE[
-#> SCOPE["Navigation and medium accuracy spatial referencing."],
+#> SCOPE["Engineering survey, topographic mapping."],#> AREA["Between 132°W and 126°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - British Columbia (BC); NorthW Territories (NWT); Yukon. United States (USA) - Alaska (AK)."],#> BBOX[0,-132,84,-126]],#> ID["EPSG",32609]]
@@ -242,7 +242,7 @@
Examples
#> ORDER[2],#> LENGTHUNIT["metre",1]],#> USAGE[
-#> SCOPE["Navigation and medium accuracy spatial referencing."],
+#> SCOPE["Engineering survey, topographic mapping."],#> AREA["Between 132°W and 126°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - British Columbia (BC); NorthW Territories (NWT); Yukon. United States (USA) - Alaska (AK)."],#> BBOX[0,-132,84,-126]],#> ID["EPSG",32609]]
@@ -286,7 +286,7 @@
Examples
#> ORDER[2],#> LENGTHUNIT["metre",1]],#> USAGE[
-#> SCOPE["Navigation and medium accuracy spatial referencing."],
+#> SCOPE["Engineering survey, topographic mapping."],#> AREA["Between 132°W and 126°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - British Columbia (BC); NorthW Territories (NWT); Yukon. United States (USA) - Alaska (AK)."],#> BBOX[0,-132,84,-126]],#> ID["EPSG",32609]]
@@ -328,7 +328,7 @@
Examples
#> ORDER[2],#> LENGTHUNIT["metre",1]],#> USAGE[
-#> SCOPE["Navigation and medium accuracy spatial referencing."],
+#> SCOPE["Engineering survey, topographic mapping."],#> AREA["Between 132°W and 126°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - British Columbia (BC); NorthW Territories (NWT); Yukon. United States (USA) - Alaska (AK)."],#> BBOX[0,-132,84,-126]],#> ID["EPSG",32609]]
@@ -370,7 +370,7 @@
Examples
#> ORDER[2],#> LENGTHUNIT["metre",1]],#> USAGE[
-#> SCOPE["Navigation and medium accuracy spatial referencing."],
+#> SCOPE["Engineering survey, topographic mapping."],#> AREA["Between 132°W and 126°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - British Columbia (BC); NorthW Territories (NWT); Yukon. United States (USA) - Alaska (AK)."],#> BBOX[0,-132,84,-126]],#> ID["EPSG",32609]]
@@ -412,7 +412,7 @@
Examples
#> ORDER[2],#> LENGTHUNIT["metre",1]],#> USAGE[
-#> SCOPE["Navigation and medium accuracy spatial referencing."],
+#> SCOPE["Engineering survey, topographic mapping."],#> AREA["Between 132°W and 126°W, southern hemisphere between 80°S and equator, onshore and offshore."],#> BBOX[-80,-132,0,-126]],#> ID["EPSG",32709]]
diff --git a/docs/reference/stack2rgb.html b/docs/reference/stack2rgb.html
index 114a452b..7b1f160e 100644
--- a/docs/reference/stack2rgb.html
+++ b/docs/reference/stack2rgb.html
@@ -19,7 +19,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/reference/str_pad2.html b/docs/reference/str_pad2.html
index 640751b2..f2c57e75 100644
--- a/docs/reference/str_pad2.html
+++ b/docs/reference/str_pad2.html
@@ -21,7 +21,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/reference/suppress_warnings.html b/docs/reference/suppress_warnings.html
index 37cc5a56..ce9df3bd 100644
--- a/docs/reference/suppress_warnings.html
+++ b/docs/reference/suppress_warnings.html
@@ -17,7 +17,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/reference/tile_utmzone.html b/docs/reference/tile_utmzone.html
index 569e68d8..9e2edaa5 100644
--- a/docs/reference/tile_utmzone.html
+++ b/docs/reference/tile_utmzone.html
@@ -18,7 +18,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/reference/tiles_intersects.html b/docs/reference/tiles_intersects.html
index f6793e4f..cd8bc1aa 100644
--- a/docs/reference/tiles_intersects.html
+++ b/docs/reference/tiles_intersects.html
@@ -18,7 +18,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/reference/trace_function.html b/docs/reference/trace_function.html
index 9cc4d2ea..7e9f1d1b 100644
--- a/docs/reference/trace_function.html
+++ b/docs/reference/trace_function.html
@@ -36,7 +36,7 @@
sen2r
- 1.5.6
+ 1.6.0
diff --git a/docs/sitemap.xml b/docs/sitemap.xml
index a70d4830..50726de3 100644
--- a/docs/sitemap.xml
+++ b/docs/sitemap.xml
@@ -4,40 +4,40 @@
/404.html
- /articles/docker.html
+ /CODE-OF-CONDUCT.html
- /articles/index.html
+ /CONDUCT.html
- /articles/installation.html
+ /ISSUE_TEMPLATE.html
- /articles/outstructure.html
+ /LICENSE.html
- /articles/sen2r_cmd.html
+ /articles/docker.html
- /articles/sen2r_gui.html
+ /articles/index.html
- /authors.html
+ /articles/installation.html
- /CODE-OF-CONDUCT.html
+ /articles/outstructure.html
- /CONDUCT.html
+ /articles/sen2r_cmd.html
- /index.html
+ /articles/sen2r_gui.html
- /ISSUE_TEMPLATE.html
+ /authors.html
- /LICENSE.html
+ /index.html/news/index.html
@@ -102,9 +102,6 @@
/reference/gdalUtil.html
-
- /reference/gdalwarp_grid.html
- /reference/gdal_abs2rel_rel2abs.html
@@ -114,6 +111,9 @@
/reference/gdal_warp.html
+
+ /reference/gdalwarp_grid.html
+ /reference/gipp.html
@@ -213,9 +213,6 @@
/reference/s2_translate.html
-
- /reference/safelist-class.html
- /reference/safe_getMetadata.html
@@ -225,6 +222,9 @@
/reference/safe_shortname.html
+
+ /reference/safelist-class.html
+ /reference/scihub_login.html
@@ -244,25 +244,25 @@
/reference/smooth_mask.html
- /reference/stack2rgb.html
+ /reference/st_as_text_2.html
- /reference/str_pad2.html
+ /reference/st_crs2.html
- /reference/st_as_text_2.html
+ /reference/stack2rgb.html
- /reference/st_crs2.html
+ /reference/str_pad2.html/reference/suppress_warnings.html
- /reference/tiles_intersects.html
+ /reference/tile_utmzone.html
- /reference/tile_utmzone.html
+ /reference/tiles_intersects.html/reference/trace_function.html
diff --git a/index.Rmd b/index.Rmd
index bf3b67b2..deef35e1 100644
--- a/index.Rmd
+++ b/index.Rmd
@@ -52,12 +52,18 @@ allows to:
* mask cloudy pixels (using the Surface Classification Map as masking layer);
* computing spectral indices and RGB images.
-**New feature**: starting from version 1.5.0,
-Sentinel-2 SAFE archives can be downloaded both from official
-[Copernicus Open Access Hub](https://scihub.copernicus.eu/) or from
-[Google Cloud Sentinel-2 bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2)
-(this option can be useful to bypass limitations of ESA Long Term Archive
-retention time; refer to [this post](https://luigi.ranghetti.info/post/safe-gcloud/) for details).
+**Important**: since November 2023,
+Sentinel-2 SAFE archives can be searched and retrieved only from [Google Cloud Sentinel-2 bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2);
+to use it, **Google Cloud SDK must be installed** and configured following the
+[official instructions](https://cloud.google.com/sdk/docs/install).
+Refer to [this post](https://luigi.ranghetti.info/post/safe-gcloud/) for details
+about how to install configure the required external dependency Google Cloud SDK.
+Notice that this mode is not optimal (a modern STAC API approach should be used),
+and searching products could be a very slow process.
+Nevertheless, the replacement of the old official data source (ESA Hub) with the
+new [Copernicus Data Space](https://dataspace.copernicus.eu/analyse/apis) is not
+planned (this package is no longer actively maintained due to a career change of the developer).
+Anyone who wants to help [can contribute](https://github.com/ranghetti/sen2r/issues/464)).
Setting the execution of this processing chain is particularly easy using the
sen2r
diff --git a/index.md b/index.md
index b9ed633a..634ced99 100644
--- a/index.md
+++ b/index.md
@@ -53,13 +53,18 @@ allows to:
layer);
- computing spectral indices and RGB images.
-**New feature**: starting from version 1.5.0, Sentinel-2 SAFE archives
-can be downloaded both from official [Copernicus Open Access
-Hub](https://scihub.copernicus.eu/) or from [Google Cloud Sentinel-2
-bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2)
-(this option can be useful to bypass limitations of ESA Long Term
-Archive retention time; refer to [this
-post](https://luigi.ranghetti.info/post/safe-gcloud/) for details).
+**Important**: since November 2023,
+Sentinel-2 SAFE archives can be searched and retrieved only from [Google Cloud Sentinel-2 bucket](https://cloud.google.com/storage/docs/public-datasets/sentinel-2);
+to use it, **Google Cloud SDK must be installed** and configured following the
+[official instructions](https://cloud.google.com/sdk/docs/install).
+Refer to [this post](https://luigi.ranghetti.info/post/safe-gcloud/) for details
+about how to install configure the required external dependency Google Cloud SDK.
+Notice that this mode is not optimal (a modern STAC API approach should be used),
+and searching products could be a very slow process.
+Nevertheless, the replacement of the old official data source (ESA Hub) with the
+new [Copernicus Data Space](https://dataspace.copernicus.eu/analyse/apis) is not
+planned (this package is no longer actively maintained due to a career change of the developer).
+Anyone who wants to help [can contribute](https://github.com/ranghetti/sen2r/issues/464)).
Setting the execution of this processing chain is particularly easy
using the \n \n \n \n \n \n NIR<\/mi>\n -<\/mo>\n \n (<\/mo>\n \n \n SWIR1<\/mi>\n -<\/mo>\n SWIR2<\/mi>\n <\/mrow>\n <\/mrow>\n )<\/mo>\n <\/mrow>\n <\/mrow>\n <\/mrow>\n <\/mrow>\n \n \n \n NIR<\/mi>\n +<\/mo>\n \n (<\/mo>\n \n \n SWIR1<\/mi>\n -<\/mo>\n SWIR2<\/mi>\n <\/mrow>\n <\/mrow>\n )<\/mo>\n <\/mrow>\n <\/mrow>\n <\/mrow>\n <\/mrow>\n <\/mfrac>\n <\/mrow>\n<\/math>"
}
],
- "pkg_version": ["1.5.4.9000"],
+ "pkg_version": ["1.6.0"],
"creation_date": ["2023-05-30 16:07:39.257282"]
}
diff --git a/man/s2_download.Rd b/man/s2_download.Rd
index 483562ff..232ef263 100644
--- a/man/s2_download.Rd
+++ b/man/s2_download.Rd
@@ -65,9 +65,9 @@ License: GPL 3.0
}
\examples{
\dontrun{
-single_s2 <- paste0("https://apihub.copernicus.eu/apihub/odata/v1/",
- "Products(\'c7142722-42bf-4f93-b8c5-59fd1792c430\')/$value")
-names(single_s2) <- "S2A_MSIL1C_20170613T101031_N0205_R022_T32TQQ_20170613T101608.SAFE"
+single_s2 <- paste0("gs://gcp-public-data-sentinel-2/L2/tiles/32/T/NR/",
+ "S2B_MSIL2A_20200804T101559_N0214_R065_T32TNR_20200804T145113.SAFE/")
+names(single_s2) <- "S2B_MSIL2A_20200804T101559_N0214_R065_T32TNR_20200804T145113.SAFE"
# (this is equivalent to:
# single_s2 <- example_s2_list[1]
# where example_s2_list is the output of the example of the
@@ -75,15 +75,7 @@ names(single_s2) <- "S2A_MSIL1C_20170613T101031_N0205_R022_T32TQQ_20170613T10160
# Download the whole product
s2_download(single_s2, outdir=tempdir())
-
-#' # Download the whole product - using aria2
-s2_download(single_s2, outdir=tempdir(), downloader = "aria2")
-
-# Download more products, ordering the ones stored in the Long Term Archive
-pos <- sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs = 4326)
-time_window <- as.Date(c("2018-02-21", "2018-03-20"))
-list_safe <- s2_list(spatial_extent = pos, time_interval = time_window)
-s2_download(list_safe, outdir=tempdir())
+file.info(file.path(tempdir(),names(single_s2)))
}
}
\references{
diff --git a/man/s2_list.Rd b/man/s2_list.Rd
index fc7fee9a..0cbb94ee 100644
--- a/man/s2_list.Rd
+++ b/man/s2_list.Rd
@@ -11,7 +11,7 @@ s2_list(
time_interval = c(Sys.Date() - 10, Sys.Date()),
time_period = "full",
level = "auto",
- server = "scihub",
+ server = "gcloud",
apihub = NA,
service = "apihub",
max_cloud = 100,
@@ -48,21 +48,17 @@ product
- "L1C": list available level-1C products
- "L2A": list available level-2A products}
-\item{server}{The servers where archives are searched.
-Available options are \code{"scihub"} (ESA Sentinel Hub) and \code{"gcloud"}
-(Google Cloud).
-Default is \code{"scihub"}, meaning that only ESA Sentinel Hub is considered.
-In case of multiple values, they are used in order of priority.
-If \code{availability = "check"}, products on LTA are always left as last choice.
+\item{server}{(deprecate) Character vector of length 1, with the names of
+the servers on which SAFE archives are searched.
+Currently, only \code{"gcloud"} (Google Cloud) is supported.
+Old \code{"scihub"} (ESA Sentinel Hub) can no more be used, since November 2023,
+when the Copernicus Sentinel Data is no longer available and has been
+replaced by the Copernicus Data Space Ecosystem.
See also the section "Details".}
-\item{apihub}{Path of the \code{apihub.txt} file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.}
+\item{apihub}{\emph{deprecated}}
-\item{service}{Character: it can be \code{"dhus"} or \code{"apihub"} (default),
-in which cases the required service is forced instead that the one present
-in the URLs passed through argument \code{s2_prodlist}.}
+\item{service}{\emph{deprecated}}
\item{max_cloud}{Integer number (0-100) containing the maximum cloud
level of the tiles to be listed (default: no filter).}
@@ -72,7 +68,7 @@ to be returned:
\itemize{
\item \code{"online"} : only archive names already available for download are returned;
\item \code{"lta"}: only archive names stored in the
-\href{https://scihub.copernicus.eu/userguide/LongTermArchive}{Long Term Archive}
+Long Term Archive
are returned;
\item \code{"check"}: all archive names are returned, checking if they are
available or not for download (see "Value" to know
@@ -102,30 +98,11 @@ The function retrieves the list of available Sentinel-2
products satisfying given search criteria.
}
\details{
-By default, SAFE archives are searched on ESA Sentinel Hub
-(argument \code{server = "scihub"}), which is the faster and stable option.
-After the reduction of the retention time to 30 days,
-it is highly probable that products older then 30 days will not be found
-online (see \url{https://github.com/ranghetti/sen2r/issues/408}).
-Moreover, after ordering them from Long Term Archive (LTA), in several cases
-corrupted archives are obtained
-(see \url{https://github.com/ranghetti/sen2r/issues/406}, and refer to this
-reference for details about the ESA LTA policy).
-
-To avoid this problems, the research and download from Google Cloud was
-recently implemented.
-Users can set \code{server = "gcloud"} to use this data exclusively, or
-\verb{server = c("scihub", "gcloud"), availability = "check"}
-to search for products available on SciHub first, or on Google Cloud
-subsequently.
\strong{Important}: to search and download from Google Cloud, Google Cloud SDK
must be installed and configured following the indications in
\url{https://cloud.google.com/sdk/docs/install}.
Notice also that querying Google Cloud can be very slow (while downloading
them is generally faster than from SciHub).
-
-Searching and downloading from Google Cloud is an experimental feature;
-in case of bugs, report them at \url{https://github.com/ranghetti/sen2r/issues}.
}
\note{
License: GPL 3.0
@@ -134,10 +111,10 @@ License: GPL 3.0
\donttest{
pos <- sf::st_sfc(sf::st_point(c(9.85,45.81)), crs = 4326)
-time_window <- as.Date(c("2016-05-01", "2017-07-30"))
+time_window <- as.Date(c("2020-08-01", "2020-08-11"))
# Full-period list
-if (is_scihub_configured()) {
+if (is_gcloud_configured()) {
example_s2_list <- s2_list(
spatial_extent = pos,
tile = "32TNR",
@@ -152,7 +129,7 @@ print(example_s2_list)
safe_getMetadata(example_s2_list, "sensing_datetime")
# Seasonal-period list
-if (is_scihub_configured()) {
+if (is_gcloud_configured()) {
example_s2_list <- s2_list(
spatial_extent = pos,
tile = "32TNR",
diff --git a/man/s2_order.Rd b/man/s2_order.Rd
index 2ae7ea0f..ef66451f 100644
--- a/man/s2_order.Rd
+++ b/man/s2_order.Rd
@@ -2,7 +2,7 @@
% Please edit documentation in R/s2_order.R
\name{s2_order}
\alias{s2_order}
-\title{Order S2 products.}
+\title{Order S2 products (deprecated).}
\usage{
s2_order(
s2_prodlist = NULL,
@@ -14,86 +14,21 @@ s2_order(
)
}
\arguments{
-\item{s2_prodlist}{Named character: list of the products to be ordered,
-in the format \code{safelist} (see \linkS4class{safelist}).
-Alternatively, it can be the path of a JSON file exported by a previous
-execution of \link{s2_order}, in case the user wants, for any reason, to
-resubmit the order.}
+\item{s2_prodlist}{deprecated}
-\item{export_prodlist}{Logical or character: if TRUE (default),
-the list of ordered products is saved in a JSON text file, so to be easily
-retrievable at a later stage with \link{safe_is_online} or \link{s2_download};
-if FALSE, no output files are generated.
-It is also possible to pass the path of an existing folder in which the
-JSON file will be saved (otherwise, a default path is used).}
+\item{export_prodlist}{deprecated}
-\item{delay}{Numeric: time frame (in seconds) to leave between two
-consecutive orders. Default is 0.5 seconds: use a higher value if you
-encountered errors (i.e. not all the products were correctly ordered).}
+\item{delay}{deprecated}
-\item{apihub}{Path of the \code{apihub.txt} file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.}
+\item{apihub}{deprecated}
-\item{service}{Character: it can be \code{"dhus"} or \code{"apihub"}, in which cases
-the required service is forced instead that the one present in the URLs
-passed through argument \code{s2_prodlist}.
-If NA (default), the service present in the URLs is maintained.}
+\item{service}{deprecated}
-\item{reorder}{Logical: If TRUE, and a json file exported by s2_order
-is passed as argument to the function, try to order again also
-the \code{"available"} and \code{"ordered"} S2 datasets.
-Otherwise, only order the \code{"notordered"} ones.}
+\item{reorder}{deprecated}
}
\value{
-A named vector, containing the subset of \code{s2_prodlist} elements
-which were ordered.
-Moreover, the vector includes the following attributes:
-\itemize{
-\item \code{"available"} with the elements of \code{s2_prodlist} which were already
-available for download,
-\item \code{"notordered"} with the elements of \code{s2_prodlist} which were not ordered
-for any reasons,
-\item \code{"path"} (only if argument \code{export_prodlist} is not FALSE) with the path
-of the json file in which the list of the products (ordered, available
-and not ordered) was saved (if \code{export_prodlist = TRUE}).
-}
+deprecated
}
\description{
-The function orders S2 products from
-\href{https://scihub.copernicus.eu/userguide/LongTermArchive}{Long Term Archive}.
-}
-\note{
-License: GPL 3.0
-}
-\examples{
-\dontrun{
-
-# Generate the lists of products
-pos <- sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs = 4326)
-time_window <- as.Date(c("2019-10-21", "2019-11-20"))
-list_safe <- s2_list(spatial_extent = pos, time_interval = time_window)
-print(list_safe)
-# (at the time the documentation was updated, this list was containing 6
-# archives already available online and 3 stored in the Long Term Archive)
-
-# Order the products
-ordered_prods <- s2_order(list_safe)
-
-# Check in a second time if the product was made available
-(order_path <- attr(ordered_prods, "path"))
-safe_is_online(order_path)
-
-}
-}
-\references{
-L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-"sen2r": An R toolbox for automatically downloading and preprocessing
-Sentinel-2 satellite data. \emph{Computers & Geosciences}, 139, 104473.
-\doi{10.1016/j.cageo.2020.104473}, URL: \url{https://sen2r.ranghetti.info/}.
-}
-\author{
-Luigi Ranghetti, phD (2019)
-
-Lorenzo Busetto, phD (2020)
+This function is deprecated and will be removed.
}
diff --git a/man/safe_is_online.Rd b/man/safe_is_online.Rd
index 670fda96..78d5ce62 100644
--- a/man/safe_is_online.Rd
+++ b/man/safe_is_online.Rd
@@ -2,62 +2,20 @@
% Please edit documentation in R/safe_is_online.R
\name{safe_is_online}
\alias{safe_is_online}
-\title{Check if SAFE is available for download}
+\title{Check if SAFE is available for download (deprecated)}
\usage{
safe_is_online(s2_prodlist = NULL, apihub = NA, verbose = TRUE)
}
\arguments{
-\item{s2_prodlist}{Named character: list of the products to be checked,
-in the format \code{safelist} (see \linkS4class{safelist}).
-Alternatively, it can be the path of a JSON file exported by \link{s2_order}.}
+\item{s2_prodlist}{deprecated}
-\item{apihub}{Path of the "apihub.txt" file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.}
+\item{apihub}{deprecated}
-\item{verbose}{Logical: if TRUE, provide processing messages summarising
-how many of the SAFE archives in \code{s2_prodlist} are available online.}
+\item{verbose}{deprecated}
}
\value{
-A logical vector of the same length and names of the SAFE products
-passed with \code{s2_prodlist},
-in which each element is TRUE if the corresponding SAFE archive is
-available for download, FALSE if it is not or NA in case of errors with
-the SAFE url.
+deprecated
}
\description{
-The function checks if the required SAFE archives are
-available for download, or if they have to be ordered from the Long Term
-Archive.
-}
-\note{
-License: GPL 3.0
-}
-\examples{
-\donttest{
-if (is_scihub_configured()) {
-
-# Generate the lists of products
-pos <- sf::st_sfc(sf::st_point(c(-57.8815,-51.6954)), crs = 4326)
-time_window <- as.Date(c("2018-02-21", "2018-03-20"))
-list_safe <- s2_list(spatial_extent = pos, time_interval = time_window)
-# (at the time the documentation was written, this list was containing 5
-# archives already available online and 2 stored in the Long Term Archive)
-
-# Check for availability
-safe_is_online(list_safe)
-
-}
-}
-}
-\references{
-L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-"sen2r": An R toolbox for automatically downloading and preprocessing
-Sentinel-2 satellite data. \emph{Computers & Geosciences}, 139, 104473.
-\doi{10.1016/j.cageo.2020.104473}, URL: \url{https://sen2r.ranghetti.info/}.
-}
-\author{
-Luigi Ranghetti, phD (2019)
-
-Lorenzo Busetto, phD (2020)
+This function is deprecated and will be removed.
}
diff --git a/man/scihub_login.Rd b/man/scihub_login.Rd
index dbd7b65e..50ed51bf 100644
--- a/man/scihub_login.Rd
+++ b/man/scihub_login.Rd
@@ -6,7 +6,7 @@
\alias{check_scihub_login}
\alias{check_scihub_connection}
\alias{write_scihub_login}
-\title{Import / export / check SciHub username and password}
+\title{Import / export / check SciHub username and password (deprecated)}
\usage{
read_scihub_login(apihub_path = NA)
@@ -25,82 +25,23 @@ write_scihub_login(
)
}
\arguments{
-\item{apihub_path}{Path of the file in which login information is saved.
-If NA (default) it is automatically read from the package default location.}
+\item{apihub_path}{deprecated}
-\item{username}{SciHub username.}
+\item{username}{deprecated}
-\item{password}{SciHub password.}
+\item{password}{deprecated}
-\item{service}{Character: it can be \code{"dhus"} or \code{"apihub"} (default).}
+\item{service}{deprecated}
-\item{check}{Logical: if TRUE (default), new credentials are checked
-before writing them on \code{apihub_path} (if they are invalid, an error
-is provided);
-if FALSE, they are directly written.}
+\item{check}{deprecated}
-\item{append}{Logical: if TRUE, new credentials are added
-to the ones existing within \code{apihub_path};
-if FALSE (default), \code{apihub_path} is replaced with the new ones.}
+\item{append}{deprecated}
}
\value{
-\code{read_scihub_login()} returns a matrix of credentials,
-in which \code{username} is in the first column, \code{password} in the second.
+deprecated
-\code{is_scihub_configured()} returns TRUE if credentials can be
-accessed, FALSE if not.
-
-\code{check_scihub_login()} returns TRUE if credentials are valid,
-FALSE elsewhere.
-
-\code{check_scihub_connection()} returns TRUE if internet connection
-is available and SciHub is accessible, FALSE otherwise.
-
-\code{write_scihub_login()} returns NULL.
+deprecated
}
\description{
-\itemize{
-\item \code{read_scihub_login()} reads the SciHub login information;
-\item \code{write_scihub_login()} saves new username and password;
-\item \code{check_scihub_login()} checks their validity;
-\item \code{is_scihub_configured()} check if SciHub credentials are configured.
-}
-
-Login information is stored in a file \code{apihub.txt} inside the
-".sen2r" subfolder of the home directory. These functions allow reading
-or writing this file, and editing it from the GUI.
-In case file \code{apihub.txt} is missing, \code{read_scihub_login()} searches inside
-the environmental variables \code{SCIHUB_USER} and \code{SCIHUB_PASSWORD}.
-}
-\details{
-Notice that new/recently updated SciHub credentials are recognised by API Hub
-with a delay of about one week (see \url{https://scihub.copernicus.eu/twiki/do/view/SciHubWebPortal/APIHubDescription} for details).
-
-For this reason, newly created credentials can not immediately be used by \code{sen2r},
-and password edits on old credentials are not immediately recognised.
-}
-\note{
-License: GPL 3.0
-}
-\examples{
-\donttest{
-is_scihub_configured()
-}
-\dontrun{
-check_scihub_connection()
-check_scihub_login("username", "password")
-write_scihub_login("username", "password")
-read_scihub_login()
-}
-}
-\references{
-L. Ranghetti, M. Boschetti, F. Nutini, L. Busetto (2020).
-"sen2r": An R toolbox for automatically downloading and preprocessing
-Sentinel-2 satellite data. \emph{Computers & Geosciences}, 139, 104473.
-\doi{10.1016/j.cageo.2020.104473}, URL: \url{https://sen2r.ranghetti.info/}.
-}
-\author{
-Luigi Ranghetti, phD (2019)
-
-Lorenzo Busetto, phD (2019)
+These functions are deprecated and will be removed.
}
diff --git a/man/sen2r.Rd b/man/sen2r.Rd
index b2b429d4..10735312 100644
--- a/man/sen2r.Rd
+++ b/man/sen2r.Rd
@@ -11,7 +11,7 @@ sen2r(
s2_levels = "l2a",
sel_sensor = c("s2a", "s2b"),
online = TRUE,
- server = "scihub",
+ server = "gcloud",
order_lta = TRUE,
apihub = NA,
downloader = "builtin",
@@ -106,13 +106,12 @@ Accepted values: "s2a" and "s2b"; default: c("s2a","s2b").}
products on SciHub and/or Google Cloud (and download if needed);
FALSE to work only with already downloaded SAFE products.}
-\item{server}{(optional) Character vector of length 1 or 2, with the names of
+\item{server}{(deprecate) Character vector of length 1, with the names of
the servers on which SAFE archives are searched.
-Available options are \code{"scihub"} (ESA Sentinel Hub) and \code{"gcloud"}
-(Google Cloud).
-Default is \code{"scihub"}, meaning that only ESA Sentinel Hub is considered.
-In case of multiple values, they are used in order of priority and
-products on LTA are always left as last choice.
+Currently, only \code{"gcloud"} (Google Cloud) is supported.
+Old \code{"scihub"} (ESA Sentinel Hub) can no more be used, since November 2023,
+when the Copernicus Sentinel Data is no longer available and has been
+replaced by the Copernicus Data Space Ecosystem.
See also the section "Details" of \code{s2_list()}.}
\item{order_lta}{(optional) Logical: TRUE (default) to order products from
@@ -120,10 +119,7 @@ the Long Term Archive if unavailable for direct download; FALSE to simply
skip them (this option has effect only in online mode).
It takes effect only if argument \code{server} includes \code{"scihub"}.}
-\item{apihub}{Path of the text file containing credentials
-of SciHub account.
-If NA (default), the default location inside the package will be used.
-It takes effect only if argument \code{server} includes \code{"scihub"}.}
+\item{apihub}{\emph{deprecated}}
\item{downloader}{(optional) Character value corresponding to the executable
which should be used to download SAFE products. It could be one among
diff --git a/vignettes/sen2r_cmd.Rmd b/vignettes/sen2r_cmd.Rmd
index 2ccd0f0c..c87ae3a8 100644
--- a/vignettes/sen2r_cmd.Rmd
+++ b/vignettes/sen2r_cmd.Rmd
@@ -188,8 +188,7 @@ The processing chain can be re-launched with the command:
```
This is for example particularly useful if a `sen2r()` processing requires
-ordering images from the LTA archive
-(see https://scihub.copernicus.eu/userguide/LongTermArchive).
+ordering images from the LTA archive.
The user can in fact, in that case: