renv::install() error when installing from older PPM snapshots

[bzkrouse]

As of {renv} 1.0.4 I am receiving an error when installing packages from older PPM snapshots. It seems to disappear from ~ 2023-11-20.

renv::install("dplyr", repos = "https://packagemanager.posit.co/cran/2023-11-19")

results in:

# Downloading packages -------------------------------------------------------
Error in match.arg(type, c("binary", "source")) : 
  'arg' should be one of “binary”, “source”
Traceback (most recent calls last):
11: renv::install("dplyr", repos = "https://packagemanager.posit.co/cran/2023-11-19")
10: retrieve(packages)
 9: handler(package, renv_retrieve_impl(package))
 8: renv_retrieve_impl(package)
 7: renv_retrieve_repos(record)
 6: renv_retrieve_repos_impl(record)
 5: name %||% renv_retrieve_repos_archive_name(record, type)
 4: renv_retrieve_repos_archive_name(record, type)
 3: renv_package_ext(type)
 2: match.arg(type, c("binary", "source"))
 1: stop(sprintf(ngettext(length(chs <- unique(choices[nzchar(choices)])), 
        "'arg' should be %s", "'arg' should be one of %s"), paste(dQuote(chs), 
        collapse = ", ")), domain = NA)

Please let me know what other information might be useful. Thanks!!

[kevinushey]

Thanks for the bug report -- I was able to reproduce locally as well. I'll take a look!

[kevinushey]

Thanks; this should now be resolved in the development version of renv. I'll try to get a new version on CRAN soon to get the fix rolled out.

[bzkrouse]

@kevinushey Thanks so much for the quick fix!!!
I tested again with a pre-defined list of packages and received a different error for {survival}. Example below:

renv::install("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")

results in:

# Downloading packages -------------------------------------------------------
- Downloading survival from P3M ...             	ERROR [error code 22]
Error: error downloading 'https://packagemanager.posit.co/cran/2023-12-06/bin/windows/contrib/4.3/survival_3.5-7.zip' [error code 22]
Traceback (most recent calls last):
8: renv::install("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")
7: retrieve(packages)
6: handler(package, renv_retrieve_impl(package))
5: renv_retrieve_impl(package)
4: renv_retrieve_repos(record)
3: renv_retrieve_repos_impl(record)
2: renv_retrieve_package(record, url, path)
1: stop(status)

[kevinushey]

Hmm, I wasn't able to reproduce that...

> renv::install("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")
# Downloading packages -------------------------------------------------------
- Downloading survival from CRAN ...            OK [7 Mb in 4.6s]
Successfully downloaded 1 package in 8.6 seconds.

The following package(s) will be installed:
- survival [3.5-7]
These packages will be installed into "~/scratch/example/renv/library/R-3.6/x86_64-w64-mingw32".

Do you want to proceed? [Y/n]: 
# Installing packages --------------------------------------------------------
- Installing survival ...                       OK [installed binary and cached in 1.0s]
Successfully installed 1 package in 1.1 seconds.

Are you able to provide any more details? Can you also try executing:

options(renv.download.trace = TRUE)
renv::install("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")

[bzkrouse]

Sure, this is on a Windows machine, and here is the full trace:

> options(renv.download.trace = TRUE)
> renv::install("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")
# Downloading packages -------------------------------------------------------
- Downloading survival from P3M ...             
# Downloading 'https://packagemanager.posit.co/cran/2023-12-06/bin/windows/contrib/4.3/survival_3.5-7.zip' [curl] --------

## Request -------------------------------------------------------------------
location
fail
silent
show-error
user-agent = "renv (1.0.5.9000); RStudio Desktop (2023.12.1.402); R (4.3.2 x86_64-w64-mingw32 x86_64 mingw32)"
url = "https://packagemanager.posit.co/cran/2023-12-06/bin/windows/contrib/4.3/survival_3.5-7.zip"
output = "[root path]/renv/binary/R-4.3/x86_64-w64-mingw32/repository/survival/renv-tempfile-4f02db2488f"
connect-timeout = "20"
retry = "3"

## Output --------------------------------------------------------------------
curl: (22) The requested URL returned error: 403

## Status --------------------------------------------------------------------
22

	ERROR [error code 22]
Error: error downloading 'https://packagemanager.posit.co/cran/2023-12-06/bin/windows/contrib/4.3/survival_3.5-7.zip' [error code 22]
Traceback (most recent calls last):
8: renv::install("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")
7: retrieve(packages)
6: handler(package, renv_retrieve_impl(package))
5: renv_retrieve_impl(package)
4: renv_retrieve_repos(record)
3: renv_retrieve_repos_impl(record)
2: renv_retrieve_package(record, url, path)
1: stop(status)

[kevinushey]

The requested URL returned error: 403

That's odd -- the HTTP error code 403 (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403) would imply that P3M is refusing to accept the request for some reason.

Does this fail in a similar way, or does this succeed?

options(download.file.method = "curl")
options(download.file.extra = "-L -f")
utils::install.packages("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")

And also:

options(download.file.method = "libcurl")
utils::install.packages("survival", repos = "https://packagemanager.posit.co/cran/2023-11-19")

This might also be useful:

print(renv:::renv_curl_exe())
system(paste(renv:::renv_curl_exe(), "--version"))

[bzkrouse]

@kevinushey those methods both succeed.

> print(renv:::renv_curl_exe())
                             curl 
"C:\\windows\\SYSTEM32\\curl.exe" 
> system(paste(renv:::renv_curl_exe(), "--version"))
curl 8.4.0 (Windows) libcurl/8.4.0 Schannel WinIDN
Release-Date: 2023-10-11

[kevinushey]

Thanks! For reference, here's what I see...

> print(renv:::renv_curl_exe())
                             curl 
"C:\\WINDOWS\\SYSTEM32\\curl.exe" 
> system(paste(renv:::renv_curl_exe(), "--version"))
curl 8.5.0 (Windows) libcurl/8.5.0 Schannel WinIDN
Release-Date: 2023-12-06
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS HSTS HTTPS-proxy IDN IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI threadsafe Unicode UnixSockets
[1] 0

It's possible that the issue is related to the specific version of curl being used, but that feels kind of unlikely? It is very surprising to me that R's downloader here would work (even when using curl) while renv's would not, though.

Could you also share the output of renv::diagnostics()?

As a workaround, you should be able to set:

options(renv.download.override = utils::download.file)

in an appropriate R startup file.

Do you have any proxy set up by any chance? What is the output of:

envvars <- Sys.getenv()
envvars[grepl("_proxy", names(envvars), ignore.case = TRUE)]

[bzkrouse]

@kevinushey Sorry for the delay in responding! I appreciate you sharing a workaround for the issue!

Here is part of my renv::diagnostics report, omitting some paths.

# Session Info ---------------------------------------------------------------
R version 4.3.2 (2023-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8 
[2] LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.2  tools_4.3.2     renv_1.0.5.9000

# Project --------------------------------------------------------------------
Project path: "~/renvtest"

# Status ---------------------------------------------------------------------
No issues found -- the project is in a consistent state.

# Packages -------------------------------------------------------------------
              Library Source   Lockfile Source Path Dependency
KernSmooth    2.23-22   CRAN       <NA>   <NA>  [2]       <NA>
MASS           7.3-60   CRAN       <NA>   <NA>  [2]       <NA>
Matrix        1.6-1.1   CRAN       <NA>   <NA>  [2]       <NA>
boot         1.3-28.1   CRAN       <NA>   <NA>  [2]       <NA>
class          7.3-22   CRAN       <NA>   <NA>  [2]       <NA>
cluster         2.1.4   CRAN       <NA>   <NA>  [2]       <NA>
codetools      0.2-19   CRAN       <NA>   <NA>  [2]       <NA>
foreign        0.8-85   CRAN       <NA>   <NA>  [2]       <NA>
lattice        0.21-9   CRAN       <NA>   <NA>  [2]       <NA>
mgcv            1.9-0   CRAN       <NA>   <NA>  [2]       <NA>
nlme          3.1-163   CRAN       <NA>   <NA>  [2]       <NA>
nnet           7.3-19   CRAN       <NA>   <NA>  [2]       <NA>
renv       1.0.5.9000 GitHub 1.0.5.9000 GitHub  [1]     direct
rpart          4.1.21   CRAN       <NA>   <NA>  [2]       <NA>
spatial        7.3-17   CRAN       <NA>   <NA>  [2]       <NA>
survival        3.5-7   RSPM       <NA>   <NA>  [2]       <NA>
utils            <NA>   <NA>       <NA>   <NA>  [2]   indirect

# ABI ------------------------------------------------------------------------
- ABI conflict checks are not available on Windows.

# User Profile ---------------------------------------------------------------
[no user profile detected]

# Settings -------------------------------------------------------------------
List of 13
 $ bioconductor.version     : NULL
 $ external.libraries       : chr(0) 
 $ ignored.packages         : chr(0) 
 $ package.dependency.fields: chr [1:3] "Imports" "Depends" "LinkingTo"
 $ ppm.enabled              : NULL
 $ ppm.ignored.urls         : chr(0) 
 $ r.version                : NULL
 $ snapshot.type            : chr "implicit"
 $ use.cache                : logi TRUE
 $ vcs.ignore.cellar        : logi TRUE
 $ vcs.ignore.library       : logi TRUE
 $ vcs.ignore.local         : logi TRUE
 $ vcs.manage.ignores       : logi TRUE

# Options --------------------------------------------------------------------
List of 8
 $ defaultPackages                     : chr [1:6] "datasets" "utils" "grDevices" "graphics" ...
 $ download.file.method                : chr "libcurl"
 $ download.file.extra                 : NULL
 $ install.packages.compile.from.source: chr "interactive"
 $ pkgType                             : chr "both"
 $ repos                               : Named chr "https://cran.rstudio.com"
  ..- attr(*, "names")= chr "CRAN"
 $ renv.consent                        : logi TRUE
 $ renv.verbose                        : logi TRUE

The output of

envvars <- Sys.getenv()
envvars[grepl("_proxy", names(envvars), ignore.case = TRUE)]

does not return anything.

Please let me know if any other information would be helpful!

[bzkrouse]

@kevinushey I have another observation. If I do the following:

options(repos = c(CRAN = "https://packagemanager.posit.co/cran/2023-11-20"))
renv::init(bare = TRUE, restart = FALSE)
renv::install("survival")

it fails. But after a restart (or via renv::init(bare = TRUE, restart = TRUE)) the installation succeeds.
Are you able to reproduce this behavior?
For our use case of {renv} it is important that the installations can occur successfully prior to a restart.

Thanks again for your help!

[kevinushey]

Sorry for the late response!

For our use case of {renv} it is important that the installations can occur successfully prior to a restart.

Can you share a bit more about how you're using renv? Would it be feasible to ensure that renv::init() is called beforehand in a separate R process, to ensure the project is already initialized?

[bzkrouse]

Just wanted to say that I investigated further and the restart doesn't seem to be the solution. I have filed #1901 instead! Thanks for your ongoing support.