Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

renv is having issues with remote packages #2040

Closed
danielvartan opened this issue Nov 21, 2024 · 20 comments
Closed

renv is having issues with remote packages #2040

danielvartan opened this issue Nov 21, 2024 · 20 comments

Comments

@danielvartan
Copy link

Hi there,

It seems that renv encounters issues when attempting to update remote packages from GitHub using renv::update(). The problem arises because renv is looking for a master branch, while many repositories today use main as the default branch. The process should not depend on the naming convention of the main branch.

Here is the error message I encountered:

#> - Checking for updated packages ... Done!
#> 
#> One or more errors occurred while finding updates for the following GitHub packages:
#> - prettycheck: error downloading 'https://api.github.com/repos/danielvartan/prettycheck/commits/master' [error code 22]
#> - rutils: error downloading 'https://api.github.com/repos/danielvartan/rutils/commits/master' [error code 22]
#> Ensure that these packages were installed from an accessible GitHub remote.

The correct URLs should be:

  • https://api.github.com/repos/danielvartan/prettycheck/commits/main
  • https://api.github.com/repos/danielvartan/rutils/commits/main

renv should automatically adapt to the default branch, whether it is main, master, or otherwise.

Thank you for looking into this!

@kevinushey
Copy link
Collaborator

Can you share the output of packageDescription("prettycheck")? Did you recently change the default branch for these packages from 'master' to 'main'? I'm wondering if the currently-installed versions of those packages are still tracking 'master', which would show up with:

RemoteRef: master

in the package's DESCRIPTION file.

@danielvartan
Copy link
Author

Hi @kevinushey,

No, this package was built from the start using a main branch. You can find both packages and the DESCRIPTION file on GitHub at the following links:

https://github.com/danielvartan/prettycheck/

https://github.com/danielvartan/rutils

@danielvartan
Copy link
Author

However, I would add that this issue is recurrent with several other remote packages, not just mine. It’s worth looking into.

@kevinushey
Copy link
Collaborator

Can you please share the output of packageDescription("prettycheck")?

@kevinushey
Copy link
Collaborator

The following works for me in an renv project. First, install an older version of the prettycheck package:

renv::install("danielvartan/prettycheck@517e473c1ffeccac6184b20639c8429a317dff60")

Then, modify the installed package's DESCRIPTION file, so that RemoteRef reads 'main':

path <- system.file("DESCRIPTION", package = "prettycheck")
desc <- readLines(path)
line <- grep("^RemoteRef:", desc)
desc[line] <- "RemoteRef: main"
writeLines(desc, con = path)

Then update it with:

renv::update(packages = "prettycheck")

Here's the full installation log I see.

> renv::install("danielvartan/prettycheck@517e473c1ffeccac6184b20639c8429a317dff60")
# Downloading packages -------------------------------------------------------
- Downloading prettycheck from GitHub ...       OK [31.1 Kb in 0.34s]
Successfully downloaded 1 package in 2.8 seconds.

The following package(s) will be installed:
- prettycheck [danielvartan/prettycheck@517e473c1ffeccac6184b20639c8429a317dff60]
These packages will be installed into "~/scratch/example/renv/library/macos/R-4.4/aarch64-apple-darwin20".

Do you want to proceed? [Y/n]: 
# Installing packages --------------------------------------------------------
- Installing prettycheck ...                    OK [built from source and cached in 1.6s]
Successfully installed 1 package in 1.7 seconds.
> path <- system.file("DESCRIPTION", package = "prettycheck")
> desc <- readLines(path)
> line <- grep("^RemoteRef:", desc)
> desc[line] <- "RemoteRef: main"
> writeLines(desc, con = path)
> renv::update(packages = "prettycheck")
- Checking for updated packages ... Done!
The following package(s) will be updated:

# GitHub ---------------------------------------------------------------------
- prettycheck   [danielvartan/prettycheck@main: 517e473c -> dbe83404]

Do you want to proceed? [Y/n]: y

# Downloading packages -------------------------------------------------------
- Downloading prettycheck from GitHub ...       OK [31.1 Kb in 0.36s]
Successfully downloaded 1 package in 2.4 seconds.

The following package(s) will be installed:
- prettycheck [danielvartan/prettycheck]
These packages will be installed into "~/scratch/example/renv/library/macos/R-4.4/aarch64-apple-darwin20".

Do you want to proceed? [Y/n]: y
# Installing packages --------------------------------------------------------
- Installing prettycheck ...                    OK [built from source and cached in 1.6s]
Successfully installed 1 package in 1.6 seconds.

@danielvartan
Copy link
Author

danielvartan commented Nov 22, 2024

Can you please share the output of packageDescription("prettycheck")?

Here you go.

packageDescription("prettycheck")
#> Package: prettycheck
#> Title: Pretty assertive programming
#> Version: 0.0.0.9000
#> Authors@R: c( person( given = "Daniel", family = "Vartanian", role =
#>         c("aut", "cre", "ccp", "cph"), email = "danvartan@gmail.com",
#>         comment = c(ORCID = "0000-0001-7782-759X") ) )
#> Description: Provides a set of functions that produce pretty and
#>         informative error messages.
#> License: MIT + file LICENSE
#> URL: https://github.com/danielvartan/prettycheck/
#> BugReports: https://github.com/danielvartan/prettycheck/issues/
#> Depends: R (>= 4.3)
#> Imports: checkmate (>= 2.3.2), cli (>= 3.6.3), curl (>= 5.2.3),
#>         data.table (>= 1.16.2), fastmatch (>= 1.1.4), glue (>= 1.8.0),
#>         hms (>= 1.1.3), lifecycle (>= 1.0.4), lubridate (>= 1.9.3),
#>         magrittr (>= 2.0.3), rlang (>= 1.1.4), rutils, stringr (>=
#>         1.5.1), tibble (>= 3.2.1), utils (>= 4.3)
#> Suggests: covr (>= 3.6.4), knitr (>= 1.48), mockr (>= 0.2.1), spelling
#>         (>= 2.3.1), testthat (>= 3.2.1.1)
#> Remotes: github::danielvartan/rutils
#> Config/testthat/edition: 3
#> Encoding: UTF-8
#> Language: en-US
#> Roxygen: list(markdown = TRUE)
#> RoxygenNote: 7.3.2
#> Collate: 'make_check.R' 'a_collate.R' 'access.R' 'any.R' 'array.R'
#>         .....
#> NeedsCompilation: no
#> Packaged: 2024-10-18 03:02:46 UTC; Daniel
#> Author: Daniel Vartanian [aut, cre, ccp, cph]
#>         (<https://orcid.org/0000-0001-7782-759X>)
#> Maintainer: Daniel Vartanian <danvartan@gmail.com>
#> Built: R 4.4.1; ; 2024-10-18 03:02:47 UTC; windows
#> 
#> -- File: C:/Users/Daniel/AppData/Local/R/win-library/4.4/prettycheck/Meta/package.rds

Created on 2024-11-22 with reprex v2.1.1

@danielvartan
Copy link
Author

Hi @kevinushey,

I don't see how adding RemoteRef: main to the package's DESCRIPTION file would make a difference. This package has never had a master branch, so referencing main would effectively behave the same as leaving that field absent.

@kevinushey
Copy link
Collaborator

Here's what I see for my installation.

> packageDescription("prettycheck")
Package: prettycheck
Title: Pretty assertive programming
Version: 0.0.0.9000
Authors@R: c( person( given = "Daniel", family = "Vartanian", role = c("aut", "cre", "ccp",
           "cph"), email = "danvartan@gmail.com", comment = c(ORCID = "0000-0001-7782-759X") ) )
Description: Provides a set of functions that produce pretty and informative error messages.
License: MIT + file LICENSE
URL: https://github.com/danielvartan/prettycheck/
BugReports: https://github.com/danielvartan/prettycheck/issues/
Depends: R (>= 4.3)
Imports: checkmate (>= 2.3.2), cli (>= 3.6.3), curl (>= 5.2.3), data.table (>= 1.16.2), fastmatch
           (>= 1.1.4), glue (>= 1.8.0), hms (>= 1.1.3), lifecycle (>= 1.0.4), lubridate (>=
           1.9.3), magrittr (>= 2.0.3), rlang (>= 1.1.4), rutils, stringr (>= 1.5.1), tibble (>=
           3.2.1), utils (>= 4.3)
Suggests: covr (>= 3.6.4), knitr (>= 1.48), mockr (>= 0.2.1), spelling (>= 2.3.1), testthat (>=
           3.2.1.1)
Config/testthat/edition: 3
Encoding: UTF-8
Language: en-US
LazyData: true
LazyDataCompression: bzip2
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.3.2
Collate: 'make_check.R' 'a_collate.R' 'access.R' 'any.R' 'array.R' 'atomic.R' 'boolean.R' .....
Author: Daniel Vartanian [aut, cre, ccp, cph] (<https://orcid.org/0000-0001-7782-759X>)
Maintainer: Daniel Vartanian <danvartan@gmail.com>
Built: R 4.4.1; ; 2024-11-21 03:58:52 UTC; unix
RemoteType: github
RemoteHost: api.github.com
RemoteUsername: danielvartan
RemoteRepo: prettycheck
RemoteRef: main
RemoteSha: dbe83404007cc8c0874092c8e9fc73dba5368163
GithubHost: api.github.com
GithubRepo: prettycheck
GithubUsername: danielvartan
GithubRef: main
GithubSHA1: dbe83404007cc8c0874092c8e9fc73dba5368163
Remotes: github::danielvartan/rutils

-- File: /Users/kevin/Library/R/arm64/4.4/library/prettycheck/Meta/package.rds 

@danielvartan
Copy link
Author

I see you are using a Mac, maybe this is a Windows issue. Below is my sessioninfo().

I also updated my packageDescription("prettycheck") response above using the reprex package outside a renv project.

sessionInfo()
#> R version 4.4.1 (2024-06-14 ucrt)
#> Platform: x86_64-w64-mingw32/x64
#> Running under: Windows 11 x64 (build 26100)
#> 
#> Matrix products: default
#> 
#> 
#> locale:
#> [1] LC_COLLATE=Portuguese_Brazil.utf8  LC_CTYPE=Portuguese_Brazil.utf8   
#> [3] LC_MONETARY=Portuguese_Brazil.utf8 LC_NUMERIC=C                      
#> [5] LC_TIME=Portuguese_Brazil.utf8    
#> 
#> time zone: America/Sao_Paulo
#> tzcode source: internal
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] digest_0.6.37     fastmap_1.2.0     xfun_0.48         glue_1.8.0       
#>  [5] knitr_1.48        htmltools_0.5.8.1 rmarkdown_2.28    lifecycle_1.0.4  
#>  [9] cli_3.6.3         reprex_2.1.1      withr_3.0.1       compiler_4.4.1   
#> [13] rstudioapi_0.16.0 tools_4.4.1       evaluate_1.0.1    yaml_2.3.10      
#> [17] rlang_1.1.4       fs_1.6.4

Created on 2024-11-22 with reprex v2.1.1

@danielvartan
Copy link
Author

I think the ideal way to test this issue is to install a GitHub package with main as its default branch in a project that uses renv, preferably on a Windows machine. Afterward, make a simple commit to the package and attempt to update it using renv::update().

It seems that renv is incorrectly defaulting to a master branch when managing remote packages.

@danielvartan
Copy link
Author

Additionally, repeat the steps above, then delete the renv library directory (./renv/library) and attempt to restore the environment using renv::restore().

@kevinushey
Copy link
Collaborator

I see something similar on Windows, though.

renv::install("danielvartan/prettycheck")
packageDescription("prettycheck")

gives me

> renv::install("danielvartan/prettycheck")
- GitHub authentication credentials are not available.
- Please set GITHUB_PAT, or ensure the 'gitcreds' package is installed.
- See https://usethis.r-lib.org/articles/git-credentials.html for more details.
# Downloading packages -------------------------------------------------------
- Downloading prettycheck from GitHub ...       OK [31.1 Kb in 0.44s]
- Downloading checkmate from CRAN ...           OK [735 Kb in 0.12s]
- Downloading data.table from CRAN ...          OK [2.5 Mb in 0.11s]
- Downloading fastmatch from CRAN ...           OK [40 Kb in 0.23s]
- Downloading rutils from GitHub ...            OK [39.9 Kb in 0.44s]
- Downloading here from CRAN ...                OK [63.5 Kb in 0.11s]
- Downloading quarto from CRAN ...              OK [149.4 Kb in 0.23s]
- Downloading rmarkdown from CRAN ...           OK [2.6 Mb in 0.12s]
- Downloading knitr from CRAN ...               OK [1.1 Mb in 0.12s]
- Downloading rstudioapi from CRAN ...          OK [337.7 Kb in 0.12s]
- Downloading qpdf from CRAN ...                OK [1 Mb in 0.12s]
- Downloading rbbt from GitHub ...              OK [16.9 Kb in 0.45s]
- Downloading progress from CRAN ...            OK [86.6 Kb in 0.11s]
- Downloading spelling from CRAN ...            OK [56.2 Kb in 0.22s]
- Downloading hunspell from CRAN ...            OK [1.4 Mb in 0.23s]
Successfully downloaded 15 packages in 10 seconds.

The following package(s) will be installed:
- checkmate   [2.3.2]
- data.table  [1.16.2]
- fastmatch   [1.1-4]
- here        [1.0.1]
- hunspell    [3.0.5]
- knitr       [1.49]
- prettycheck [danielvartan/prettycheck]
- progress    [1.2.3]
- qpdf        [1.3.4]
- quarto      [1.4.4]
- rbbt        [paleolimbot/rbbt]
- rmarkdown   [2.29]
- rstudioapi  [0.17.1]
- rutils      [danielvartan/rutils]
- spelling    [2.3.1]
These packages will be installed into "~/AppData/Local/R/win-library/4.4".

Do you want to proceed? [Y/n]: 
# Installing packages --------------------------------------------------------
- Installing checkmate ...                      OK [installed binary and cached in 0.32s]
- Installing data.table ...                     OK [installed binary and cached in 0.66s]
- Installing fastmatch ...                      OK [installed binary and cached in 0.27s]
- Installing here ...                           OK [installed binary and cached in 0.31s]
- Installing knitr ...                          OK [installed binary and cached in 0.68s]
- Installing rmarkdown ...                      OK [installed binary and cached in 0.94s]
- Installing rstudioapi ...                     OK [installed binary and cached in 0.32s]
- Installing quarto ...                         OK [installed binary and cached in 0.31s]
- Installing qpdf ...                           OK [installed binary and cached in 0.28s]
- Installing progress ...                       OK [installed binary and cached in 0.27s]
- Installing rbbt ...                           OK [built from source and cached in 1.9s]
- Installing hunspell ...                       OK [installed binary and cached in 0.32s]
- Installing spelling ...                       OK [installed binary and cached in 0.27s]
- Installing rutils ...                         OK [built from source and cached in 4.5s]
- Installing prettycheck ...                    OK [built from source and cached in 4.4s]
Successfully installed 15 packages in 21 seconds.

The following loaded package(s) have been updated:
- rstudioapi
Restart your R session to use the new versions.

> packageDescription("prettycheck")
Package: prettycheck
Title: Pretty assertive programming
Version: 0.0.0.9000
Authors@R: c( person( given = "Daniel", family = "Vartanian", role = c("aut", "cre", "ccp", "cph"),
           email = "danvartan@gmail.com", comment = c(ORCID = "0000-0001-7782-759X") ) )
Description: Provides a set of functions that produce pretty and informative error messages.
License: MIT + file LICENSE
URL: https://github.com/danielvartan/prettycheck/
BugReports: https://github.com/danielvartan/prettycheck/issues/
Depends: R (>= 4.3)
Imports: checkmate (>= 2.3.2), cli (>= 3.6.3), curl (>= 5.2.3), data.table (>= 1.16.2), fastmatch (>=
           1.1.4), glue (>= 1.8.0), hms (>= 1.1.3), lifecycle (>= 1.0.4), lubridate (>= 1.9.3),
           magrittr (>= 2.0.3), rlang (>= 1.1.4), rutils, stringr (>= 1.5.1), tibble (>= 3.2.1), utils
           (>= 4.3)
Suggests: covr (>= 3.6.4), knitr (>= 1.48), mockr (>= 0.2.1), spelling (>= 2.3.1), testthat (>= 3.2.1.1)
Config/testthat/edition: 3
Encoding: UTF-8
Language: en-US
LazyData: true
LazyDataCompression: bzip2
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.3.2
Collate: 'make_check.R' 'a_collate.R' 'access.R' 'any.R' 'array.R' 'atomic.R' 'boolean.R' .....
Author: Daniel Vartanian [aut, cre, ccp, cph] (<https://orcid.org/0000-0001-7782-759X>)
Maintainer: Daniel Vartanian <danvartan@gmail.com>
Built: R 4.4.1; ; 2024-11-22 22:21:25 UTC; windows
RemoteType: github
RemoteHost: api.github.com
RemoteUsername: danielvartan
RemoteRepo: prettycheck
RemoteRef: main
RemoteSha: dbe83404007cc8c0874092c8e9fc73dba5368163
GithubHost: api.github.com
GithubRepo: prettycheck
GithubUsername: danielvartan
GithubRef: main
GithubSHA1: dbe83404007cc8c0874092c8e9fc73dba5368163
Remotes: github::danielvartan/rutils

-- File: C:/Users/kevin/AppData/Local/R/win-library/4.4/prettycheck/Meta/package.rds 

Again, please note that RemoteRef: main has been written into the installed package's DESCRIPTION file. How are you installing your packages?

@danielvartan
Copy link
Author

Again, please note that RemoteRef: main has been written into the installed package's DESCRIPTION file. How are you installing your packages?

For remote packages, I use the remotes package ― remotes::install_github("danielvartan/prettycheck").

@danielvartan
Copy link
Author

As I said before, I still don't understand how adding RemoteRef: main in the DESCRIPTION file would make a difference, since this package already uses (since the start) main in its default branch.

@kevinushey
Copy link
Collaborator

Thanks, that helps me understand the issue.

renv uses the RemoteRef field when updating packages. The intention here being, you might have installed some packages tracking specific branches (e.g. develop; dev); that information gets encoded into the package's DESCRIPTION file and re-used when renv::update() is called.

Unfortunately, the remotes package just encoded the ref as HEAD, which isn't particularly useful. In this scenario, it would make sense for renv to ask the repository what the default branch name is when renv::update() is invoked.

@danielvartan
Copy link
Author

Ah, I see – makes sense now.

I hope this issue helps with the implementation.

Thank you for your assistance, @kevinushey!

@stibu81
Copy link

stibu81 commented Dec 9, 2024

I just wanted to point out that the issue also exists on Ubuntu. I have this problem with a package of my own and renv itself (which I installed from GitHub):

renv::update()
#> - Checking for updated packages ... Done!
#> 
#> One or more errors occurred while finding updates for the following GitHub packages:
#> - renv: error downloading 'https://api.github.com/repos/rstudio/renv/commits/master' [error code 22]
#> - simpleTaxonomy: error downloading 'https://api.github.com/repos/stibu81/simpleTaxonomy/commits/master' [error code 22]
#> Ensure that these packages were installed from an accessible GitHub remote.
#> 
#> - All packages appear to be up-to-date.

The packages from GitHub have been installed with pak::pak(). I have added RemoteRef: main to my DESCRIPTION file (without committing it), but this has not helped.

If I install my package with renv::install("stibu81/simpleTaxonomy"), the problem goes away.

If there is anything else I could try out that would be helpful, please let me know.

@kevinushey
Copy link
Collaborator

Thanks. I believe I understand the issue now; it's related to pak (and other tools) writing out

RemoteRef: HEAD

when installing packages in this way, and renv not translating that to the proper default remote. A workaround would be to install the package with pak::pkg_install("stibu81/simpleTaxonomy@main").

@stibu81
Copy link

stibu81 commented Dec 10, 2024

Explicitly specifying the branch when installing with pak did not help. But the new commit seems to solve the problem. Thanks!

@kevinushey
Copy link
Collaborator

Phew, glad to hear it. Thanks for testing and reporting back!

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants