Provide wrapper around chromote

[hadley]

Which implements https://chromedevtools.github.io/devtools-protocol/tot/Input

[hadley]

Now on CRAN: https://cran.r-project.org/web/packages/chromote/

[hadley]

library(chromote)

b <- ChromoteSession$new()
b$Page$navigate("https://www.r-project.org/")
b$view()

x <- b$DOM$getDocument()
p <- b$DOM$querySelector(x$root$nodeId, "p")
b$DOM$getOuterHTML(p$nodeId)

[hadley]

And need to implement interface similar to https://github.com/rstudio/shinytest/blob/main/R/shiny-driver.R so you can "click" and set values

[hadley]

• return html (instead of promise)
• new R6 chromote object
• html wait for element to appear
• implement click and double click
• implement scroll
• type text (https://playwright.dev/docs/api/class-locator#locator-type)
• press keys (https://playwright.dev/docs/api/class-locator#locator-press)
• figure direct xpath out

[hadley]

Sample problems:

# https://twitter.com/ryanburge/status/1592910141128867841 --------------------

url <- "https://www.christianitytoday.com/ct/search/?query=abortion"
ct <- read_html_live(url)

items <- ct %>% html_elements(".ais-Hits-item")
items %>% html_element(".title-2") %>% html_text2()
items %>% html_element(".d-inline-block") %>% html_text2()

# https://twitter.com/minebocek/status/1592917026049183745 --------------------

url <- "https://www.forbes.com/top-colleges/"
sess <- read_html_live(url)
sess |> 
  html_elements(".TopColleges2023_tableRowGroup__65RHn") |> 
  html_elements(".TopColleges2023_tableRow__BYOSU")

# https://twitter.com/joshmccrain/status/1592910563327496199 -------------------
# http://joshuamccrain.com/tutorials/web_scraping_R_selenium.html

sess <- read_html_live("https://www.fcc.gov/media/engineering/dtvmaps")
sess$view()

sess$type("#startpoint", "77003")
sess$press("#startpoint", "Enter")

sess |> 
  html_table() |> 
  html_element("#contourdata > table:nth-child(3)") |> 
  select(-1) |> 
  filter(Callsign != "")

sess$press("#startpoint", "ArrowLeft", modifiers = c("Alt", "Shift"))
sess$press("#startpoint", "Delete")

# https://twitter.com/jdsdog10/status/1592912450625884160 ---------------------

url <- "https://www.fantasypros.com/nfl/rankings/half-point-ppr-rb.php"
sess <- read_html_live(url)
sess$view()
sess |> html_element("#ranking-table") |> html_table()

# https://twitter.com/sctyner/status/1592921110730797057 -----------------------
# need to click "load more" a few times

sess <- read_html_live("https://www.bodybuilding.com/exercises/finder")
sess |> html_elements(".ExResult-row") |> length()
sess$click(".ExLoadMore-btn")
sess |> html_elements(".ExResult-row") |> length()
sess$click(".ExLoadMore-btn")
sess |> html_elements(".ExResult-row") |> length()