Add services for managing the driver processes (#2)

Rather than requiring users of the library to start their own driver processes (like chromedriver), provide the ability to start them in the library.

This is largely a simplified copy of the ruby selenium library's implementation: https://github.com/SeleniumHQ/selenium/blob/e675f9bf66537930de9ebf2be2de44635821db22/rb/lib/selenium/webdriver/common/service.rb#L27

The only part that I know that I wasn't able to implement is the exit hook piece https://github.com/SeleniumHQ/selenium/blob/e675f9bf66537930de9ebf2be2de44635821db22/rb/lib/selenium/webdriver/common/service_manager.rb#L52
This is because the way the `at_exit` hook works causes problems with Spec. Spec uses an `at_exit` hook to run the tests so if this library had one that means it would always run before any of the tests ran. It will need to be very clear that the user _MUST_ call `driver.stop` in order to end the process.

.github/workflows/specs.yml

@@ -8,11 +8,25 @@ on:
 
 jobs:
   verify-chrome:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Setup Crystal
+      run: curl -sSL https://dist.crystal-lang.org/apt/setup.sh | sudo bash
+    - name: Install Crystal
+      run: sudo apt install crystal
+    - name: Install dependencies
+      run: shards install
+    - name: Run tests
+      run: crystal spec --tag "~chrome"
+      env:
+        SELENIUM_BROWSER: chrome
+  verify-chrome-external-process:
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v2
     - name: Start Chromedriver
-      run: chromedriver --port=4444 --url-base=/wd/hub &
+      run: chromedriver &
     - name: Setup Crystal
       run: curl -sSL https://dist.crystal-lang.org/apt/setup.sh | sudo bash
     - name: Install Crystal
@@ -27,8 +41,6 @@ jobs:
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v2
-    - name: Start Geckodriver
-      run: geckodriver --port=4444 &
     - name: Setup Crystal
       run: curl -sSL https://dist.crystal-lang.org/apt/setup.sh | sudo bash
     - name: Install Crystal

README.md

@@ -25,7 +25,7 @@ require "selenium"
 ### Creating a Driver
 
 ```crystal
-driver = Selenium::Driver.for(:chrome)
+driver = Selenium::Driver.for(:chrome, base_url: "http://localhost:9515")
 ```
 
 Available drivers:
@@ -34,6 +34,17 @@ Available drivers:
 - :firefox (using geckodriver)
 - :remote (general purpose)
 
+### Running with a service
+
+Rather than running chromedriver yourself, you can give the driver a service which will run the process for you.
+
+```crystal
+driver = Selenium::Driver.for(:chrome, service: Service.chrome(driver_path: "~/.webdrivers/chromedriver"))
+```
+
+You must call `driver.stop` when you are finished or it will leave the service running.
+Consider using [webdrivers.cr](https://github.com/matthewmcgarvey/webdrivers.cr) for automatically installing drivers and managing the driver path for you.
+
 ### Creating a Session
 
 ```crystal
@@ -46,18 +57,23 @@ Use the appropriate `Capabilities` class for whichever browser you choose.
 
 ## Development
 
-To run the tests you must have the appropriate driver running.
-The tests use chrome by default. The command to start chromedriver as the tests expect is:
+Run `crystal spec` to run the tests. It will run the tests in headless mode.
 
-```bash
-chromedriver --port=4444 --url-base=/wd/hub
+To run the tests with chrome headlessly:
+
+```crystal
+SELENIUM_BROWSER=chrome crystal spec --tag "~chrome"
 ```
 
-Run `crystal spec` to run the tests. It will run the tests in headless mode.
+To run the tests with firefox headlessly:
 
-To run the tests with firefox you will need to have the geckodriver running and run `SELENIUM_BROWSER=firefox crystal spec --tag "~firefox"`
+```crystal
+SELENIUM_BROWSER=firefox crystal spec --tag "~firefox"
+
+```
 
-Using the tag `~firefox` is to avoid running the tests that are known to break the specified browser. Please feel free to attemp a fix for them.
+The tag skips any specs that are know to break with those browsers.
+Running just `crystal spec` will use chrome.
 
 ## Contributing
 

spec/spec_helper.cr

@@ -1,8 +1,30 @@
 require "spec"
 require "http"
+require "webdrivers"
 require "../src/selenium"
 require "./support/**"
 
+class Global
+  @@driver : Selenium::Driver?
+  @@cap : Selenium::Capabilities?
+
+  def self.set(driver, cap)
+    @@driver = driver
+    @@cap = cap
+  end
+
+  def self.create_session
+    @@driver.not_nil!.create_session(@@cap)
+  end
+
+  def self.stop
+    @@driver.not_nil!.stop
+  end
+end
+
+driver, capabilities = Selenium::TestDriverFactory.build(ENV["SELENIUM_BROWSER"]? || "chrome")
+Global.set(driver, capabilities)
+
 server = TestServer.new(3002)
 
 Spec.before_each do
@@ -11,34 +33,15 @@ end
 
 Spec.after_suite do
   server.close
+  Global.stop
 end
 
 spawn do
   server.listen
 end
 
-def browser
-  ENV["SELENIUM_BROWSER"]? || "chrome"
-end
-
-def build_session
-  if browser == "chrome"
-    driver = Selenium::Driver.for(:chrome)
-    capabilities = Selenium::Chrome::Capabilities.new
-    capabilities.args(["no-sandbox", "headless", "disable-gpu"])
-    driver.create_session(capabilities)
-  elsif browser == "firefox"
-    driver = Selenium::Driver.for(:firefox, base_url: "http://localhost:4444")
-    capabilities = Selenium::Firefox::Capabilities.new
-    capabilities.args(["-headless"])
-    driver.create_session(capabilities)
-  else
-    raise ArgumentError.new("unknown browser for running tests: #{browser}")
-  end
-end
-
 def with_session
-  session = build_session
+  session = Global.create_session
   yield(session)
 ensure
   session.delete unless session.nil?

spec/support/test_driver_factory.cr

@@ -0,0 +1,43 @@
+class Selenium::TestDriverFactory
+  def self.build(browser) : Tuple(Driver, Capabilities)
+    case browser
+    when "chrome"
+      build_chrome_driver
+    when "firefox"
+      build_firefox_driver
+    when "chrome-no-service"
+      build_chrome_driver_no_service
+    else
+      raise ArgumentError.new("unknown browser for running tests: #{browser}")
+    end
+  end
+
+  def self.build_chrome_driver : Tuple(Driver, Capabilities)
+    capabilities = Chrome::Capabilities.new
+    capabilities.args(["no-sandbox", "headless", "disable-gpu"])
+    driver = Driver.for(:chrome, service: Service.chrome(driver_path: Webdrivers::Chromedriver.install))
+    {driver, capabilities}
+  end
+
+  def self.build_firefox_driver : Tuple(Driver, Capabilities)
+    capabilities = Firefox::Capabilities.new
+    capabilities.args(["-headless"])
+    driver = Driver.for(:firefox, service: Service.firefox(driver_path: Webdrivers::Geckodriver.install))
+    {driver, capabilities}
+  end
+
+  def self.build_chrome_driver_no_service : Tuple(Driver, Capabilities)
+    capabilities = Chrome::Capabilities.new
+    capabilities.args(["no-sandbox", "headless", "disable-gpu"])
+    driver = Driver.for(:chrome, base_url: "http://localhost:9515")
+    {driver, capabilities}
+  end
+
+  private def self.chrome?(browser)
+    browser == "chrome"
+  end
+
+  private def self.firefox?(browser)
+    browser == "firefox"
+  end
+end

src/selenium/chrome/service.cr

@@ -0,0 +1,5 @@
+class Selenium::Chrome::Service < Selenium::Service
+  def default_port : Int32
+    9515
+  end
+end

src/selenium/driver.cr

@@ -1,27 +1,25 @@
 class Selenium::Driver
-  DEFAULT_CONFIGURATION = {
-    base_url: "http://localhost:4444/wd/hub",
-  }
-
   def self.for(browser, **opts)
-    options = DEFAULT_CONFIGURATION.merge(opts)
     case browser
     when :chrome
-      Chrome::Driver.new(options)
+      Chrome::Driver.new(**opts)
     when :firefox, :gecko
-      Firefox::Driver.new(options)
+      Firefox::Driver.new(**opts)
     when :remote
-      Remote::Driver.new(options)
+      Remote::Driver.new(**opts)
     else
       raise ArgumentError.new "unknown driver: #{browser}"
     end
   end
 
   getter http_client : HttpClient
   getter command_handler : CommandHandler
+  getter service : Service?
 
-  def initialize(opts)
-    @http_client = HttpClient.new(base_url: opts[:base_url])
+  def initialize(base_url : String? = nil, @service : Service? = nil)
+    @service.try &.start
+    base_url ||= @service.not_nil!.base_url
+    @http_client = HttpClient.new(base_url: base_url)
     @command_handler = CommandHandler.new(@http_client)
   end
 
@@ -37,4 +35,8 @@ class Selenium::Driver
 
     Status.from_json(data["value"].to_json)
   end
+
+  def stop
+    service.try &.stop
+  end
 end

src/selenium/firefox/service.cr

@@ -0,0 +1,9 @@
+class Selenium::Firefox::Service < Selenium::Service
+  def default_port : Int32
+    4444
+  end
+
+  def stop
+    stop_process(process)
+  end
+end

src/selenium/service.cr

@@ -0,0 +1,102 @@
+abstract class Selenium::Service
+  CONNECTION_INTERVAL = 1.seconds
+  CONNECTION_TIMEOUT  = 5.seconds
+
+  def self.chrome(**opts)
+    Chrome::Service.new(**opts)
+  end
+
+  def self.firefox(**opts)
+    Firefox::Service.new(**opts)
+  end
+
+  private property process : Process?
+
+  def initialize(@driver_path : String, @port : Int32 = default_port, @args = [] of String)
+  end
+
+  def start
+    start_process
+    verify_running
+  end
+
+  def stop
+    send_shutdown_command
+    process.try &.wait
+  ensure
+    stop_process(process)
+  end
+
+  def base_url : String
+    "http://localhost:#{@port}"
+  end
+
+  abstract def default_port : Int32
+
+  private def start_process
+    @process = Process.new(
+      @driver_path,
+      ["--port=#{@port}"] | @args,
+      shell: spawn_in_shell?,
+      output: {% if flag?(:DEBUG) %} STDOUT {% else %} Process::Redirect::Close {% end %},
+      error: {% if flag?(:DEBUG) %} STDERR {% else %} Process::Redirect::Close {% end %}
+    )
+  end
+
+  private def verify_running
+    result = with_timeout { listening? }
+    raise "Unable to connect to driver process. Try running in DEBUG mode to find more information." unless result
+  end
+
+  private def with_timeout
+    max_time = Time.utc + CONNECTION_TIMEOUT
+
+    until Time.utc > max_time
+      return true if yield
+
+      sleep CONNECTION_INTERVAL
+    end
+
+    false
+  end
+
+  private def listening?
+    TCPSocket.new("localhost", @port).close
+    true
+  rescue
+    false
+  end
+
+  private def send_shutdown_command
+    return if @process.nil? || @process.try &.terminated?
+
+    HTTP::Client.new(host: "localhost", port: @port) do |client|
+      client.connect_timeout = 10
+      client.read_timeout = 10
+      headers = HTTP::Headers{
+        "Accept"       => "application/json",
+        "Content-Type" => "application/json; charset=UTF-8",
+        "User-Agent"   => "selenium/#{Selenium::VERSION} (crystal #{os})",
+      }
+      client.get("/shutdown", headers)
+    end
+  end
+
+  private def stop_process(process)
+    return if process.nil? || process.terminated?
+
+    process.kill
+  end
+
+  private def spawn_in_shell?
+    os != "linux"
+  end
+
+  private def os
+    {% if flag?(:linux) %}
+      "linux"
+    {% else %}
+      "macos"
+    {% end %}
+  end
+end