Skip to content

promptapi/scraper-cli

Repository files navigation

Version Go Documentation Go Report Card Build Status

Prompt API - Scraper - Command Line Interface

Do you like Scraper API? Don’t you like to access this beauty from your terminal?


Requirements

  1. You need to # for Prompt API
  2. You need to subscribe scraper api, test drive is free!!!
  3. You need to set PROMPTAPI_TOKEN environment variable after subscription.

If you have golang environment installed, you can fetch via

$ go get -u github.com/promptapi/scraper-cli/cmd/scraper-cli

Or you can download binaries from releases page


Usage

$ scraper-cli -h

usage: scraper-cli [flags...]

  scraper-cli is a command-line interface for Prompt API's Scraper API. Details
  can be found:

      https://promptapi.com/marketplace/description/scraper-api

  you need to # for Prompt API to get your PROMPTAPI_TOKEN. you can #
  from:

      https://promptapi.com/##-form

  application looks for PROMPTAPI_TOKEN environment variable. if you pass 
  "token" flag, this will override environment variable lookup.

  required flag(s):

  -url          web url/address to scrape


  optional flags:

  -country      2 character country code
  -token        promptapi apikey instead of PROMPTAPI_TOKEN env-var
  -username     for HTTP Realm auth username
  -password     for HTTP Realm auth password
  -cookie       URL Encoded cookie header
  -referer      HTTP referer header
  -selector     CSS style selector path such as: a.btn div li
  -version      display version information
  -header       request header(s)
  -help, -h     display help


  examples:

  $ scraper-cli -help
  $ scraper-cli -url "https://promptapi.com"
  $ scraper-cli -url "https://promptapi.com" -country "EE"
  $ scraper-cli -url "https://promptapi.com" -country "EE" -selector "a.btn div li"

  $ PROMPTAPI_TOKEN="your-api-key" scraper-cli -url "https://promptapi.com"
  $ scraper-cli -url "https://promptapi.com" -token "your-api-key"
  $ scraper-cli -url "https://promptapi.com" -token "your-api-key" -header "X-Referer: https://www.google.com"
  $ scraper-cli -url "https://promptapi.com" -token "your-api-key" -header "X-Referer: https://www.google.com" -header "X-Custom-Header: Hello"

License

This project is licensed under MIT


Contributer(s)


Contribute

All PR’s are welcome!

  1. fork (https://github.com/promptapi/scraper-cli/fork)
  2. Create your branch (git checkout -b my-feature)
  3. commit yours (git commit -am 'Add awesome features...')
  4. push your branch (git push origin my-feature)
  5. Than create a new Pull Request!

This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.