Skip to content
@spider-rs

spider-rs

Your friendly neighborhood spiderbot.

Popular repositories Loading

  1. spider spider Public

    A web crawler and scraper for Rust

    Rust 1.5k 125

  2. spider-py spider-py Public

    Spider ported to Python

    Rust 67 8

  3. spider-nodejs spider-nodejs Public

    Spider ported to Node.js

    Rust 40 7

  4. headless-browser headless-browser Public

    Headless browser - reliable high performance setup for cloud providers like Fargate, CloudRun, and more.

    Rust 33 5

  5. ua_generator ua_generator Public

    Pre-compiled random real User-Agents. Updated weekly for windows, mac, linux, and android

    Rust 27 3

  6. html2md html2md Public

    Fast HTML to markdown lib for Rust

    HTML 24 1

Repositories

Showing 10 of 21 repositories
  • headless-browser Public

    Headless browser - reliable high performance setup for cloud providers like Fargate, CloudRun, and more.

    spider-rs/headless-browser’s past year of commit activity
    Rust 33 MIT 5 0 0 Updated Feb 26, 2025
  • spider_firewall Public

    Firewall for spider 13kb (daily-updates)

    spider-rs/spider_firewall’s past year of commit activity
    Rust 0 MIT 0 0 0 Updated Feb 26, 2025
  • spider Public

    A web crawler and scraper for Rust

    spider-rs/spider’s past year of commit activity
    Rust 1,514 MIT 125 3 0 Updated Feb 26, 2025
  • ua_generator Public

    Pre-compiled random real User-Agents. Updated weekly for windows, mac, linux, and android

    spider-rs/ua_generator’s past year of commit activity
    Rust 27 MIT 3 0 0 Updated Feb 24, 2025
  • html2md Public

    Fast HTML to markdown lib for Rust

    spider-rs/html2md’s past year of commit activity
    HTML 24 MIT 1 0 0 Updated Feb 22, 2025
  • bad_websites Public

    websites that should be blocked via firewall

    spider-rs/bad_websites’s past year of commit activity
    0 0 0 0 Updated Feb 15, 2025
  • spider-nodejs Public

    Spider ported to Node.js

    spider-rs/spider-nodejs’s past year of commit activity
    Rust 40 MIT 7 0 0 Updated Jan 28, 2025
  • spider-py Public

    Spider ported to Python

    spider-rs/spider-py’s past year of commit activity
    Rust 67 MIT 8 0 0 Updated Jan 28, 2025
  • spider-clients Public

    Python, Javascript, and Rust libraries for the Spider Cloud API.

    spider-rs/spider-clients’s past year of commit activity
    Python 10 MIT 5 0 0 Updated Jan 21, 2025
  • spider-wasm Public

    The spider project ported to wasm [wip]

    spider-rs/spider-wasm’s past year of commit activity
    Rust 0 MIT 0 0 0 Updated Jan 6, 2025
View all repositories