Skip to content

codesplicer/crawler

Repository files navigation

Web Crawler

Crawls a site and produces a sitemap of internal links

Usage

make build # Uses godep to install all dependencies to vendor/ ./bin/crawler https://monzo.com

Testing

make test

Author

About

Concurrent sitecrawler implemented in Go

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published