Skip to content

BastienF/ScalaCrawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ScalaCrawler

Scala implementation of basic HTTP crawler

sbt assembly
java -jar -DretryNumber=<numberOfRetryOnFail=5> -Ddepth=<linkFollowedDepth=6> -DstartUrl=<startingCrawlURL=https://en.wikipedia.org/wiki/Scala_(programming_language)> -Dhosts=<hostsToCrawl=https://en.wikipedia.org,https://fr.wikipedia.org> -DcsvResultPath=./result.csv [-DbasicAuth=<username:password> -DnotTTY] target/scala-2.11/ScalaCrawler-assembly-1.0.jar

About

❄️ Scala implementation of basic HTTP crawler

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages