This crawler will ask for website, Project(Folder to create), Threads to use.
This script will spider websites for your input.
Required Libraries will be pre installed when you install Python, if not then...
Run:
pip3 install -r requirements.txt
To Run : user@host:~ python3 crawler.py
Have fun with web crawler