Skip to content

Web Crawler and API for the data of the Public Security Departments that will be used in the Stay Safe app

License

Notifications You must be signed in to change notification settings

fga-eps-mds/2020.1-stay-safe-secretary-service

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Secretary Service

Secretary service of Stay Safe project

Run Flask API

If you haven't build yet:

$ docker-compose build api db 

After that you just need to run:

$ docker-compose up api db

To have access to pdb (debug library) when running, use the following command:

$ docker-compose run --service-ports api python -u main.py

Run pylint

$ docker-compose run api sh -c "pylint **/*.py"

Pytest / Coverage

To run tests

$ docker-compose run api coverage run -m pytest

Run Sonarqube

$ docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest
$ docker run -ti -v $(pwd):/usr/src --link sonarqube newtmitch/sonar-scanner -Dsonar.projectName="Secretary Service" -Dsonar.projectKey=secretaryservice

Link to see the scans projects: http://localhost:9000/projects

To report results with Coverage

$ docker-compose run api coverage report -m

Run Crawlers

$ docker-compose run crawler_crimes scrapy crawl spider_name

Crontab

Crontab is a time-based job scheduler in Unix-like operating systems, it doesn't need to install if you're using a Unix based SO.

Setting Crontab

Enter the following command to edit your crontab file:

$ crontab -e

If it's empty you should set the first line of the file with:

SHELL=/bin/bash

After that you just need to set the command to run a job.

Example

To run a sh file at 2am of the first day of the month:

0 2 1 * * /home/user/stay-safe/src/crawlers/df.sh

To understand more about crontab schedule expressions click here

Every sh file must be in the correct permission, to set the correct permission run:

chmod +x script-name-here.sh

About

Web Crawler and API for the data of the Public Security Departments that will be used in the Stay Safe app

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages