Skip to content

Latest commit

 

History

History
42 lines (28 loc) · 1.25 KB

README.md

File metadata and controls

42 lines (28 loc) · 1.25 KB
Raw

LPSE Parser

LPSE Parser - as reflected by its name - is a data parser for LPSE websites

Developed for KPP Pratama Kotabumi.

Requirements

Configuration

In vars.py, several parameters need to be configured

  • govName is the name of the website (without the trailing go.id or any subdomain)
  • staticCode is the last 3 numbers in Kode column of /eproc4/lelang
  • lowNum and highNum is the lower and upper bounds for the iteration.

Running

Use python main.py to scrape several information from pengumumanlelang page. The result will be recorded in results folder in csv format.

NOTE

I am not responsible to any damage to the server due to the excessive requests sent from the script.

PROJECT STATUS

The project is under development where other pages will be scraped in addition to the existing ones. A database might also be needed to manage complex data structure, rather than the current mechanism of storing the data to csv format.

TODO

  • Create other pages' scrapers
  • Build a database system
  • Build a reporting system (POD might be interesting)
  • Auto update for new data