Skip to content

Latest commit

 

History

History
44 lines (38 loc) · 1.59 KB

README.md

File metadata and controls

44 lines (38 loc) · 1.59 KB

Instructions

Pre-Requisites

  • Python
  • Python bindings for Selenium
  • Chrome WebDriver

Downloading Python bindings for Selenium

pip install selenium

Downloading Browser Drivers

Selenium requires a driver to interface with the chosen browser. Get the required drivers from here

Browser Driver Link
Chrome https://sites.google.com/a/chromium.org/chromedriver/downloads
Edge https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/
Firefox https://github.com/mozilla/geckodriver/releases
Safari https://webkit.org/blog/6900/webdriver-support-in-safari-10/

Once the driver files are downlaoded, copy them to /usr/bin or /usr/local/bin

Database tables

  • Create a schema. Change the name of the shcema in the code to the name of your schema.
  • Create a table using this script:
CREATE TABLE `real_estate_listings` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `created_ts` datetime NOT NULL,
  `source_id` int(10) unsigned NOT NULL,
  `posted_time` datetime NOT NULL,
  `title` varchar(200) CHARACTER SET utf8mb4 NOT NULL DEFAULT '',
  `link` varchar(500) NOT NULL DEFAULT '',
  `source_listing_id` varchar(20) NOT NULL DEFAULT '',
  `location` varchar(200) CHARACTER SET utf8mb4 NOT NULL DEFAULT '',
  `category` varchar(100) CHARACTER SET utf8mb4 NOT NULL DEFAULT '',
  `message` text NOT NULL,
  `contact_name` varchar(100) NOT NULL DEFAULT '',
  `contact_phone_numbers` varchar(100) NOT NULL DEFAULT '',
  `contact_email_addresses` varchar(300) NOT NULL DEFAULT '',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1015 DEFAULT CHARSET=latin1;