Scrape websites with scraper and parse data to elasticsearch / kibana / logstash

This project received 41 bids from talented freelancers with an average bid price of $614 USD.

Get free quotes for a project like this
Project Budget
$250 - $750 USD
Total Bids
Project Description


- Show my products, and show competitor (29 sites) prices of that same product.

(!!! there needs to be a solution to link productname of theirs to mine productnames. I have 'Brand', and 'Modeltype' to make a link easyer!)

- Visualize price progression over time with kibana per product / store. (Schedule a scraper every night!)

- Show if a price is dropping or going higher per product.

Use the opensource python scraper 'Scrapy'.

Feature Goals:

Adding sales of my shop, adding positions of product in categories of competitors and mine.

The scraper will run on a DigitalOcean VPS. You install the server + scraper etc.

The Kibana Logstash etc. server will run on other server.

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online