Build 11 tiny scrapers to supplement main scrapers
$250-750 AUD
In Progress
Posted over 8 years ago
$250-750 AUD
Paid on delivery
Stagecoach Software Pty Ltd is developing a website to contain statistical profiles of a particular industry in Australia. Our stack is Ruby on Rails 4.2, Ruby 2.2, Passenger 4 and MySQL 5.5. We run a 3 server configuration on Engine Yard with dedicated instances for our db, web app and utilities. Our database receives about 30,000 new rows per day through scraping operations (22 different scrapers using nokogiri). Presently our app instance only provides admin operations; the consumer front end is under development.
Currently our team comprises 4 backend ruby developers and 1 front end developer.
Each of our scrapers records the URL of each relevant product. We wish to record the last date each product is advertised for sale. We therefore need 11 nokogiri micro scrapers developed that will log on to the URL of each current product and insert the current date into a designated column if the advertisement is still current.
Each scraper must run as a discrete *.rb file and be capable of running as a cron. Code must be included to make each scraper record start and stop times and number of columns populated on an admin page that is already doing the same for all existing cron jobs. Further instructions will be given to the successful bidder.
All code must be uploaded to github and approved by the lead developer before deployment.
This is one of a pipeline of projects in this enterprise. Further ruby projects will be available if the above tasks are carried out well. Applications from a freelancer experienced with Engine Yard are welcomed.
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi
Hi.. Expert Web Scraper & Data Minor here. I have done too many similar project in past. Having best scraping tools and experience i assure you 100% accurate and good quality work. I have huge scraping experience. looking for your reply. Thanks
Hi, I have 5+ years experience in ruby for creating web apps and web scraping. I recently scraped Amazon, Yell, Ebay and Zillow. Have experience using Mechanize, Nokogiri and Anemone. Let me know if you would be interested. thanks