Build a Scrapy scraper

Completed Posted 6 years ago Paid on delivery
Completed Paid on delivery

Scrape a real estate website for all properties and agents

Utilizing Scrapy & Scrapinghub

All fields to be scraped will be provided

• Scraper must be efficient

○ Don't check every URL on the site

○ Fast to find items

• Scraper must be accurate

○ No false negatives with regards to duplication

○ Limited or no duplication

○ Field values are populated if they exist

○ Field values contain correct data

• Scraper must include a data sanitation pipeline

○ Scraper will output to a file any item removed

• Scraper must include a pipeline to feed to a Wordpress site via a custom API

○ Feed properties into WP site with no dupes and correct values in fields

○ Feed agents into WP site with no dupes and correct values in fields

○ Associate agents to properties utilizing an existing property field

○ Upload images and associate to property utilizing existing WP field

○ Upload images and associate to agent

○ Export a list of statuses, types, locations, etc. that did not match

• Before payment

○ Scraper must run on the entire site at least once for results to be verified

Spot checking will be performed to ensure accuracy

First complete the scraper logic then complete the pipelines

Data Scraping Python Scrapy Web Scraping

Project ID: #15347328

About the project

12 proposals Remote project Active 6 years ago

Awarded to:

Luqman1234

I will write a spider for you in `scrapy` with both parsing, crawling logic and Item Pipeline which will make any sanitizing checks easy to be performed on the scraped data Relevant Skills and Experience I am a specia More

$45 USD in 10 days
(2 Reviews)
1.8

12 freelancers are bidding on average $66 for this job

mantislin

Hi sir, I am scraping expert, I have did more than 350+ scraping project, please check my feedback then you will know. Can we discuss more details about this project? then I will provide example data/scr More

$96 USD in 2 days
(295 Reviews)
7.6
Nada100200

Hello Client, Hope you are doing well ! I have great experience of extracting information from websites . I provide best solutions at fastest speed with the cheapest cost. Your satisfaction is my only priority. I wou More

$10 USD in 3 days
(16 Reviews)
5.6
anzarulislam

Hi.. I am interested to your project . I believe that I'll do this work properly and in timely . I am agree with your budget. And I'm sure that I'll finish the task with your full satisfaction . Lets start More

$35 USD in 10 days
(7 Reviews)
4.4
gallimard

I've already Done that before I can show u some of my previous projects if u want

$35 USD in 3 days
(1 Review)
4.4
adilsyed123

Expert here to do and complete this project professionally and quickly, I have done similar projects, I can do trial work before hire me sir , will wait for your reply, Thanks, Syed

$50 USD in 5 days
(7 Reviews)
3.4
Xeeshanah

Hello there, Read your project description and i can build a scrapper that will full fill all the specification you mentioned in the project description. I'm a professional computer scientist. Relevant Skills and Expe More

$50 USD in 1 day
(2 Reviews)
1.6
joselumibaoiii

A proposal has not yet been provided

$55 USD in 5 days
(0 Reviews)
0.0
rosyrozzah

write the spider for the scraper, include the Item pipelines and the feed export to store the data either locally or in Amazon s3

$77 USD in 4 days
(0 Reviews)
0.0