Web Crawler

I have a website that goes like this:

[login to view URL], where NNNN is a number starting with 1 and going upto 20000.

So it is

[login to view URL]

[login to view URL] etc.

When you reach the site, it will have three results:

1. Page would show: No practice ID

2. Page would show:


This site is not configured

3. A page without #1 or #2 above.

For the three examples output above, please see the attached files.

The crawler should identify the URLs that belong to #3 above and spit out an Excel file. I have defined an outline of the workflow below.

If you use Python, you will call ‘urllib2’ or ‘requests’ – requests is easy to work with.

These libs allows to call a URL and evaluate the response.

First is to check the response code – 200 or 404 or 500.

Next, these libs provide access to the response body.

Response body can be loaded as text for simple text search or it can be loaded as xml for advance traversal.

[Removed by Admin]

Thank you!

Skills: Javascript, PHP, Python, Software Architecture, Web Scraping

See more: web crawler python 3000, python web crawler, simple python web crawler, personal web crawler, simple web crawler, web crawler mascot, how to use web crawler, web crawler tutorial point, web crawling history, aol web crawler, what happened to web crawler, python building web crawler, python web crawler lines, programming web crawler python, python web crawler scraper, python basic web crawler, python web crawler mp3, python web crawler mysql, advanced python web crawler, simple python based web crawler

About the Employer:
( 0 reviews ) Barcelona, Spain

Project ID: #18005472

18 freelancers are bidding on average €52 for this job


Hello !! I am Gang Lee and I read your project. But I have something to ask you. I can decide how it will take time and the budget after let me know about your project more detail. Your idea is amazing, and it wil More

€23 EUR in 1 day
(742 Reviews)

Hi I can provide these results in few hours. Are interested in the program or only the results? Thanks

€60 EUR in 0 days
(216 Reviews)

I can provide you simple scraping Python script that will use requests and lxml to parse HTML. It will check each page in range from 1 till 20000 and output into CSV file. Please show sample output you want so I can More

€30 EUR in 1 day
(504 Reviews)

Hi I have mastered scrapping and I have already done like this job. My name is Shan Bin and I'm a Chinese developer. I have 6 years of web scraper development experience such this projects. And I have good skills wi More

€111 EUR in 1 day
(120 Reviews)

Hi, Our Zend Certified PHP Engineers with 11+ years of experience can do your project precisely and exactly as per your requirements. We have expertise in PHP/ MySQL / JQuery / CSS / JavaScript / HTML / XML / SEO More

€24 EUR in 1 day
(206 Reviews)

Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within More

€110 EUR in 1 day
(71 Reviews)

Hi, I have done several projects similar with this. Please view my portfolios and contact me to discuss your project. Let me know the details via interview. Thank you. Regards,

€23 EUR in 1 day
(41 Reviews)

i can build code for you and don't worry watch my examples for you [login to view URL]

€200 EUR in 3 days
(54 Reviews)

Hi Nice to meet you. I'm python expert and my last project on is also python scraping task. My past works: Youtube comment scrapping Realestate property list to csv Jobsite content to csv And scra More

€29 EUR in 1 day
(57 Reviews)

hi there I have 10 years experience in Python and web development. I did lots of projects in web world, develop in most web technologies. 8 years developing Django sites and services. I did lots of projects in scra More

€88 EUR in 1 day
(30 Reviews)

Sir/Madam, I am an experienced Python developer with 2 years of experience in web scraping using selenium, requests and beautiful soup. I can do this project for you. Please go through my profile. I look forward to wo More

€39 EUR in 1 day
(39 Reviews)

Hello There, I have rich experience in working with web scraping, web crawling. I would like to do your project. Message me for more details. We can discuss more over chat.

€25 EUR in 1 day
(19 Reviews)

.......................................................................................................................................................................................................................... More

€88 EUR in 1 day
(2 Reviews)

Hi sir I will do it fast.

€24 EUR in 4 days
(0 Reviews)

Hi there. I am a professional scripts-utilities developer. I develop scripts-utilities including web scraping scripts. Ready to work on your projects any time

€30 EUR in 3 days
(0 Reviews)

Hello there, I am an experienced python developper. I am very good at finding solutions and writing efficient code. I am a quick programmer and I know how to take advantage of all the device functionalities. Advantage More

€12 EUR in 2 days
(0 Reviews)
€14 EUR in 1 day
(0 Reviews)

Hi, I have a great experience in writing scrapers (more than 70 for such web sites as [login to view URL] or [login to view URL]). Сan complete your project quickly, efficiently and qualitatively. Regards

€14 EUR in 1 day
(1 Review)