Custom index bot and ranking algorithm

In Progress Posted Apr 22, 2009 Paid on delivery
In Progress Paid on delivery

We would like to build an index of websites and rank them based off of their interconnectedness by evaluating the number of unique inbound links each receives from other sites in the index.

## Deliverables

Here is the desired solution:

1) Start with a list of predetermined websites in a database.

2) Scan each of the pages in the database and record outbound links to other sites.

3) Add the newly discovered sites and pages to a database, creating an index of all sites and their pages.

4) Calculate the number of unique links that each site on the list receives from other sites on the list (include both the original “seed?? list sites and the sites discovered in step #2) not counting links that go directly to the home page of the site. For example, [url removed, login to view] should be counted but a link to [url removed, login to view] should not be counted.

5) Rank the sites on the list based off of the number of valid links from other sites on the list so that the site with the most valid links is #1, the site with the second most valid links is #2 and so on.

6) Run steps 1-5 every 24 hours scanning all sites in the index and adding new sites that are discovered and re-calculating rankings.

7) After each time a scan is run an email including the list of sites in the index in their rank order should be sent to a pre-determined address.

Engineering MySQL PHP Project Management Software Architecture Software Testing

Project ID: #2684593

About the project

2 proposals Remote project Active Apr 23, 2009

Awarded to:

sufertashu

See private message.

$199 USD in 14 days
(7 Reviews)
4.9

2 freelancers are bidding on average $312 for this job

smartdatasoftvw

See private message.

$425 USD in 14 days
(23 Reviews)
4.3