spider app in .net 4

App get input list of forum url and then spiders all the forums and finds ALL the messages (forum posts as many levels down it can do. Then in all posts - then it extracts all the EXTERNAL links it finds (EXTERNAL means any url that does not belong to the domain that it is spidering. So, if it is spidering [[login to view URL]][1] - then it only extracts urls from posts that does not urls which has "somedomain" as part of the url). it is a spider app that spiders forums - all postings it can possible find in up and downlgoing direction - and then it spiders them and saves any EXTERNAL links (links that does not include teh forum domain that is being spidered as part of teh domain name) - and then save those external urls in output file "[login to view URL]". Program should also try to find forum profiles and save them to "[login to view URL]" - Program must then spider in up and downgoing direction s it finds all posts exampel of url to spider: <[login to view URL]> <[login to view URL]> So, in forum posts - then program only extracts urls in posting swhere the url does not contain "[login to view URL]" (when it spiders that forum) . App must be .net 4 and multithreaded with 50 links and 3 retries and 60 seconds timeout. You mjust test teh app on your computer for 24 hours as burn-in test IMPORTANT Please write what you have done previously of coding of spiders that is related to this, why you are the rioght coder for this project and why your spiders are rock stable and spiders thoroughly.