We need a web crawler that will go through a specific merchant's website (e.g. [url removed, login to view], [url removed, login to view], [url removed, login to view]) to pull product listings that the merchant offers. The product listings should be stored in a database that can be easily exported into an Excel / CSV / Tab delimited text file.
At a minimum, for each product, we would need to store Merchant Name, Product Name, Manufacturer, Price, Product Code, Description. We do not need all information from the website, just information on all products that the specified merchant has listed.
Script should be performance-based and not be taxing on the merchant servers. Immediate need is to run the crawler on specifically [url removed, login to view] to retrieve all products listed on their website. Request could be expanded to further merchants and therefore script may need to be adaptable.