Need to code some rules to eliminate duplicates

We recently paid a freelancer to take a public real estate feed (available in XML and API format): [login to view URL] The freelancer sucessfully pulled from this feed by a cronjob daily and fed it into our personal database: [login to view URL] Although it works for the most part there are a lot of errors and A LOT of DUPLICATES. We need a responsive and fast (looking to get this done in 2 weeks time, less is always better) to fix the freelancer's code or write their own that will: 1. Eliminate duplicates 2. Fix the feed to bring in apartments and buildings from Nestio daily There is a section of the urbansherpany manager that shows active apartments for a specific building. An apartment is active is it is listed in the Nestio feed. Once the listing is removed from the feed, or it is not there the next time the cron runs, the apartment number should no longer be "Active" but should still live in the manager and database. Some other rules: - Ave, Av, Avenue all meant the same - St, Street - E, East - W, West - N, North - S, South - Management, mgmt - regarding apartment numbers: #2L, 2-L, 2L all the same - nothing case sensitive (i.e. BOLD New York and Bold New York should be treated as the same) - regarding streets, 35th and 35 should be the same - 8th and Eighth should be the same - phone numbers, price, showing info, incentives etc within a listing should not result in a new listing, only if the address or apartment number are new should a new listing be made. Once again I can not stress this enough: PLEASE BE RESPONSIVE AND GET THE JOB DONE IN A TIMELY MANNER. We need a way to prevent duplicates from entering our database while still updating the information from nestio. The work is 80% there but looking for someone to take us home that last 20%. The last freelancer I had became unresponsive and went way over schedule. Before you bid please make sure that you are available and can get this done within two weeks time. Before I am going to award a bid, I would like to talk to the new freelancer and go over everything so there is NO miscommunication.