Python App Performance Improvement

We need to improve performance of a Python app. It is a simple spider. Collects data and reports them back to primary app. Its task is to visit URLs input by users, crawl them, collect data and report back. Currently, can handle only a few thousands URL per day. It must be able to handle 1+ million. Also, URL checking queue gets blocked for whatever reason and that must be debugged as well. Might be Python issue or other. ============== SKILLS: ============== Python, Terminal access WHM cPanel = optional = PHP Laravel Vue ============== TASK: ============== - improve performance of the spider - debug and fix any issues ============== DEADLINE ============== 11th May 2018 ============== BUDGET ============== $120 Primary requirement is Python but PHP, Laravel, Vue knowledge would help. You will be provided access to GitHub to check out the code. Also test user account is available so you can see it in action. Also, please mention word "bee" at the start of your reply to prove that you read this far.