I need an expert in extracting data from retail websites.
There is always issues our existing team comes up against.
- Need for different headers
- Proxy conditions - Data Center (waterfalls into) Residential / Unblocker Proxies
- Cost Management
- Algorithms to prevent over spend or over time on each page or stuck conditions
BOTH Hardcode (PHP) and Softcode coding (XPATH 1.0) will be required
This job will be full time, adding sites - and multiple people will be employed in Round 1 - which we will reduce to a most effective team after some paid test period.
You will have access to a testing panel only to run tests on the code - and the application code will be kept separate.
An NDA may be required.
Please reply with some details as to your scraping ability and expertise and how you would best for example - be able to access pages on a site like [login to view URL]
Ability to complete multiple sites a day is an advantage.
Ability to FIND new sites to add across multiple countries (that we do not already have) is also an advantage.
Some under-represented countries for our application may be required to research and add first - UK / Canada / Germany / Australia. And of course, USA.
VPN essential for these.
Proxy tools include Buy Proxies and Luminati / Scraper API - your input is also useful on tool and proxy choice evolution.