Crawl website and copy all records (minus one condition)
$30-250 USD
Sürüyor
İlan edilme: 10 yıldan fazla önce
$30-250 USD
Teslimde ödenir
I need a website's records copied by crawling. The website currently has about 250,000 pages. I need you to crawl all the pages and copy them into an organized database. There is one condition where you should not copy pages if the title of the page contains a comma (this will be explained in greater detail via PM).
You MUST have a lot of experience crawling websites.
You must create a fast crawler so that the website is copied in a reasonable time.
The website must be crawled correctly so that proper database is created.
Please PM your experience with crawler. I will PM you the website that needs to be copied and give you all details.