Our requirement is a full featured dynamic distributed crawler, which need to crawl all the shopping sites in India. The crawler need to extract all the information about the products it’s price, offers, seller and etc. from all the shopping sites and need to store all this collected data in a fully structure and perfectly indexed database for easy extraction of data from the DataBase. The dynamic pages of those shopping sites should be crawled repeatedly within in minimum span of time. The crawler should be dynamic and distributed in nature. The DataBase should be well structured, well organized and perfectly indexed. As on above we need a full functioning stable crawler, database system, proper indexing mechanism.
Note: With your bids share your proposed design model for the above requirements.
Bu iş için 1 freelancer ortalamada ₹77777 teklif veriyor
Hello, Below my developement experience - VB5/6 and MSSql developer from more than 15 years - PHP/MySQL, Wordpress/joomla/prestashop from more than 10 years - I'm the founder and developer of opensource project Daha Fazla