I need a solution for extraction data in "near real time" from different websites but with the same general topic.
I don't know the numbers of starting pages (maybe 500.000)
The software should do:
- Autodiscovery new URLs (based on specific rules)
- Extract and save some particular data (object name, description, price, ...) and general data (HTML title, author, ...)
- These data must be updated continuously
- These data should be saved in a database
- I need a JSON API that should inquiry that database with these functionalities
- parametric search
- aggregate search
- full-text search
- In case of error or new needs, It should be reprocessed all or some pages.
- Easy to add new sites.
- Enable to scrape ajax based websites.
I'd like to have a solution that can be easy to install, configure and most of all easy to update in case of errors.
Bu iş için 57 freelancer ortalamada €3756 teklif veriyor
Hello there! I would like to be considered for your project. We can build a script to scrape multiple websites according to your requirement.
Greetings, I am an experienced professional scrapper and have done similar projects in the past. Same can be verified from my profile. Let me allow to assist you with your requirements. Thanks
Hi there how are you I am a senior python developer and i have used python and selenium . Please contact me and tell me about your scraping project . thanks for your posting
Hello, I am Python expert and expereienced on scraping web sites. I can build a nice app for your usage. Can we discuss details over chat please? Best Regards
Hello, I have gone through your Project requirements and Yes I can do it and also assure you that I can do this job perfectly. Hope to hear you soon. Thank you.