I have a program that gathers data from websites (adult). Each website is represented by a class with several methods (get articles from index, get data from article, etc.)
I need to create new classes for data collecting from a particular website (one for each website), no login is required. The data will then be structured as an object and returned.
I can provide you with some examples from other websites, usually it is very similar.
I have about 100 websites in a queue, waiting to be created. Some websites have a very similar structure, so the differences between those are minimal, usually some html tag has a different name/id.
I would prefer to split this project into more milestones - each milestone would represent 20 pages (or less/more depends what you prefer) and when the milestone is done, then you would start to work on a next batch of websites
First milestone will be a little bit harder/more complex (depends on your skills), but I need those sites to be done first.
In this case I would prefer a solution like this (if it is even possible):
- when captcha happens, sends the captcha link by email and wait till the captcha is filled on another device
- usually on mobile phone I will open the email and fill the captcha
- when it's done, process continues
Or if not possible, I would imagine something like this:
- when captcha happens, opens chrome with google captcha and waits till the captcha is done manually
- when captcha is filled, close/hide the chrome window and continue in the process
Other websites don't require the captcha bypass, so they won't be that complex.
When possible, I always prefer to use htmlagility pack, otherwise I'm using selenium chrome.
Except the captcha, it is not a complex scripting. Maybe a little time consuming, but I can provide you with a couple of done website where you can check how it was done and re-use the code for creating new classes.
In proposal, please let me know if you have done something similar in the past. And what would be the price.
Thank you and kind regards,
Nice to meet you on freelancer. My name is Da Zhang. I am an experienced C# programmer with more than 3 years scraping experience. Look at my upwork profile also: [login to view URL]~016a0f5cbf2aeb73ea? Daha Fazla
Bu iş için 25 freelancer ortalamada $508 teklif veriyor
Hi Matej You described proposal very carefully. Thanks again for your detailed explanation. I can help you with this project. I will use selenium mainly for this project and can do this project in 5 days.
Hi I have done this type of project in past. I can do this project. I have few questions about the requirements. Please message me so that I can discuss the project in details with you. Warm Regards Amit Bandana
Dear Client, I have read your requirement in detail. I have several projects of scraping with C#, Python, etc. We can start work immediately, and I can lead this project to perfect result. Thanks, Regards.