Tamamlanmış

Selenium and Scrapy web scraper doesn't work (score = 0.1)

Hello,

I own a web scraper in Python 3, which uses Selenium. It connects to good quality private proxy and scrapes the website using Chrome browser, exporting content to HTML files.

The problem is the scraper is getting 0.1 score (1.0 is maximum), which is too low and searching on site is being blocked due to unusual traffic. When searching manually without using the scraper, my score is 0.9 (when logged to Gmail account).

I assume that the problem is Google's verification system v3 (no images, just score) is detecting the bot.

Scraper remembers the login and password to Gmail account, as it should increase the score but Google blocked the try of logging in, due to critical error (unauthorized use of password).

Scraper uses chrome driver.

0.5 score would be enough to scrape.

I own a source code.

If you are interested in fixing this problem, please post an offer.

Important: Please describe what is your idea on fixing the problem. If you advice to create totally new scraper, please also describe how it would work. I respect your know-how, but please be precise as possible.

Thank you.

Beceriler: Javascript, Python, Scrapy, Selenium Webdriver, Web Scraping

Daha fazlasını gör: excel web freeze pane doesnt work, create web scraper, need web scraper takes search data excel, javascript web scraper, wget web scraper, net web scraper, Python web scraper, web scraper plus, velocityscape web scraper plus, velocityscape web scraper plus getafreelancer, web scraper outsourcing, online web scraper, using web scraper, selenium web scraper, virtuemart doesn`t work move hosting server, joomla administrator menu doesn`t work mootools, freelancer com doesn t work

İşveren Hakkında:
( 0 değerlendirme ) Tokyo, Japan

Proje NO: #19978336

Seçilen:

agusarev

It could be reCAPTCHA made by google. First of all, you need to check html responses to see, if it exists. If problem is connected with captcha, you need to use api from paid source, that will take captcha and solve it Daha Fazla

%selectedBids___i_period_sub_7% gün içinde 55%project_currencyDetails_sign_sub_9% %project_currencyDetails_code_sub_10%
(2 Değerlendirme)
1.6

Bu iş için 19 freelancer ortalamada $147 teklif veriyor

sky930320

Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spide Daha Fazla

$200 USD in 7 gün içinde
(33 Değerlendirme)
6.8
mananraja

hi, I have read you description in detail. I have experience with SCRAPY AS WELL AS SELENIUM scrapers in PYTHON. I can look at the code and resolve the issue. ALSO, i believe scrapers made using WEBREQUESTS are Daha Fazla

$155 USD in 3 gün içinde
(138 Değerlendirme)
6.2
harrysart

Dear Hiring Manager.⭐ I am a python programmer and I have faced many python problems.. Hope you give this project to me.. Relevant Skills and Experience I have worked with python + web development with Django framewor Daha Fazla

1 gün içinde %bids___i_sum_sub_32%%project_currencyDetails_sign_sub_33% USD
(68 Değerlendirme)
6.0
zhaohm7

Hello Thanks for your posting project. I saw your description carefully. I can help you with my best. I am a web expert who have 10+ years of experience. My skills are : Html5,Css,,Jquery, MySQL PHP, Laravel and Python Daha Fazla

$140 USD in 7 gün içinde
(23 Değerlendirme)
5.7
yongjin818

Dear As an expert in web scraping, I developed 1000+ spiders using python, scrapy, php,nodejs,c# and proxies. And I scraped data and download images, files for CEOs and output result as csv, json, xml(yellow, ebay, w Daha Fazla

$140 USD in 7 gün içinde
(66 Değerlendirme)
6.0
kkc264043kkc

want to see the source code. can try with proxy rotation and user agent. These are my skills related to web scraping and crawling Have done scraping in CasperJS Phantomjs, python. Have done testing and automation Daha Fazla

$166 USD in 2 gün içinde
(44 Değerlendirme)
5.7
susanna2018

Hi, Client!! @$@$@$PLEASE CONTACT ME. I CAN DO IT WONDERFULLY.$@$@$@ ##% Your idea is attracted me to your world.%## I am Python expert. i have various skills. @example@ python selenium and beautiful soup, regu Daha Fazla

$140 USD in 3 gün içinde
(34 Değerlendirme)
5.3
Stephenrajs

Hi there, I have scraped amazon, aliexpress, yellow pages ,yelp ,zomato etc. I have 500GB internet with 20mbps speed, i have 6 systems I can do it. I have done many related projects like this. If you are provi Daha Fazla

$90 USD in 3 gün içinde
(90 Değerlendirme)
5.1
whiteeagle0001

Hi Thanks for the opportunity to place bid on this project. I carefully read the comments and thought about the points to consider in the project. I have rich experience in your project. I can help you surely and Daha Fazla

$155 USD in 3 gün içinde
(18 Değerlendirme)
4.4
iusmanadil

Hi, Python Ninja here! I have understood your requirement with the Selenium and Scrapy web scraper work, and I am very confident with that. I can make it work for you. I am Usman. I have 6 years of experience Daha Fazla

$250 USD in 5 gün içinde
(16 Değerlendirme)
4.4
xiaomingming

Hello dear. I have read all description. I m a senior web developer working for 6+ years and my skills and experiences are perfectly fitted on this project. It is my pleasure to work with you, and I ensure to deliver Daha Fazla

$140 USD in 7 gün içinde
(1 Yorum)
4.2
maryumakhter5

Hey there! Selenium scraper here, I can do any sort of data mining or web scraping that you need to be done in a reasonable amount of time. I have years of Python, HTML, CSS and JavaScript experience under my belt Daha Fazla

$120 USD in 3 gün içinde
(4 Değerlendirme)
3.7
zpedia723

Hi, I'm interested in your job description. I wrote several scraping scripts with python and c++. As I have gained rich experience with web scraping, I'm pretty sure I can deliver satisfactory product for you. I loo Daha Fazla

$200 USD in 3 gün içinde
(8 Değerlendirme)
4.2
tidralidra

The blocking issue can be because of Google ReCaptcha. i faced this problem in when i was try to scrap google search results on Mass amount. i didnt totally bypass it but i was able to avoid it for much longer time usi Daha Fazla

$50 USD in 5 gün içinde
(1 Yorum)
0.7
ivanpilipovic

I can help you, I need to see code to see how scraper works. Some of projects -> [login to view URL]

1 gün içinde %bids___i_sum_sub_32%%project_currencyDetails_sign_sub_33% USD
(0 Değerlendirme)
0.0
evonsystems7

"Hi, Hope you are doing well! Thanks for sharing your project requirement with us. It will be our great pleasure to work on your project. I have checked your requirement, yes we can do it, because we already work on si Daha Fazla

$155 USD in 7 gün içinde
(1 Yorum)
0.0
vikasp007

Hello there, Hope you are doing good...!!! As per your requirement, I am very much suitable for the job kindly invite me for a chat for further discussion regarding the project. I have more than 6 years of e Daha Fazla

$155 USD in 3 gün içinde
(0 Değerlendirme)
0.0
Fahadubitian

Hi, hope you are going well. This is Fahad Yameen and want to work with you in this project. I am not saying that I am the best developer to solve this task. But I can assure you that I will give my 100% to analyze and Daha Fazla

$250 USD in 14 gün içinde
(0 Değerlendirme)
0.0