Kapalı

Fix Existing Scrapers

We have two scrapers that do not work consistently. We need them fixed.

Both scrapers are written in Python 2.7 using Scrapy. Both use Crawlera as the proxy to avoid getting banned.

One is for [login to view URL] and the other is for [login to view URL]

If possible, we would like the scrapers to retrieve the data directly from the HTTP response rather than rendering the page and then scraping it.

Both spiders are always running on a Google Cloud Platform Debian Virtual Machine (Google Compute Engine) and write all results to a MySQL 5.7 2nd generation database also on Google Cloud Platform.

When you submit your proposal, tell me how much experience you have with scrapy so that I know you read this.

Sometimes they work correctly, and retrieve data and write it to the database. Other times, they appear to get stuck on something and do not retrieve any data and write 0 records to the database. This project is to fix this inconsistency and to make the scrapers reliable, robust, and running continuously without problems, errors, or issues.

Beceriler: Web Scraping, Yazılım Mimarisi, Scrapy, Python, Data Scraping

Daha fazlasını gör: fix browser alignment issues, fix browser compatibility issues, fix server configuration issues, fix css mouse issues, fix java scripting issues, fix stickam reconnecting issues, fix existing code, fix stickam connecting issues, fix browser incompatibility issues dreamweaver, fix browser compatibility issues dreamweaver file, dreamweaver fix browser compatibility issues, fix browser compatibility issues firefox, fix existing, fix existing auction site, fix ie6 css issues service, java script fix ie6 css issues, fix expanding box issues, fix website compatibility issues dreamweaver, fix cross browser issues, fix existing css

İşveren Hakkında:
( 3 değerlendirme ) Riverton, United States

Proje NO: #20556520

Bu iş için 22 freelancer ortalamada $137 teklif veriyor

zekovicm

Hi there, As an web scraping expert i can get it done for you. Please feel free to contact me to discuss further details. Best regards Miljan

$166 USD in 4 gün içinde
(146 Değerlendirme)
7.6
DevStar925

How are you? As you can see from my profile, I am Python expert, And I have a good experience in Trading Bot project. I would like to discuss in detail via chat. Thanks

$140 USD in 7 gün içinde
(68 Değerlendirme)
7.3
C3guru

Hello How are you? I'm very interested in your project I have built many python scripts So I'm sure I can complete your script Please share me more details over chat If you assign the job to me, I can start right now Daha Fazla

$140 USD in 7 gün içinde
(42 Değerlendirme)
7.0
Mickelson

Hi Nice to meet you. I have enough experience in python script. Below the libraries are I used in past project. selenium, pandas, matplotlib, lxml, beautifulsoup, scipy, and other useful libraries. I have written some Daha Fazla

$140 USD in 7 gün içinde
(117 Değerlendirme)
6.8
liveexperts123

Hi there, I have read your project description and i'm confident i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the Daha Fazla

$155 USD in 3 gün içinde
(66 Değerlendirme)
7.3
umg536

Hi there, please leave a message on my chat so we can discuss the budget and deadline of the project. I have read your project description and i'm confident i can do this project for you perfectly. Thanks . .

$155 USD in 3 gün içinde
(21 Değerlendirme)
6.2
sky930320

Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spide Daha Fazla

$140 USD in 7 gün içinde
(24 Değerlendirme)
6.3
kkc264043kkc

have an 3+ years experience in the web scraping and crawling with python Scrapy Can fix the scraper. want to know more about it. want to look at the script. These are my skills related to web scraping and crawling Daha Fazla

$200 USD in 2 gün içinde
(58 Değerlendirme)
6.2
brianconey

Hello How are you I have checked your description I am a python expert and I am sure i can fix the scraper it is possible python request and beautifulsoup I can execute your script on google cloud platform It is th Daha Fazla

1 gün içinde %bids___i_sum_sub_32%%project_currencyDetails_sign_sub_33% USD
(42 Değerlendirme)
6.2
kunitsynartem

Hello! I have 2 years of experience in web scraping using Python and Scrapy and I'm interested in your project for your scrapers fixing. Is it possible to see the code of your scrapers? Thanks in advance for your answe Daha Fazla

$140 USD in 7 gün içinde
(39 Değerlendirme)
5.9
Itblast

I am familiar with multiple programming languages, though most of my experience lies in scripting languages like python and object-oriented languages such as C# and Java. I have also worked a fair amount with various t Daha Fazla

$155 USD in 3 gün içinde
(10 Değerlendirme)
5.2
NIKE9

Thanks for your job posting. I am a full-stack web developer with 5+ years of experience and I am very interested in your project. My main skill is Python, PHP, Ruby and I have the relevant javascript skills such Daha Fazla

$155 USD in 3 gün içinde
(12 Değerlendirme)
5.4
shivansh121

Hi Employer, I can  extract any data from any [login to view URL] I have scraped the data from these websites & stored scraped data into CSVs and excels as per my clients requirement & I have made desktop applications u Daha Fazla

$88 USD in 3 gün içinde
(15 Değerlendirme)
4.8
oswaldoodavidal

Hello, I have experience in web scraping with Python. My experience with Scrapy goes from extracting data from simple websites that doesn't require JavaScript rendering to extracting data and automating processes with Daha Fazla

$250 USD in 10 gün içinde
(7 Değerlendirme)
4.5
innovationlight

Hi, I am python/php web scraping expert with more than 5 years experience in the field. Pls find my portfolios and good reviews. I have experience scraping recaptcha powered google site including amazon, dice, airline Daha Fazla

$100 USD in 3 gün içinde
(11 Değerlendirme)
4.1
seocloaker81

Hi sir, I have 4 years of experience in Scrapy & 7 years of experience in data extraction, & I am proficient in using tools such as Selenium, as well as an expert in creating Python/.Net scrapers. Regards Navpreet Sing Daha Fazla

$45 USD in 3 gün içinde
(5 Değerlendirme)
3.9
emely001

Hello, This happens to me sometimes, when I do scraping with scrapy that stays hung. This solved by configuring its timeout, or using threads in this way if some thread doesn't receive data another thread will do it. Daha Fazla

$140 USD in 2 gün içinde
(6 Değerlendirme)
3.0
rutujaww

Hello, I have read your project description. I have done about 4 projects with scrapy with selenium and geckodriver. I have experienced some difficulties with these and was able to solve some of them. If I can take a Daha Fazla

$171 USD in 7 gün içinde
(2 Değerlendirme)
3.0
sevenstarservice

Hello, I'm a full time freelancer and am free now. I have read your project description I am very interested in this project. I have a lot of experience in this type of work. I'm sure I can complete it on time w Daha Fazla

$155 USD in 3 gün içinde
(2 Değerlendirme)
2.7
HamzaZahid5

Hi I have more than 5 years experience in Python, Web scraping, Database (SQL), Html 5, CSS, PHP(Laravel), Excel, Google Sheet, Python, Data Entry, Software Engineering, Data management, SQAE and Data Analysis. If poss Daha Fazla

$150 USD in 3 gün içinde
(2 Değerlendirme)
0.5