Kapalı

Puppeteer - Website Crawling project

Hi developers,

I hope you are all having a lovely day.

I need a crawler that crawls websites, most importantly the ads on these websites. The crawler screenshots the website, the ads and saves the target and source urls of the ads. The crawler is optimized for a small list of websites, it should be easy however if new website styles can be added easily. There exists a prototype that already implements these features, so this project will mostly be re-coding the prototype but better and more professional. The prototype is JavaScript, there can be many things copied from the prototype. The new crawler should be coded new and implemented in TypeScript. Also, the finished data is stored in AWS S3 and a database (Preferrably AWS DynamoDB).

There should be an API implemented (secured with Bearer Token only) that exposes the crawled data and lets the API consumer add new URLs to crawler.

It is very important, that the new crawler is reliable and the code should be as structured and modular as possible, to add logic, if e.g. a website is added. The crawler+API should run in Docker on an AWS EC2 instance.

I will provide you with the source code of the app, as soon as we start chatting.

Technologies required:

- TypeScript

- Puppeteer

- Node.js/Express

- AWS S3, EC2, DynamoDB

- Docker

Best regards,

Florian

Beceriler: Amazon Web Services, Web Scraping, Node.js, Typescript, Docker

Müşteri Hakkında:
( 2 değerlendirme ) Guldental, Germany

Proje NO: #33772770

Bu iş için 22 freelancer ortalamada €499 teklif veriyor

AwaisChaudhry

Hey there, I have 4 years of working experience in web technologies and I have a strong working knowledge of every possible framework which is in the market. In terms of client-end I know: Node.js, Web Scraping, Docke Daha Fazla

€750 EUR in 18 gün içinde
(17 Değerlendirme)
7.0
PoojaRautela417

Hi there, I can write script in Node.js using Puppeteer according to your requirement. Please ping me back so I can start right away. Looking forward to hearing from you soon. Thanks & Regards Pooja Bohra

€500 EUR in 5 gün içinde
(70 Değerlendirme)
6.2
(29 Değerlendirme)
5.5
(38 Değerlendirme)
5.5
(6 Değerlendirme)
4.9
mykhailodew

Hello, sir! How are you? I am a web scraping specialist. I have rich experience about web scraping. I've been using bs4, selenium or scrapy... I've ever scrapped dozens of sites at once also. At that time, there were a Daha Fazla

1 gün içinde %bids___i_sum_sub_32%%project_currencyDetails_sign_sub_33% EUR
(16 Değerlendirme)
4.9
oleksdevr

Hi . I am a fully experienced NodeJS developer. I have strong background in Web crawling using Puppeteer. I’m confident I can deliver high quality result. I’d like to discuss more about the project. I can start you Daha Fazla

€500 EUR in 7 gün içinde
(6 Değerlendirme)
4.8
rashidamjad

Hi there, I am Rashid, Hope you are doing great. Firstly about the Puppeteer - Website Crawling project that I have a great experience in it. I am very confident to pull it off once awarded. I am a Full stack developer Daha Fazla

€650 EUR in 15 gün içinde
(7 Değerlendirme)
5.0
Devrits

Hi!, I’ve carefully checked your requirements and really interested in this job. I’m full stack node.js developer working at large-scale apps as a lead developer with U.S. and European teams. I’m offering best quality Daha Fazla

€750 EUR in 4 gün içinde
(21 Değerlendirme)
4.4
(6 Değerlendirme)
4.3
brananikolic0091

Hi, I am a senior developer with 5+ years of experience. I just have read your description and can start right now. Skill set - languages : Javascript/Typescript, Python, PHP, Golang - frameworks : Next, Nuxt, Nest, No Daha Fazla

€500 EUR in 7 gün içinde
(1 Yorum)
3.4
dandiuzhenko

Hi, Dear I've read your requirements very carefully and I`m really interest in your project. I have 7+years of great experience in ExpressJS, Node.js. Also I've 3years of working experience in Docker, AWS, Puppeteer. Daha Fazla

€500 EUR in 7 gün içinde
(1 Yorum)
3.2
EDGESOL

Hi, I Hope your life is going good!! We are experts in Auction Website & Mobile App development I will happy to show you a demo via personal chat. I have read your requirement Puppeteer - Website Crawling project an Daha Fazla

€555 EUR in 4 gün içinde
(1 Yorum)
2.8
(1 Yorum)
2.8
(2 Değerlendirme)
2.4
mandionisio154

Hello, Florian After carefully reviewing the experience requirements for the job description, I feel that I am suitable for the job. I have 7+ years of rich experience in full stack development. I am specialized in Nod Daha Fazla

€500 EUR in 7 gün içinde
(2 Değerlendirme)
2.0
maksimjevdjic122

Hello, how are you? As a senior developer, I always concern with Security while development. Skill Set: - Basic skill C, C++ I have developed anti-virus, VPN, game engines an Daha Fazla

€555.56 EUR in 10 gün içinde
(0 Değerlendirme)
0.0
vladyslavmazurok

Hi! I checked you need a highly proved javascript programmer for your project. For the last 8 years, I worked on various javascript libraries like React/Redux, Angular, Node and gained knowledge with payment integratio Daha Fazla

€555 EUR in 7 gün içinde
(0 Değerlendirme)
0.0
ayesha0124

Hi, I can help u as i have done several similar jobs related to Docker, Node.js, Typescript, Amazon Web Services and Web Scraping, I have read the details and furthermore discuss about it, plz discuss with me in detail Daha Fazla

€750 EUR in 8 gün içinde
(0 Değerlendirme)
0.0
lakimo21

Greetings ! I am skilled Frontend engineer . I am experienced in such as Web Scraping, Docker, Amazon Web Services, Node.js and Typescript. I can start right away and want to discuss in detail. Please contact me to Daha Fazla

€400 EUR in 5 gün içinde
(0 Değerlendirme)
0.0