Web crawler bot / scraper application

This will be several projects. We would like to hire a team or someone that can work with a team to do different parts of this project. We currently have code for this project in PHP, but would like to revamp or redo the entire application. This will be a ongoing project. These are the different parts of the project in a general overview:

1. We need an application that will be able to crawl different websites. We will need to login in to the website, scrape data from the website, and depending on the data that we gather, submit certain data to the website.

- This can be be in any language, but we like this to be fast. Currently, from the time it finishes parsing a page to submit the data, it takes 3 seconds. We would like to lower this time to about 1 second. We would like someone who can optimize the application and code and identify bottle necks.

- We might want multiple crawler instances to be running at once so would like someone familiar with threading and process forking.

- This will need to be maintainable, so might want this to be in OOP. We will want to to be able to add different sites to scrape to our application easily. Might want to have like a plugin system to easily plug in new sites.

- We want someone that can start a project from beginning and also be able to work with existing code and integrate their implemenation elegantly.

2. We need an admin area, preferably web based. The admin area will manage our application and should allow us to run our application and see output from our application.

3. This application will be database driven. Data for the sites we are parsing will be stored in the database. Will need someone or team that will have experience in good database design. The database will need to be easily maintainable and extensible.

4. This application will be run across different servers for different clients. This data will be stored in the database to be accessed by the application.

Desire a team/individual with experience in:

- Many different languages

- Database Design

- OOP and Design Patterns

- Process forking/threading

- Performance optimization

I will need to have access and the final rights and ownership to the source code.

Please reply with:

1. What technologies you will do this in
2. If you think you can develop the scraper/crawler to parse a page and submit to 2 pages in 1 second or less

Beceriler: .NET, C++ Programlama, Java, MySQL, PHP

Daha fazlasını görün: web crawler bot php source code, website for bottle design, web page design languages, web hire, web design patterns, web design overview, web design beginning, web-crawler, web based languages, web based language, web 2.0 languages, submit to web crawler, source code in oop, php design patterns, patterns to do a website, parts of web design, parts database access, new hire process, new hire application, languages web, java web projects with source code, java web code, java start web, java projects hire, java design patterns

İşveren Hakkında:
( 1 değerlendirme ) Richardson, United States

Proje NO: #721188

25 freelancer bu iş için ortalamada 1164$ teklif veriyor


Please check PMB for details.

in 30 gün içinde1100$ USD
(81 Değerlendirme)

We can help in your project, please check PMB and our ratings/reviews to get idea of our experience.

in 7 gün içinde750$ USD
(249 Değerlendirme)

Quality is a great issue. Please have a look at our profile, reviews and portfolio ( [url removed, login to view] ) before selecting the winning bidder. Thanks, Tonmay

in 35 gün içinde1500$ USD
(130 Değerlendirme)

Hi I can develop the scrappers in perl, PHP and .net technologies. I have done many bots and scrappers using these technologies. Can send some sample apps also

in 18 gün içinde1000$ USD
(138 Değerlendirme)

Hello, I have placed bid on this project on GURU. Please have a look on that. Thanks, Maulik

in 20 gün içinde750$ USD
(14 Değerlendirme)

Hi, please see PMB.

in 10 gün içinde1000$ USD
(9 Değerlendirme)

I am Ranjana from Sisoft Info Solutions, Delhi NCR, India ([url removed, login to view]). We are a team of highly skilled professionals including project managers, domain experts, designers, analyst programmers and quality assuranc Daha fazlası

in 20 gün içinde1200$ USD
(14 Değerlendirme)

Please check my PM.

in 30 gün içinde1750$ USD
(12 Değerlendirme)

Hi, I can do this with .Net C# programming language, "Html Agility Pack" HTML parser and ASP.Net MySql/MSSql for a admin area. Regards... Dmitry

in 30 gün içinde1400$ USD
(6 Değerlendirme)

Please check the pmb. ------------------------ 1. C++ Socket 2. Depend on the internet speed, in high-speed, no problem within 1 seconds

in 10 gün içinde1000$ USD
(3 Değerlendirme)

Dear Sir, Please check PMB Thank you

in 22 gün içinde1200$ USD
(10 Değerlendirme)

We can help you on this.

in 60 gün içinde1500$ USD
(3 Değerlendirme)

Hello, I could do it in Python in no time at all.

in 6 gün içinde750$ USD
(6 Değerlendirme)

please contact us, we can finish this job for you,thanks

in 30 gün içinde1200$ USD
(1 Değerlendirme)

Hi, i am ready to start ur projects. Please check ur PM. Best regards...

in 3 gün içinde1100$ USD
(3 Değerlendirme)

Please check PM

in 60 gün içinde1000$ USD
(5 Değerlendirme)

1. I have implement such a software in VC (I think there is the fastest than any other tools.), which is named webpig. You can download it from [url removed, login to view], the forum is http://bbs.webpig.o Daha fazlası

in 30 gün içinde1500$ USD
(0 Değerlendirme)

Hi, I am keen to work on this project and if the work progresses well, i can involve couple of my friends also to completed the deliverables on time and focus on Quality and Optimization of the code. 1. What t Daha fazlası

in 35 gün içinde900$ USD
(0 Değerlendirme)

Saw you project and it looks like we can do this for you probably in less time. We are from the States so communication should not be an issue and we can PM you the tech proposal right away.

in 20 gün içinde1300$ USD
(0 Değerlendirme)

Please check PM

in 10 gün içinde900$ USD
(0 Değerlendirme)