This will be several projects. We would like to hire a team or someone that can work with a team to do different parts of this project. We currently have code for this project in PHP, but would like to revamp or redo the entire application. This will be a ongoing project. These are the different parts of the project in a general overview:
1. We need an application that will be able to crawl different websites. We will need to login in to the website, scrape data from the website, and depending on the data that we gather, submit certain data to the website.
- This can be be in any language, but we like this to be fast. Currently, from the time it finishes parsing a page to submit the data, it takes 3 seconds. We would like to lower this time to about 1 second. We would like someone who can optimize the application and code and identify bottle necks.
- We might want multiple crawler instances to be running at once so would like someone familiar with threading and process forking.
- This will need to be maintainable, so might want this to be in OOP. We will want to to be able to add different sites to scrape to our application easily. Might want to have like a plugin system to easily plug in new sites.
- We want someone that can start a project from beginning and also be able to work with existing code and integrate their implemenation elegantly.
2. We need an admin area, preferably web based. The admin area will manage our application and should allow us to run our application and see output from our application.
3. This application will be database driven. Data for the sites we are parsing will be stored in the database. Will need someone or team that will have experience in good database design. The database will need to be easily maintainable and extensible.
4. This application will be run across different servers for different clients. This data will be stored in the database to be accessed by the application.
Desire a team/individual with experience in:
- Many different languages
- Database Design
- OOP and Design Patterns
- Process forking/threading
- Performance optimization
I will need to have access and the final rights and ownership to the source code.
Please reply with:
1. What technologies you will do this in
2. If you think you can develop the scraper/crawler to parse a page and submit to 2 pages in 1 second or less
Bu iş için 25 freelancer ortalamada $1164 teklif veriyor
We can help in your project, please check PMB and our ratings/reviews to get idea of our experience.
Quality is a great issue. Please have a look at our profile, reviews and portfolio ( [url removed, login to view] ) before selecting the winning bidder. Thanks, Tonmay
Hi I can develop the scrappers in perl, PHP and .net technologies. I have done many bots and scrappers using these technologies. Can send some sample apps also
Hello, I have placed bid on this project on GURU. Please have a look on that. Thanks, Maulik
I am Ranjana from Sisoft Info Solutions, Delhi NCR, India ([url removed, login to view]). We are a team of highly skilled professionals including project managers, domain experts, designers, analyst programmers and quality assuranc Daha Fazla
Hi, I can do this with .Net C# programming language, "Html Agility Pack" HTML parser and ASP.Net MySql/MSSql for a admin area. Regards... Dmitry
Please check the pmb. ------------------------ 1. C++ Socket 2. Depend on the internet speed, in high-speed, no problem within 1 seconds
please contact us, we can finish this job for you,thanks
Hi, i am ready to start ur projects. Please check ur PM. Best regards...
1. I have implement such a software in VC (I think there is the fastest than any other tools.), which is named webpig. You can download it from [url removed, login to view], the forum is http://bbs.webpig.o Daha Fazla
Hi, I am keen to work on this project and if the work progresses well, i can involve couple of my friends also to completed the deliverables on time and focus on Quality and Optimization of the code. 1. What t Daha Fazla
Saw you project and it looks like we can do this for you probably in less time. We are from the States so communication should not be an issue and we can PM you the tech proposal right away.