Data collection / web page scraping

We need someone to create a script that will extract data from a few search and shopping engines for us. We need to scrape the top engines that have paid listings on a set of keywords in order to gather certain information. The same process will be used to gather this information from the shopping engines. This information collected will be extracted into a database. Project outline described briefly below.

A brief description of the requirements are as follow:

We need a program that can scrape the following information (title, description, URL, and placement (1st , 2nd, etc) for paid or sponsored listings from 5 different search engines, and 3 shopping engines. The program will grab the keywords that will be searched from a database or excel file. The program needs to multi-tread the queries to be efficient. The program will run (pulling from a list of keywords) until all the keywords have been exhausted. Keywords may be batched; IE 5000 keywords per day or something like that. The program needs to handle at least 5000 keywords per day. When complete with the entire list, it will start over from the beginning.

The results from the data collection / scrapping will be added to a database. The results from the scraping will need to be processed by the database.


I have attached an excell files(named [url removed, login to view]) which will show you what the output will include.

A single URL might have thousands of keywords. Along with the associated titles, descriptions, and placement.. We will need some reporting created to properly analyze the data.

About us.

We a online advertiser with thousands of keywords (keywords change and are added to daily). We want to see what other advertisers in our pace are doing, in order to be competitive.

This project needs to be completed ASAP

Beceriler: .NET, Java, Perl, Visual Basic

Daha fazlasını görün: we excell, web search engines list, web scraping process, web page scrapping, top data, to create web page, set data, search files on web, scraping data from web database, run java program online, online advertisers, online advertiser, multi program online, java start web, i want to create web page, excell program, data process online, data 0, daily advertiser, competitive set example, advertisers online, java data collector, webpage data collection, perl webpage scraping, scrape webpage perl

İşveren Hakkında:
( 7 değerlendirme ) Las Vegas, United States

Proje NO: #89459



Hello, Please look at the PMB. Thanks, Sergey

5 gün içinde 300$ USD
(26 Değerlendirme)

12 freelancers are bidding on average $205 for this job


I can do with Perl. I can create demo for you

in 14 gün içinde270$ USD
(116 Değerlendirme)

Please check PMB

in 2 gün içinde100$ USD
(0 Değerlendirme)

i do it many time

in 30 gün içinde100$ USD
(0 Değerlendirme)

I have already created a content harvestor kind of a program and is very much succeeded using perl. I think it will not take a lot of time for me and can meet the expected in this schedule.

in 7 gün içinde100$ USD
(0 Değerlendirme)

i have already worked on this project.. please contact me for further details

in 5 gün içinde100$ USD
(0 Değerlendirme)

Please see the pmb.

in 0 gün içinde190$ USD
(0 Değerlendirme)

Lets Start, Please see us in PM so that we can discuss about project and our experience

in 15 gün içinde200$ USD
(0 Değerlendirme)

Ready to start immediately on your project, as we are the experts in such things. Waiting for your selection. Thanks!

in 7 gün içinde300$ USD
(0 Değerlendirme)

We are web design ,graphic design and website development company comprises a pool of professionals who deliver quality work on a wide range of communication web site design and web application projects. The web desi Daha fazlası

in 8 gün içinde250$ USD
(0 Değerlendirme)

Dear Sirs, This project seems to be very interesting for us, please see PM.

in 15 gün içinde300$ USD
(0 Değerlendirme)

Hi, I have 5 years experience in VB,C#,ASP.NET,especially in VB. Just before several days I had left my original outsourcing IT company in SZ in order to have further time to pursuit personal career target, I can conce Daha fazlası

in 6 gün içinde250$ USD
(0 Değerlendirme)