Web Scraping Engine - 5 Sites Initially

We are launching a new aggregation service for specialty financial data. The service will work just like Yodlee but serve different data related to financial transactions. All sites are secured with credentials, which we will store, and some have challenge questions.

We need a web scraping engine developed that can operate in real time and in batch. The engine must be able to scale to an enterprise level and be multi-threaded. We will store the user credentials so the engine must be able to cycle through different credentials to gather the data for that user and populate a database. Some of the sites are not straight-forward and use Ajax and other technologies that make scraping more difficult. For this project we will want five sites scraped initially but will add many more if the engine works properly.

In short, we are looking for web scraping experts that can develop not only the initial product but also help maintain it and add sites. We can provide some initial site credentials as samples if you like.

We have developed a proof of concept using the technologies listed below but we are open to whatever works.

Selenium Webdriver





Entity Framework



HTML Agility Pack


Beceriler: Veri Madenciliği, Web Scraping

Daha fazlasını görün: yodlee help, webscraping service, web scraping c#, web scraping api, web framework ajax, data web scraping, data mining data scraping service, data challenge, concept challenge, challenge web, challenge sites, a web service for scraping, asp net scraping, asp.net experts, asp net experts, ajax web framework, webdriver selenium, web technologies 2014, mvc entity framework, entity framework mvc, asp.net mvc ajax, data mining service, Webscraping, web scraping service, web api

İşveren Hakkında:
( 2 değerlendirme ) Alpharetta, United States

Proje NO: #6523536

26 freelancer bu iş için ortalamada 569$ teklif veriyor


I specialize in web scraping jobs like this. You can see on my profile that I have completed many similar jobs. I can have this project done properly and in a timely manner.

in 3 gün içinde1578$ USD
(52 Değerlendirme)

Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi

in 6 gün içinde573$ USD
(98 Değerlendirme)

Hi.. Expert Web Scraper & Data Minor here. I have done too many similar project in past. Having best scraping tools and experience i assure you 100% accurate and good quality work. I have too too scraping experience. Daha fazlası

in 10 gün içinde555$ USD
(111 Değerlendirme)

I am an expert in crawling and willing to discuss in depth about the project details and deliver the same.

in 10 gün içinde526$ USD
(42 Değerlendirme)

hi, i am expert in web scraping and interested in this project, let me do this work with perfection, accuracy and according to your requirements, plz contact with me so we can discuss further about the project and plz Daha fazlası

in 10 gün içinde263$ USD
(86 Değerlendirme)

Hello Sir, I have much experience on web scraping and data mining . I have worked on many project using Selenium Webdriver, Phantomjs, C# and other scraper engine . I use selenium and Phantomjs to scrape data from Daha fazlası

in 10 gün içinde511$ USD
(54 Değerlendirme)

Hello, Thank you for inviting me to bi on your project. I have expertise in most of the technologies you've listed - and have vast experience is scraping off sites of differing structures and complexities. Do se Daha fazlası

in 10 gün içinde555$ USD
(15 Değerlendirme)

A proposal has not yet been provided

in 10 gün içinde555$ USD
(13 Değerlendirme)

I am an scrapy expert, and have many scrapy experience here. I am interested in your project and have another whole toolset for your project, please contact me through your PM and discuss more details for your project, Daha fazlası

in 10 gün içinde555$ USD
(9 Değerlendirme)

Hi, I can develop excel based VBA scrapper in any technology (Java or Ajax based), auto login and logout with pop-up data I have already developed gmail auto login crawler - please visit my profile Please shar Daha fazlası

in 10 gün içinde250$ USD
(3 Değerlendirme)

I'm interesting your project very well I'm a Good C/C++, C#, PHP, Web Scrap, Compiler-Compiler expert. I understand your req exactly. I m quite well experienced in these jobs, for example Fifa auto buyer etc. So Daha fazlası

in 7 gün içinde500$ USD
(4 Değerlendirme)

Hi, I have great experience in website data extraction. i have done the extraction of many sites like [url removed, login to view],[url removed, login to view],[url removed, login to view],[url removed, login to view],[url removed, login to view],[url removed, login to view] and many more i have read th Daha fazlası

in 10 gün içinde555$ USD
(6 Değerlendirme)

A proposal has not yet been provided

in 10 gün içinde555$ USD
(1 Değerlendirme)

I am a professional and an experienced academic and web market researcher, fully proficient in marketing and research writing styles. I can find the minuet information from the web; I have an excellent record of accomp Daha fazlası

in 10 gün içinde444$ USD
(8 Değerlendirme)

Hi there. I have a lot of experience with this sort of project. I've created a lot of robust multiprocess web automation console apps for everything from as simple as scraping data to complicated tasks on various socia Daha fazlası

in 10 gün içinde777$ USD
(2 Değerlendirme)

Hello Dear my name is Ibrahim Ragab I'm Professional English-Arabic translator , creative Arabic writer and Communication [url removed, login to view] I'm an administrative assistant specializes in all kind of data entry ser Daha fazlası

in 10 gün içinde388$ USD
(3 Değerlendirme)

I can parse 20K data profile or higher per day depending on the requirements. Besides, I can used Python APIs like BeautifulSoup, multiprocessing/Pool and urllib2 for efficient processing and fast turn around time for Daha fazlası

in 14 gün içinde555$ USD
(2 Değerlendirme)

​ ​​​​​​Current WEB SCRAPING Jobs:​​ ​[url removed, login to view] ​[url removed, login to view]​ ​[url removed, login to view] Daha fazlası

in 7 gün içinde561$ USD
(1 Değerlendirme)

I have 6 years of experience with data extracting and manipulating. The output for scraping process can be various from XML, CSV or database. I use Perl to do all of my scraping with LWP and HTMLUnit library.

in 10 gün içinde555$ USD
(0 Değerlendirme)

I am applying for this job because I just completed a Web scraping project that used Python and Selenium. For your task I would add a MySQL database that is going to get updated from the Python script. The tools I'm us Daha fazlası

in 10 gün içinde588$ USD
(0 Değerlendirme)