
Closed
Posted
Paid on delivery
I am looking for an experienced developer to create a system that uses an existing API (already developed and tested) built on Apify to scrape real estate ads from [login to view URL] and store them in a MySQL database. The project involves the following main components: 1. API Integration: The API has been custom-built for scraping real estate data from immobiliare.it. I will provide the specific search URLs (e.g., Example URL) that serve as input for the scraping. The system should automate scraping daily, ideally in the morning, to ensure consistent data collection. 2. MySQL Database Setup: The scraped data must be stored in a MySQL database hosted on a server I will provide (possibly on a low-cost service like Aruba, but I am open to recommendations). Key requirements:Add only new ads each day (based on unique IDs assigned by [login to view URL]). Maintain historical data to build a long-term archive of ads. Avoid duplicate entries. 3. Data Validation: Include a mechanism to check whether older ads are still active online. If an ad is no longer available, calculate how many days it was online (from the date it first appeared) as an indicator of market dynamics. 4. Web-Based Query Interface: Develop a secure, password-protected web interface to: Query the database and filter data based on my criteria (e.g., filtering by price range, area, number of rooms, etc.). Apply calculations such as percentiles or averages on specific subsets of data (e.g., "Calculate the price corresponding to the 85th percentile of prices per square meter in a specific microzone"). Download query results in CSV or Excel format. 5. Hosting and Maintenance: The system and database should run on a server that is always connected to the internet. Include basic guidance or documentation for deployment on the server. 6. Future Considerations: Flexibility to add new functionalities or adjust existing features as needed. Deliverables: Fully functional system hosted on the provided server. MySQL database with daily data updates. Web-based interface with querying and download capabilities. Basic documentation for usage and maintenance. Required Skills: API integration (preferably experience with Apify). MySQL database design and optimization. Backend development for automated scraping and data validation. Frontend development for the web-based query interface. Experience with hosting setups (e.g., on services like Aruba). Budget & Timeline: Please provide your best estimate for costs and delivery time. Notes for Applicants: Be prepared to discuss your approach to data deduplication, historical tracking, and real-time performance monitoring.
Project ID: 38847037
53 proposals
Remote project
Active 1 yr ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
53 freelancers are bidding on average €1,212 EUR for this job

With over a decade of experience in Full Stack Development, I have a proven track record of successfully handling challenging projects. I've completed over a thousand projects for organizations of various sizes, specializing in WordPress, Mobile App Design, SAAS Development, and Brand Identity. I am committed to delivering high-quality work and exceptional support to build long-term relationships. Let's discuss your project requirements and bring your ideas to life efficiently.
€1,350 EUR in 5 days
8.7
8.7

Hi sir, This is Lin and I am scraping expert, I scraped 600+ websites, please check my reviews then you will know. I will create scraping tool with PHP & CURL that extracts all requested data, And provides data in the required format (JSON, CSV, EXCEL, SQL...). Can we discuss more details about this project? Thanks, Lin
€1,125 EUR in 7 days
8.5
8.5

Greetings! I'm Adel, an expert in web scraping and data extraction. With years of experience in building robust scraping systems, I'm confident in delivering a tailored solution that meets your requirements. I'll leverage my expertise with Apify to seamlessly integrate the existing API, ensuring efficient and reliable data collection from immobiliare.it. The system will be designed for daily automated scraping, with intelligent deduplication mechanisms to maintain a comprehensive historical archive while avoiding redundancies. Regarding the MySQL database, I'll optimize the schema for efficient storage and querying, incorporating data validation checks to track ad lifetimes accurately. The web interface will provide a user-friendly experience for querying and filtering data based on your criteria, with export capabilities for further analysis. To ensure a smooth deployment, I'll provide comprehensive documentation and guidance for hosting and maintenance on the provided server. Additionally, I'll build in flexibility to accommodate future enhancements or adjustments as needed. I'm excited to discuss my approach in more detail. Could you please clarify the expected data volume and any specific performance requirements? This will help me better estimate the project timeline and ensure optimal resource allocation.
€1,200 EUR in 7 days
7.6
7.6

With over 7 years in the field, I am well-versed in API Integration, MySQL database design, backend automation and frontend development - all the right skills for your real estate data project. My previous experience with Apify makes me a strong contender, as I already have knowledge of extracting data from platforms like immobiliare.it. In addition, my solid grasp on Python gives me an edge while working with established tools like Beautifulsoup and Selenium for effective web scraping. Setting up a foolproof MySQL database is crucial and I assure you that maintaining data integrity will be a top priority; no duplicates will find their way into the system. Furthermore, my extensive knowledge of data analysis using libraries such as Pandas, Seaborn and Numpy ensures that your data not only will be archived efficiently but presented back to you in an understandable manner through insightful calculations and variables like percentiles as you’ve mentioned. Lastly, my hosting and maintenance skills would come into play here too. I am experienced in setting up servers and deploying systems for constant online connection. Plus, my quick responsiveness to queries or tweaks in the future would ensure that your tool remains current. Partnering with me means not just getting the job done skillfully but also laying the foundation of a long-term professional bond.
€1,125 EUR in 7 days
7.6
7.6

Hi, my name is Puru. I read your `Real Estate Data Scraping and Analysis Tool` project description carefully before bidding. I got what you need and ready to go ahead as soon as we can clarify further project details. With over 8 years of expertise in PHP, Python, Web Scraping, MySQL and API Integration, I'm confident in delivering a tailored software solution to meet your needs. As a top 1% freelancer, I've consistently achieved 5-star ratings from 100+ clients, reflecting my commitment to quality and client satisfaction. Check out my portfolio: https://www.freelancer.com/u/Guptapuru304 to see similar examples of my work. Kindly get back to me if you wish to proceed with a top-rated and reliable partner here. I deliver best quality at a reasonable price, and I am open for longterm partnerships. Kind Regards, Puru
€1,500 EUR in 8 days
7.7
7.7

Hello, good time Hope you are doing well I'm expert in MATLAB/Simulink, Python, HTML5, CSS3, Java, JavaScript and C/C#/C++ programming and by strong mathematical and statistical background, have good flexibility for solve your project. I have many experience practical and theoretical in implementation different algorithms (such as: state estimation and Kalman filter, design controller, analysis closed loop stability, signal and systems, signal processing, heuristic optimization, fuzzy logic, neural network and machine/deep learning fields). Evidence of this claim exist in the portfolio. I have read your project description and I can help you (without any plagiarism). Please send me the details of your project. Thanks for attention 100% Jobs Completed, 100% On Budget, 100% On Time ⭐⭐⭐⭐⭐ 5-star reviews
€1,125 EUR in 7 days
5.7
5.7

With over a decade of experience in web application development, my proficiency extends to PHP and Python, two crucial languages in realizing your project needs. My previous work with real estate data analysis with relational databases (specifically MySQL) will certainly be valuable and coincides perfectly with your requirement. Moreover, I possess distinctive skills in API integration, having specifically worked with Apify in the past, and a demonstrated clear understanding and competency with e-commerce platforms like eBay and Amazon that should be applicable to this project. In the realm of scraping and automation, I employed both PERL and Python scripting. I can bring forth this vast experience to automate the scraping process for your specific URLs, saving you valuable time and ensuring consistent data collection on a daily basis. Additionally, my understanding of efficient backend development will enable me to make sure that just new ads are added each day while avoiding duplicate entries, as stipulated. I look forward to working together!
€1,225 EUR in 15 days
5.9
5.9

Nice to meet you MRR1968,The requirements of your project match my areas of work and skills, to introduce myself. My name is Anthony Muñoz and i am the lead engineer for DS Pro IT agency. I have worked for over 10 years as a Full-Stack and software development engineer and have successfully done multiple jobs. It will be a pleasure to work together to make your project. Feel free to discuss about the project with me, greetings.
€2,114 EUR in 7 days
5.5
5.5

Hey Hope this message finds you in the best of tech-savvy spirits! As a web scraping specialist and data scientist, I bring together a diverse set of technologies and tools to extract valuable data from the web efficiently and effectively. My expertise encompasses Python-based libraries like BeautifulSoup, Scrapy, and Selenium, ensuring that I can adapt to various scraping challenges and requirements. Additionally, I have experience with data storage and processing technologies such as SQLite, MongoDB, and Pandas, allowing me to handle, organize, and analyze the scraped data with precision. Whether it's e-commerce data, news articles, social media content, or any other web source, I have the technical prowess to craft custom scraping solutions that deliver clean, structured data for your specific needs. Let's collaborate to harness the power of these technologies and create a web scraping solution that provides you with the valuable insights you seek.
€1,125 EUR in 7 days
4.7
4.7

As an experienced developer, I am confident that I am the best fit for your Real Estate Data Scraping and Analysis Tool project. With over two decades in the field, I have amassed a versatile skillset that touches on every level of your requirements. Starting from my proficient command in API integration -- including in-depth experience with Apify, through to my MySQL expertise, which extends to designing and optimizing databases, I ensure a robust and scalable solution for your data storage needs. Additionally, my strong background in backend development using Python will enable me to effectively automate the scraping process while ensuring data integrity and avoiding duplicates. Conversely, my grasp of front-end technologies such as PHP will facilitate the creation of a user-friendly web interface for efficient data querying and downloading. Notably, with hosting setups being one of my core competencies, I assure you a reliable system deployed on your provided server and that stays connected to the internet at all times. With basic documentation that I provide for all projects I deliver, I will also ensure the system remains easy to maintain long after project completion should you require future adjustments or new functionalities. My approach is focused on producing solutions that are high-quality, durable, and tailored specifically to meet the unique needs of your business.
€3,000 EUR in 30 days
4.2
4.2

Hello, ✵✬✭✮ As senior scraper developer, I can complete your project perfectly!✵✬✭✮⚝ I can finish right now. I saw your post and understood it clearly. I can get this done at short time, because i have a lot of experiences in python development(14 years). I have developed exact similar project from australian employer. Please send me message, so i am hoping to discuss with you for this project. Waiting for your reply. Best regards. Lizi
€750 EUR in 7 days
4.2
4.2

Hi, my name is Osama Khan. I read your `Real Estate Data Scraping and Analysis Tool` project description carefully before bidding. I got what you need and ready to go ahead as soon as we can clarify further project details. With over 8 years of expertise in PHP, Python, Web Scraping, MySQL and API Integration, I'm confident in delivering a tailored software solution to meet your needs Kindly get back to me if you wish to proceed with a top-rated and reliable partner here. I deliver best quality at a reasonable price, Kind Regards, OSAMA KHAN
€1,525 EUR in 30 days
4.9
4.9

Hello, I am very happy to work for you. First of all, I am sure I will complete this project within 3 days. Here is a brief summary of the solutions I propose: 1. API Integration with custom search URLs. 2. MySQL Database setup with daily automated data collection & maintenance. 3. Data validation for active ads and market indicators. 4. Web-based query interface with advanced filtering and download options. 5. Hosting on a reliable server with deployment guidance. 6. Future-proofing for feature enhancements. I will wait for your contact.
€750 EUR in 3 days
3.1
3.1

Hello Marco Rino R. ❤️ Thank you sincerely for considering my bid! ❤️ I'm Michael,With over 8 years in PHP, Python, Web Scraping, MySQL, API Integration, I’m confident in delivering high-quality, effective solutions. I recently completed a similar project, so I’m ready to start right away and bring clean code and attention to detail to ensure top results. Please feel free to reach out—I’d love to discuss how I can contribute. Best Regards, Michael
€999 EUR in 7 days
3.0
3.0

Hello Marco, We went through your project description and it seems like our team is a great fit for this job. We are an expert team which have many years of experience on PHP, Python, Web Scraping, MySQL, API Integration Lets connect in chat so that We discuss further. Regards
€800 EUR in 7 days
0.0
0.0

Hello, https://www.freelancer.com/u/toscano97 I read your requirements fully and I am very interesting. I have rich experience in this field and I can do it. I implemented various project similar to this project in my past experiences, so I would like to grab this opportunity and will work till you get 100% satisfied with our work. Please come over chat and discuss your requirement in a detailed way. Thank you.
€1,200 EUR in 7 days
0.0
0.0

CATANIA, Holy See (Vatican City State)
Payment method verified
Member since Jan 14, 2013
€30-250 EUR
$2-8 USD / hour
€30-250 EUR
€8-30 EUR
€8-30 EUR
$250-750 USD
₹12500-37500 INR
£750-1500 GBP
₹1000-1500 INR
$10-30 USD
$30-250 USD
$10-30 USD
€3000-5000 EUR
$30-250 USD
£20-250 GBP
$250-750 USD
$30-250 AUD
₹1500-12500 INR
₹600-1500 INR
$30-250 USD
₹100-400 INR / hour
$25-50 USD / hour
₹600-7000 INR
$250-750 USD
₹600-1500 INR