
In Progress
Posted
Paid on delivery
Build a Job Scraper from Official Company Career Portals Project Overview I'm looking for a developer to build an automated job scraper that collects software engineering and AI-related job openings directly from companies' official career websites across the United States. The goal is to discover new job postings as early as possible, consolidate them into a structured database, and provide accurate job details with direct links to the original application pages. Job Categories Software Development Engineer (SDE) / Software Engineer Data Engineer AI / Machine Learning Engineer Generative AI / LLM Engineer RAG Engineer / AI Agent Engineer Requirements Scrape jobs directly from official company career portals, not just LinkedIn, Indeed, or other job boards. Support multiple career systems, including Workday, Greenhouse, Lever, Ashby, SmartRecruiters, iCIMS, and custom company career websites where feasible. Prioritize major US tech companies such as Google, Amazon, Meta, Microsoft, Apple, NVIDIA, OpenAI, and well-funded startups. Collect US-based jobs only, including US remote positions. Capture job title, company, location, work arrangement, description, salary when available, original posting date when available, first-discovered timestamp, and direct application URL. Automatically run on a schedule and detect newly posted, duplicate, updated, and expired listings. Leave unavailable information blank rather than inventing or incorrectly inferring it. Use permitted APIs, feeds, and compliant scraping methods. Deliverables Working scraper with source-specific connectors. Structured job database or export. Automated scheduled collection. Deduplication and job-status tracking. Setup instructions, source documentation, and complete source code. A demonstration using real job listings from multiple official company career portals. I'm looking for a reliable, maintainable scraper with real data collection—not a frontend mockup or a scraper limited to a few job boards.
Project ID: 40783245
81 proposals
Remote project
Active 7 hours ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs

I can build a scheduled scraper that pulls new US software, data and AI openings from company career sites into one database, with each listing's original apply link attached. The hard part is that each careers page is its own system, and some load their jobs with JavaScript, so I first check which sites publish a feed and which need a rendered page. My REST API to database sync item on my Freelancer profile does paginated pulls with retries and incremental sync, which is close to what these connectors need. I am free to start today. I would work in this order: 1. Check which target sites serve a public feed and which need a rendered page. 2. Build connectors for the first two or three companies you pick, with the apply link as the unique key. 3. Add the schedule, new, updated and expired status tracking, and the CSV export. Check the listings against the live career pages before you pay for the rest. - 50 USD for steps 1 and 2: a database of live listings from your first companies, ready the same day you award. - 100 USD for step 3, the scheduled runs, status tracking and export, once the whole job is done. Other career sites are quoted after the first run.
$150 USD in 7 days
4.5
4.5

$0 USD in 3 days
6.2
6.2
81 freelancers are bidding on average $117 USD for this job

Scraping directly from Workday, Greenhouse, Lever, Ashby, and custom portals is the core challenge here, and each system needs its own connector since they expose data differently. I'd build this in Python, using API endpoint harvesting where available (Greenhouse and Lever have public JSON feeds) and Playwright-based scraping for Workday and custom portals. Each job record gets a hash on URL plus title plus company for deduplication, with APScheduler handling recurring runs. New, updated, and expired statuses are tracked each cycle. Results go into SQLite or Postgres with all listed fields, leaving salary and posting date blank when the source doesn't provide them. One question: do you have a target company list, or should I define the initial sources based on your job categories? Azeem Amin
$250 USD in 9 days
9.0
9.0

A reliable job scraper depends on robust web scraping and automation—not a frontend mockup—and that is a core part of my work at A4Logic. Since 2011, I have built scalable web, AI, automation, API, and SaaS solutions, including systems that collect and organize data from online sources. My Python and web-scraping expertise is well suited to building source-specific connectors for Workday, Greenhouse, Lever, Ashby, SmartRecruiters, iCIMS, and feasible custom career portals. I can structure the collected listings with accurate job fields, direct application URLs, discovery timestamps, and blank values where information is unavailable. My experience with REST APIs, third-party integrations, MySQL, PostgreSQL, dashboards, and cloud deployment also supports scheduled collection, deduplication, update detection, and job-status tracking. I focus on clean, maintainable code and can provide the complete source code, setup documentation, source notes, structured database/export, and a real multi-portal demonstration. I would be glad to build this official-career-portal scraper for you.
$150 USD in 3 days
8.6
8.6

Reposted reqs under new IDs and "Remote" roles that are not US-eligible will quietly pollute the feed; both get caught before they reach your database. Day 1: a live sample of a few hundred listings from several portals so you can check fields, category tags and location filtering. Day 2: full delivery running on its schedule, with code and setup notes. 1. Preferred storage: Postgres, SQLite, or CSV/JSON export? 2. Do you have a starting company list, or should we seed one (roughly how many)? We can start today and walk you through the sample in chat.
$70 USD in 2 days
8.1
8.1

Building a reliable job-scraping system requires strong backend development and data handling, and Python is one of my core skills for creating automated software solutions. I can develop a maintainable scraper focused on official company career portals, with support for structured data collection, direct application URLs, and US-only filtering. My experience across custom software, SaaS platforms, API integrations, databases, and automation enables me to handle source-specific connectors for systems such as Workday, Greenhouse, Lever, Ashby, SmartRecruiters, iCIMS, and feasible custom portals. I can structure the collected fields you need, preserve unavailable values as blank, and implement deduplication, update detection, expiration tracking, scheduled collection, and export/database storage using Python, SQL, MongoDB, REST APIs, or PHP where appropriate. I will provide complete source code, setup instructions, source documentation, automated collection, and a real demonstration using listings from multiple official portals. Choose me to build a practical, maintainable scraper rather than a frontend mockup.
$140 USD in 7 days
7.7
7.7

Scraping official company career portals directly — before listings surface on LinkedIn or Indeed — is exactly what this project requires, and I have built portal-specific connectors for Workday, Greenhouse, Lever, and Ashby before. Here is my execution plan: Portal-Specific Connectors: Dedicated scrapers per ATS using BeautifulSoup and Playwright, because Workday renders job data client-side via XHR calls while Greenhouse exposes a public JSON API — a single shared scraper fails both. Structured Job Database: Every listing stored with title, company, location, work arrangement, salary, original posting date, first-discovered timestamp, and direct application URL. Fields with no available data stay blank, never inferred. Deduplication and Status Engine: Hash-based deduplication flags new, updated, and expired listings automatically on every scheduled run, so you always know which postings are genuinely fresh. Scheduled Automation: Cron-based scheduler with per-cycle logs, configurable per portal frequency. One problem worth raising upfront: several custom career pages at companies like Apple and NVIDIA actively block headless browsers. I will handle this using rotating user agents and request pacing to stay compliant and avoid being blocked silently. Delivery in 7 days, USD 120, with a live demo pulling real listings from at least five official portals.
$120 USD in 7 days
7.1
7.1

Hi, I’ve read your project description and understand that you need help with a maintainable scraper that collects US software engineering and AI job openings directly from official company career portals. I can help you achieve this by building source-specific collection logic, structured exports/database output, scheduled runs, and reliable deduplication and status tracking. I have experience in PHP, JavaScript, Node.js, API development, automation, and fixing a custom Google search scraper with complex regex logic. I focus on clear communication, attention to detail, and delivering work that meets the client's expectations. My approach: - Review target companies, ATS platforms, and permitted collection methods. - Build connectors for the agreed official career portals and normalize title, company, location, description, salary, dates, and application URL. - Add scheduled collection, first-discovered timestamps, duplicate/update detection, and expired-listing tracking. - Provide source code, setup instructions, source documentation, and a demonstration with real listings. I’d be happy to discuss your project and get started as soon as possible. Could you please clarify which company portals or ATS platforms should be included in the first delivery phase? Thank you for your time. I look forward to working with you! Best regards, Teo
$250 USD in 21 days
7.2
7.2

Hi, I can build an automated job scraper using Node.js, REST APIs, and a structured database to collect software engineering and AI-related jobs directly from official company career portals. Key features: - Connectors for Greenhouse, Lever, Ashby, Workday, and other supported sources - Scheduled job collection and new-posting detection - Deduplication and tracking of updated or expired listings - Job details, salary when available, discovery timestamps, and original application links - US-only job filtering, including remote positions - Database/export, setup documentation, and complete source code I’ll prioritize permitted APIs and compliant collection methods, leaving unavailable fields blank rather than guessing. The architecture will be modular so additional career portals can be added easily. I’d first confirm the target company list and available source interfaces, then provide a realistic timeline and milestone-based plan. Looking forward to discussing the project! Thanks.
$500 USD in 12 days
7.1
7.1

Hi, I've reviewed your project details and have a clear understanding of what you're looking to achieve. I'm confident I can help bring your project to life with a strong focus on quality, functionality, and timely delivery. My approach is simple. I'll start by understanding your exact requirements, then work through the project step by step, keeping you updated throughout the process. I'll also make sure your feedback is incorporated so the final result meets your expectations. I do have a few questions to clarify the scope and ensure we're on the same page before getting started. I'd be happy to discuss the details with you and explore the best way to move forward. Looking forward to hearing from you!
$30 USD in 2 days
6.7
6.7

I will build a robust job scraper that targets official company career portals to collect software engineering and AI-related job openings across the United States. The scraper will be designed to automate the collection of job details including title, company, location, and direct application links, ensuring that we capture the most current listings while also supporting multiple career systems. I will incorporate deduplication and job-status tracking to maintain a clean and structured job database. With my experience in Python and web scraping using BeautifulSoup and other tools, I will ensure that the solution is reliable and maintainable. Additionally, I will provide clear setup instructions and documentation, along with a demonstration of the scraper using real job listings from various portals. My work is backed by a 5.0 Freelancer rating and 111 client reviews. Clients have described me as reliable, detail-oriented, responsive, and professional. They've praised my clear communication, quick turnaround, accurate results, and ability to understand requirements and solve difficult problems. One long-term client has worked with me for over three years and highlighted my trustworthiness and willingness to go above and beyond. Could you clarify if you have specific companies in mind for the initial scraping? Also, do you have a preferred database format for the job listings? Best, Hussein
$140 USD in 5 days
6.5
6.5

Hello! I will create a PHP script to scrape data you need Please provide the details I have extensive experience in writing PHP scripts for data scraping Please see my reviews for reference.
$190 USD in 2 days
6.7
6.7

The biggest risk here isn't scraping, it's drift. Greenhouse, Lever and Ashby expose public JSON endpoints, but Workday and iCIMS change payloads and session tokens often, so one generic parser will quietly break and miss new postings. Quick questions - how many companies should v1 cover at launch? And should results land in PostgreSQL or a plain CSV export? Here is the architectural approach: - WEB SCRAPING: Source connectors for Workday, Greenhouse, Lever, Ashby, SmartRecruiters and iCIMS, plus B
$130 USD in 3 days
6.4
6.4

Hello, Problem: Collecting job listings from official career portals manually is time-consuming and makes it difficult to maintain accurate, up-to-date job data. Solution: I will develop a Python-based job scraper to collect publicly available listings from supported career portals, including job titles, company names, locations, descriptions, and application links. I will implement structured data storage, duplicate detection, error handling, and scheduled scraping where permitted by each portal's terms and technical restrictions. Result: You will receive organized, searchable job data with a maintainable scraping workflow that reduces manual effort. With 20 years of software development experience, I build reliable automation tools and data-driven applications. 1. Which official career portals should be included? 2. Do you prefer CSV, Excel, or database storage? Thanks
$130 USD in 7 days
6.0
6.0

I can build a maintainable job scraper that collects software engineering and AI-related openings directly from official company career portals, with source-specific connectors for platforms such as Greenhouse, Lever, Ashby, and Workday where permitted access is available. I’ll store job details in a structured database, including title, company, US location, work arrangement, description, salary and posting date when available, first-discovered timestamp, and original application URL. The system will run on a schedule, detect new and updated listings, remove duplicates, and mark expired jobs without inventing missing information. I’d use Python for the collection pipeline, PostgreSQL for structured storage, and scheduled workers for automated updates. I’ll also provide source documentation, setup instructions, complete source code, and a demonstration using real listings from multiple official career portals. Which companies should be included in the initial release, and how frequently would you like the scraper to check for new jobs? Best regards, Amarjeet
$99 USD in 5 days
6.1
6.1

My PHP experience makes me well suited to build the server-side foundation for your official career-portal scraper, including structured job collection, database storage, scheduled processing, and status tracking. I can develop a practical backend that keeps original application URLs and preserves unavailable fields as blank rather than making unreliable assumptions. My broader web development background includes HTML, CSS, JavaScript, WordPress, Shopify, and custom website work, giving me a solid understanding of varied web structures and responsive career portals. I can apply that experience to source-specific connectors, clean job records, duplicate detection, and a maintainable export or database—focused on real listings from official company sites rather than a frontend mockup. I am ready to deliver complete source code, setup documentation, source notes, automated collection, and a demonstration using listings from multiple portals. Choose me to build a reliable PHP-based job scraping solution for your project.
$100 USD in 6 days
5.4
5.4

Hello I can build a Python-based job scraper using official company career portals and APIs, with support for multiple ATS platforms, automated scheduling, deduplication, new and expired job tracking, and structured CSV/SQLite exports. I’ll provide clean source code, setup instructions, and a demonstration using real US-based software and AI job listings. Regards Muhammad
$100 USD in 1 day
5.5
5.5

Hi, I can develop a Python-based automation system that targets official company career portals like Workday, Greenhouse, and Lever to capture SDE and AI job listings. This solution will focus on bypassing generic job boards to fetch direct application URLs and structured data specifically for US-based roles. Using Python and BeautifulSoup, I will implement source-specific connectors for major tech companies and common ATS platforms. I will structure the system to handle deduplication and track job status changes automatically. My background in Python and Java allows me to build a maintainable architecture that handles the specific challenges of dynamic career sites while maintaining clean data. As a first step, I will map the specific URL patterns for the primary ATS providers mentioned to ensure the scraper identifies new postings early. Do you have a preferred database type, such as PostgreSQL or a simple JSON export, for the initial data storage? Let's discuss how to prioritize the initial list of tech companies for the first demonstration.
$130 USD in 10 days
5.1
5.1

A reliable scraper for official career portals requires strong backend development and disciplined data handling, and PHP is the core skill I offer for building this type of automated system. With more than 15 years in the technical field, I can develop a maintainable solution focused on real data collection rather than a frontend mockup. I can build the PHP-based collection and database layer to support multiple portal connectors, capture the requested job fields, preserve direct application URLs, and leave unavailable values blank. The system can also include scheduled runs, duplicate detection, first-discovered timestamps, and tracking for new, updated, and expired listings, with source documentation and complete source code included. My approach emphasizes clear communication, quality, and continued support after delivery. I would be glad to build and demonstrate a dependable official-portal job scraper that you can maintain and extend over time.
$30 USD in 7 days
5.1
5.1

You miss out on new US AI and software job postings because they only hit job boards after they're on company career sites. You'd spot openings at Google, Amazon, Meta and more the moment they're live, with every job detail and direct application link ready to go. First I confirm the exact fields you need captured for SDE, Data Engineer and AI roles. Then I build connectors for Workday, Greenhouse, Lever and custom portals, with deduplication and status tracking. Finally, you get a database plus setup notes so you can run scheduled scraping and see real listings. You'll get clear updates as I add each company source, so you always know what's covered. Want jobs from any company not on your list, or stick to the big names?
$95 USD in 14 days
4.8
4.8

Python job scraper for official career portals (Workday, Greenhouse, Lever, Ashby and more) with scheduled runs. You want new US engineering and AI roles found early straight from company career sites, with accurate data and no guessing. My approach: a Python pipeline with one connector per system. Greenhouse, Lever, Ashby and SmartRecruiters have public job feeds, Workday needs its career-site JSON calls, and custom sites get Playwright or BeautifulSoup parsers where terms allow. Jobs go into PostgreSQL with title, company, location, description, salary, dates and application URL, blank when unavailable. A scheduler would detect new, duplicate, updated and expired listings and keep US roles only. I have 9 years of full-stack experience with Python, Node.js, React.js and AI/ML. I built Jyotish AI, a live AI app on Android and web. iCIMS and bespoke sites vary a lot, so I would add them gradually. My price of USD 110 is for a first phase of three or four connectors with the database and scheduler, and the final quote depends on confirming the company list. Which companies matter most first, and do you want a dashboard or CSV export? Happy to chat here on Freelancer.
$110 USD in 14 days
4.6
4.6

Hello, I checked your project "Official Career Portal Job Scraper" and already have a clear understanding of your requirements and how to deliver them efficiently. I have solid experience in PHP, Python, and I've successfully completed similar projects by delivering high-quality, scalable, and reliable solutions. Why choose me? * Strong expertise in PHP, Python * Clean, optimized, and scalable implementation * Fast communication and regular progress updates * Focused on delivering results, not just completing the task I'm ready to start immediately and would be happy to discuss the details. Best regards, Umer
$40 USD in 1 day
4.3
4.3

Hyderabad, India
Payment method verified
Member since Apr 6, 2020
₹600-1500 INR
£250-750 GBP
₹1500-12500 INR
₹75000-150000 INR
₹750-1250 INR / hour
$250-750 USD
$25-50 USD / hour
$250-750 USD
₹12500-37500 INR
$15-25 USD / hour
$30-250 USD
₹600-1500 INR
₹1500-12500 INR
$10-30 USD
$250-750 USD
$250-750 USD
₹600-1500 INR
€250-750 EUR
₹1500-12500 INR
€8-30 EUR