Track new job openings at specific companies without scraping Indeed or LinkedIn
Scraping Indeed or LinkedIn Jobs looks like the obvious way to watch who's hiring. In practice you fight Cloudflare challenges, logins, rate limits and HTML that changes every few weeks, and...
Résumé
Scraping Indeed or LinkedIn Jobs looks like the obvious way to watch who's hiring. In practice you fight Cloudflare challenges, logins, rate limits and HTML that changes every few weeks, and the listings you do get are often days old, reposted from somewhere else. That somewhere else is usually the company's own careers page, and most careers pages are powered by an applicant tracking system (ATS) that publishes a clean, public JSON feed. Watching those feeds directly is faster, more reliable and fully public. Step 1: find which ATS a company uses Open the company's careers page and look at where the "Apply" links point: If links go to... The ATS is Public feed boards.greenhouse.io/acme or job-boards.greenhouse.io/acme Greenhouse https://boards-api.greenhouse.io/v1/boards/acme/jobs jobs.lever.co/acme Lever https://api.lever.co/v0/postings/acme?mode=json jobs.ashbyhq.com/acme Ashby https://api.ashbyhq.com/posting-api/job-board/acme apply.workable.com/acme Workable https://apply.workable.com/api/v1/widget/accounts/acme jobs.smartrecruiters.com/Acme SmartRecruiters https://api.smartrecruiters.com/v1/companies/Acme/postings The part after the domain (acme) is the board slug. Step 2: poll the feed and keep only new jobs import json, pathlib, requests SEEN = pathlib.Path("seen.json") seen = set(json.loads(SEEN.read_text())) if SEEN.exists() else set() def greenhouse(slug): r = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs", timeout=30) for j in r.json()["jobs"]: yield j["id"], j["title"], j["location"]["name"], j["absolute_url"] new = [] for slug in ["stripe", "databricks", "notion"]: for job_id, title, loc, url in greenhouse(slug): key = f"gh:{slug}:{job_id}" if key not in seen: seen.add(key) new.append((slug, title, loc, url)) SEEN.write_text(json.dumps(sorted(seen))) for row in new: print(" | ".join(row)) Run it from cron once a day and you have a "new openings" alert for any list of companies, with no browser, proxies or logins. Step 3: scale past a handful of companies The manual approach breaks down when you want hundreds of companies across several ATSs, because every ATS has a different schema and you first need to know each company's slug. That's the problem we built ATS Jobs Search on Apify to solve. It covers seven ATSs (Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio), uses one output schema, has a built-in directory of about 7,900 companies' boards, and has an "only new jobs since last run" mode: from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") run = client.actor("siftwright_data/ats-jobs-search").call(run_input={ "searchDirectory": True, "keywords": ["data engineer", "analytics engineer"], "locations": ["Remote"], "onlyNewJobs": True, }) for job in client.dataset(run["defaultDatasetId"]).iterate_items(): print(job["companyName"], "|", job["title"], "|", job["url"]) It costs $1 per 1,000 jobs, and Apify's free monthly credit covers a lot of testing. Prefer a file or a ready-made feed? Everything at once: today's full snapshot (about 400,000 open jobs from 7,500+ companies, one CSV) is a $39 instant download. A daily niche feed (for example "remote data jobs" or "healthcare roles in Texas") delivered to a spreadsheet or your inbox: email siftwright.data@gmail.com and we'll quote it. Happy hunting.