Skip to content
Toutes les missions
Veille marché · Dev-to Publiée 1 month ago

Every open job at 500 companies, as JSON, without scraping a single job board

Most "job data" projects start by scraping LinkedIn or Indeed, and most of them die there: captchas, logins, lawsuits, and listings that are already three reposts away from the employer. The...

Entreprise are already three reposts away from the employer.
Mode Remote

Résumé

Most "job data" projects start by scraping LinkedIn or Indeed, and most of them die there: captchas, logins, lawsuits, and listings that are already three reposts away from the employer. There is a quieter source. Most tech companies run their career page on an applicant-tracking system — Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee, Workable, Personio, Breezy, Pinpoint, Rippling, Teamtailor — and every one of those systems publishes a job feed so the career page can be embedded on the company's site. First-party, current, no login, no anti-bot. The feeds Greenhouse https://boards-api.greenhouse.io/v1/boards/{company}/jobs?content=true&pay_transparency=true Lever https://api.lever.co/v0/postings/{company}?mode=json Ashby https://api.ashbyhq.com/posting-api/job-board/{company}?includeCompensation=true SmartRecruiters https://api.smartrecruiters.com/v1/companies/{company}/postings Recruitee https://{company}.recruitee.com/api/offers/ Workable https://apply.workable.com/api/v1/widget/accounts/{company}?details=true Personio https://{company}.jobs.personio.de/xml Breezy HR https://{company}.breezy.hr/json Pinpoint https://{company}.pinpointhq.com/postings.json Rippling https://api.rippling.com/platform/api/ats/v1/board/{company}/jobs Teamtailor https://{career-site-host}/jobs.rss {company} is the slug in the career-page URL: jobs.lever.co/spotify → spotify. The catch: eleven shapes The feeds agree on nothing. Greenhouse sends the description as HTML that has been entity-escaped once (&lt;p&gt;), pay ranges in cents, and the location as { "name": … }. Lever sends createdAt as a millisecond string and splits the description across descriptionPlain, lists[] and additionalPlain. Ashby nests compensation three levels deep. SmartRecruiters paginates and needs one more request per posting for the text. Personio is XML, Teamtailor is RSS with its own namespace, and Rippling's feed has no descriptions at all. Un-escaping Greenhouse, for instance, is a two-step: import { load } from 'cheerio'; export function htmlToText(html: unknown): string | null { if (!html) return null; let h = String(html); if (!h.includes('<') && h.includes('&lt;')) h = load(`<x>${h}</x>`)('x').text(); // un-escape once h = h.replace(/<\s*br\s*\/?>/gi, '\n').replace(/<\/(p|div|li|h[1-6]|ul|ol|tr)>/gi, '\n').replace(/<li[^>]*>/gi, '• '); return load(`<x>${h}</x>`)('x').text().trim() || null; } (Workday, the twelfth system, has no feed. Its career sites are allowed by robots.txt and every job page carries schema.org JobPosting as JSON-LD, so it is read the old-fashioned way: listing, then page.) One shape out I normalised all twelve into 26 fields — title, department, team, locations[], workplace_type (remote / hybrid / onsite), salary_min, salary_max, salary_currency, posted_at, apply_url, plain-text description, and a stable id (ats:company:job) so a daily run can be diffed in one line. A real run on 2026-09-17: three boards (Duolingo on Greenhouse, Spotify on Lever, Ramp on Ashby) → 304 jobs in 18 seconds, 224 of them (74%) with a published pay range already parsed to numbers. Two things I deliberately leave out: recruiter names and mailboxes that some feeds carry, and any e-mail or phone number inside a description (replaced with [email removed]). It is job data, not people data. Use it If you would rather not maintain twelve parsers, it is on Apify Store as Career Site Jobs Scraper — paste career-page URLs, filter by title keywords / location / "posted within N days", $1 per 1,000 jobs, callable from the API or from an AI agent over MCP: https://apify.com/dododata/career-site-jobs-scraper If a feed changes shape, it is fixed within 48 hours — that promise is on the listing.