Skip to content
Toutes les missions
Veille marché · Dev-to Nouvelle Publiée 5 hours ago

Remote jobs from Remote OK and We Work Remotely in one deduplicated list with Python

Remote job newsletters, niche job boards and job-search tools all start from the same few boards. Remote OK and We Work Remotely both publish public feeds, but in different formats, with dif...

Entreprise tribution
Mode Remote

Résumé

Remote job newsletters, niche job boards and job-search tools all start from the same few boards. Remote OK and We Work Remotely both publish public feeds, but in different formats, with different categories and with the same job often posted on both. This post pulls both boards into one deduplicated list in Python, filters to remote roles open to one country with a minimum salary, takes a quick salary snapshot, and sets up a daily feed of new jobs. Disclosure: I'm Aurelix, and I built Remote Job Board Scraper, the tool used here. It's a paid tool (prices below). What each row holds 28 fields per job: title, company, company_logo_url, tags. category (one of 12 normalised categories) and the board's own category_raw. employment_type (normalised, e.g. FULL_TIME, CONTRACT) and employment_type_raw. locations (the allowed regions as published) and a worldwide flag. salary_min, salary_max, salary_currency, salary_interval where the board publishes a salary. posted_at, expires_at, job_url, apply_url, source, an attribution line, and also_listed_on when the same job is on the other board. With includeDescription on, description_text and description_html. Python and data jobs open to Canada or the US pip install apify-client import csv import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("aurelix/remote-jobs-scraper").call( run_input={ "keywords": ["python", "data engineer"], "categories": ["software-development", "data-ai"], "regions": ["Canada", "USA"], "minSalary": 80000, } ) jobs = list(client.dataset(run["defaultDatasetId"]).iterate_items()) fields = ["title", "company", "category", "locations", "salary_min", "salary_max", "salary_currency", "posted_at", "source", "job_url"] with open("remote_jobs.csv", "w", newline="", encoding="utf-8") as f: w = csv.DictWriter(f, fieldnames=fields, extrasaction="ignore") w.writeheader() w.writerows(jobs) regions matches words in the published location text (country names, two-letter codes such as GB, or time zones such as UTC), and includeWorldwide (on by default) keeps jobs open to anyone anywhere. minSalary compares the top of the published range, converted to yearly, and drops jobs without a salary. A quick salary snapshot from collections import Counter from statistics import median paid = [j for j in jobs if j["salary_max"] and j["salary_interval"] == "year" and j["salary_currency"] == "USD"] if paid: print(f"{len(paid)} of {len(jobs)} jobs publish a yearly USD salary") print(f"median top of range: ${median(j['salary_max'] for j in paid):,.0f}") print("by category:", Counter(j["category"] for j in jobs).most_common()) print("companies hiring most:", Counter(j["company"] for j in jobs).most_common(5)) New jobs every morning run_input = { "keywords": ["python", "data engineer"], "regions": ["Canada"], "incremental": True, "stateKey": "remote-python-canada", "slackWebhookUrl": "https://hooks.slack.com/services/...", } With incremental on, each run returns only jobs no earlier run returned, matched across both boards. Save the input as a task in the Apify Console and schedule it daily: Slack gets a short digest of the new jobs, and nothing is sent when there are none. webhookUrl posts the same digest as JSON to Zapier, Make or n8n. Why this tool Two boards, one schema, with a normalised category and employment type. A job on both boards comes back once. Region and minimum-salary filters. An only-new mode matched across both boards, with a Slack or webhook digest. Cheaper than the leading tools. Limits Coverage is what the boards publish: Remote OK's feed gives its ~100 newest jobs and We Work Remotely's feeds their newest jobs per category. By default each board returns up to 100 jobs; maxResultsPerSource: 0 returns everything they publish. Categories come from keyword rules on titles and board categories; unusual titles can land in other. Region matching is by word, synonym and country code, not geography: Europe does not match a list of European countries. Neither board publishes seniority, so that column stays empty. Apply links go to the job page on the source board, as the boards require. Cost $0.90 per 1,000 jobs on Apify's free plan, from $0.48 per 1,000 on paid plans, plus a start fee per run. A job on both boards is charged once. The current prices are on the Store page; Apify's free plan includes $5 of credit a month, about 5,500 jobs. Use it from an AI assistant The tool also works through Apify's hosted MCP server, so Claude, ChatGPT or Cursor can run it from a request like "remote data engineer jobs open to Canada paying at least $100k, posted this week". Server URL: https://mcp.apify.com/?tools=aurelix/remote-jobs-scraper. Links Remote Job Board Scraper on the Apify Store