Aller au contenu
Toutes les missions
Veille marché · Dev-to Nouvelle Publiée il y a 1 jour

Fifty Indeed Country Sites Gave Me 4,961 Jobs and Zero Salaries. One Request Header Said en-US

I run a visa agency in Bali and build Apify Actors on the other half of my week. One of them reads Indeed through the GraphQL endpoint its mobile app uses: 100 jobs per request with full des...

Entreprise a gave me was
Mode Remote

Résumé

I run a visa agency in Bali and build Apify Actors on the other half of my week. One of them reads Indeed through the GraphQL endpoint its mobile app uses: 100 jobs per request with full descriptions, no login, no browser. It has been shipping for months, and the column people buy it for is salary. Yesterday I finally asked a question I should have asked in month one: how often does Indeed actually print pay, per country? The answer my own data gave me was: outside the United States, never. Not rarely. Never. So I ran the cleanest test I could: one query (manager), one big city per site, 100 rows per site, 51 Indeed country sites, two paired calls each — one with the header the Actor had been sending for months, one with the locale of the site being searched. locale sent sites job rows rows with a salary indeed-locale: en-US 50 non-US 4,961 0 indeed-locale: en-<country> the same 50 4,961 1,030 Zero. Not one of 4,961 rows from 50 country sites. The US site in the same sweep priced 98 of 100 rows. The header, not the market Here is the whole thing, same endpoint, same query, two minutes apart: indeed-co: GB indeed-locale: en-US → 100 jobs, 0 with compensation, no currency indeed-co: GB indeed-locale: en-GB → 100 jobs, 100 with compensation, GBP The compensation node is simply absent from every job object in the first response. Nothing errors, nothing warns, the row count is identical, and your dataset gets 100 perfectly good London jobs with an empty salary column. Then you tell your customer that British employers don't publish pay. What woke up when the locale matched the site: site priced rows, en-US priced rows, matching locale currency uk 0 100 GBP it 0 90 EUR sg 0 82 SGD ca 0 74 CAD in 0 73 INR ph 0 67 PHP fr 0 46 EUR jp 0 20 JPY Thirty-two of the 50 sites started paying out on en-<country> alone, in 26 currencies. Of the 18 that stayed at zero, 14 came alive once I sent the site's own language instead of English — pt-PT 29 priced rows, zh-TW 50, tr-TR 9. Four stayed empty under every locale I tried: Norway, Finland, Israel, Ukraine. Those four I'll call a market fact rather than a bug. One caution on reading tables like this. Two calls to the same site with the same locale gave me 84 priced rows and then 41; another gave 70 and then 4. A relevance-sorted result set is not stable between calls, so small differences across a row like this one mean nothing. The only signal I trust here is zero versus not zero, which is exactly what the header flipped. The locale also decides which jobs exist I assumed the locale was a presentation setting. It isn't — it reaches the matching. "staff gudang", Jakarta, en-ID → 1 job "staff gudang", Jakarta, id-ID → 45 jobs, 22 of them priced "Staff gudang" is Indonesian for warehouse staff, and on the English locale the Indonesian site had one job for it. Going the other way, ar-EG returned 14 rows for the English word "manager" where en-EG returned 100. The locale is part of the query, not part of the skin, so the Actor now exposes it as an input. The thing that bit back Sending en-US to Germany had been doing something useful I never credited it for: it translates the job attribute chips into English. "Vollzeit" came back as "Full-time". My row builder read those English labels to fill employmentType, jobTypes, workplaceType and schedule. Fix the locale, and: Berlin, 72 jobs rows with employmentType en-US (English labels) 49 de-DE (German labels) 0 Paris went from 80 of 100 to 0 the same way — CDI is not in anybody's English dictionary. I had traded the salary column for the contract-type column. The way out was sitting in the response the whole time. Each attribute has a key as well as a label, and the key doesn't move: CF3CP = "Full-time" / "Vollzeit" / "Tempo integral" So the structural columns are now built from attribute ids — CF3CP, 75GKK, NJXCK, DSQF7 for remote, PAXZC for hybrid — and the English labels are only a fallback. Berlin on a German locale now types 45 of 72 rows and keeps its EUR salaries. The visible cost: skills on a non-English site now arrive in that site's language (Systemadministration, not System administration). I'd rather hand over the employer's own words than an English translation with the money missing. Act two: "$18 an hour" is not a range While I was in there, 127 US rows were bothering me. They had a currency and a pay period and no numbers: { "currencyCode": "USD", "baseSalary": { "unitOfWork": "HOUR", "range": {} } } An empty object is a very specific kind of clue — it means the server rendered a type my query didn't ask for. range is a union, introspection on the endpoint is disabled, but you can make a GraphQL server tell you the type names anyway: ask for 22 invented fragments and read which ones come back as Unknown type. Four survived: Range { min max } AtLeast { min } AtMost { max } Exactly { value } My query asked for ... on Range { min max } and nothing else. So every exact wage ("$18 an hour"), every floor ("from $55,000") and every ceiling ("up to $240,000") came back as {} and was stored as null. In 800 US rows: shape rows Range 559 Exactly 84 AtLeast 39 AtMost 4 127 of 686 priced rows, 18.5%, silently dropped. And the loss was not spread evenly — it lived exactly where hourly work lives: search priced rows lost before the fix python developer, New York 274 10 (3.6%) registered nurse, Chicago 262 47 (18%) warehouse associate, Dallas 150 70 (47%) Salaried roles quote bands; a warehouse shift quotes one number. A pipeline built on a developer search looks fine and quietly loses half the pay data on the jobs that are most likely to state it. Rows now carry salaryKind — range, exact, atLeast, atMost — because salaryMin: 55000, salaryMax: null and a $55k–$55k job are different facts. Before and after, same seven searches 1,472 rows, three US cities plus London, Berlin, Singapore and Jakarta, old parser against new: before after rows with a salary 561 (38%) 933 (63%) non-US rows with a salary 0 of 672 247 of 672 currencies USD USD, GBP, EUR, SGD, IDR rows with employmentType 1,119 1,091 And on the platform, build 0.1.44, a 10-row London run: 7 rows with GBP figures, 6 with a work-arrangement label. Before today that run returned 10 rows and an empty salary column, every time, forever. Two smaller truths from the same sample: compensation.estimated — Indeed's own salary guess — arrived on 0 of 800 US rows, so my field promising to flag estimates was always false; and recruit.detailedSalary, the human string ("$93,000 - $150,000 a year"), was null on 1,472 of 1,472 rows. It was in my README as an example. It is out of my README now. Where this doesn't help you If you average salaryMin across a US search you will still get nonsense: in one 800-row sample, 398 figures were hourly and 278 annual. Group by salaryPeriod before you aggregate anything, and drop atLeast/atMost rows from any average — a floor is not a midpoint. If you need pay data from Norway, Finland, Israel or Ukraine, this endpoint has none to give at any locale. And if you need Indeed plus LinkedIn plus company career pages in one schema, that's Jobs API, not this one. The Actor is Indeed Jobs & Salary Scraper, $0.003 a row, no browser: curl -X POST "https://api.apify.com/v2/acts/lergassy~indeed-jobs-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{"keywords":["customer success manager"],"location":"London","country":"uk","maxJobsPerQuery":50}' One last confession, because it was the most expensive five minutes of the day. My release gate takes whatever build carries the next tag, runs it, and promotes it if rows come back. The tag was still sitting on a build from four days earlier. The gate dutifully tested it, found 10 good rows, and pushed my customers backwards by seven builds. Rolled back in two minutes, then I taught the gate to refuse any build older than the one already live. A release tool that can only move forward should be told so explicitly; mine wasn't.