Stably scrape job postings from recruitment platforms including Indeed and LinkedIn. Supports remote/full-time/salary filtering, custom proxies, and multi-dimensional precise search. Deploy with one click to obtain overseas job data.
A CoreClaw worker that aggregates job listings across multiple platforms using JobSpy.
| Platform | Key | Status | Notes |
|---|---|---|---|
| Indeed | indeed | ✅Stable | Best coverage, no rate limiting |
| ✅Stable | May rate-limit after ~10 pages per IP | ||
| Glassdoor | glassdoor | ⚠️ Unreliable | API changes, location parsing issues |
| ZipRecruiter | zip_recruiter | ❌ Blocked | Cloudflare WAF 403 Forbidden |
| Google Jobs | ⚠️ Unreliable | Requires specific google_search_term syntax | |
| Bayt | bayt | ❌ Blocked | 403 Forbidden (anti-bot) |
| Naukri | naukri | ❌ Blocked | Requires CAPTCHA |
| BDJobs | bdjobs | ❌ Bug | Upstream bug: missing user_agent param |
Recommended: Use ["indeed", "linkedin"] for reliable results.
| Parameter | Type | Default | Description |
|---|---|---|---|
site_name | stringList | ["indeed", "linkedin"] | Job boards to search (multiple allowed).Concurrency split field |
search_term | string | "Software Engineer" | Job title or keyword |
location | string | "New York" | City, state, or country |
country_indeed | select | "usa" | Country for Indeed/Glassdoor (21 options) |
distance | integer | 50 | Search radius in miles |
is_remote | boolean | false | Filter remote-only jobs |
job_type | select | "" | Employment type (fulltime, parttime, contract, internship, etc.) |
results_wanted | integer | 50 | Results per site |
description_format | select | "markdown" | Job description format (markdown / html / plain) |
enforce_annual_salary | boolean | true | Convert all salaries to annual |
linkedin_fetch_description | boolean | false | Fetch full LinkedIn descriptions (slower) |
hours_old | integer | 0 | Only jobs posted within N hours (0 = no filter) |
offset | integer | 0 | Skip N results (pagination) |
google_search_term | string | "" | Separate search term for Google Jobs |
easy_apply | boolean | false | Filter one-click apply jobs (Indeed/LinkedIn) |
linkedin_company_ids | string | "" | Comma-separated LinkedIn company IDs |
user_agent | string | "" | Custom User-Agent header |
verbose | select | 1 | Log level (0=Errors, 1=Warnings, 2=Info) |
proxies | string | "" | Proxy URL (leave empty for platform built-in) |
Each row contains 35 fields:
| Category | Fields |
|---|---|
| Job Identity | id, site, job_url, job_url_direct |
| Job Info | title, company, location, date_posted, job_type, is_remote |
| Salary | salary_source, interval, min_amount, max_amount, currency |
| Job Details | job_level, job_function, listing_type, description |
| Company | company_industry, company_url, company_logo, company_url_direct, company_addresses, company_num_employees, company_revenue, company_description, company_rating, company_reviews_count |
| Skills & Experience | skills, experience_range, emails |
| Other | vacancy_count, work_from_home_type |
| Status | status, error |
Three proxy modes (in priority order):
proxies parameter (e.g. socks5://user:pass@host:port)PROXY_AUTH environment variable| Platform | Solution |
|---|---|
| ZipRecruiter | Blocked by Cloudflare WAF. No workaround available. |
| Glassdoor | Try different location format (e.g. "USA" instead of "New York") |
| Google Jobs | Use google_search_term with specific syntax from Google Jobs UI |
| Bayt | Blocked by anti-bot. No workaround available. |
| Naukri | Requires CAPTCHA. No workaround available. |
| BDJobs | Upstream bug. Wait for JobSpy update or exclude from search. |
results_wantedproxies parameter| Package | Purpose |
|---|---|
python-jobspy | Multi-platform job scraper |
pandas | DataFrame processing |
PySocks | SOCKS5 proxy support |
grpcio | CoreClaw SDK gRPC communication |
protobuf | Protocol Buffers runtime |
python-dateutil | Date parsing |
This worker wraps JobSpy which is licensed under the MIT License.
Explore more popular scrapers from our marketplace