Jobspy - Linkedin, Glassdoor Scraper

Pricing

Jobspy - Linkedin, Glassdoor Scraper

odin-kael/jobspy

Stably scrape job postings from recruitment platforms including Indeed and LinkedIn. Supports remote/full-time/salary filtering, custom proxies, and multi-dimensional precise search. Deploy with one click to obtain overseas job data.

Try for Free

What is Jobspy ？

A CoreClaw worker that aggregates job listings across multiple platforms using JobSpy.

Platform	Key	Status	Notes
Indeed	indeed	✅Stable	Best coverage, no rate limiting
LinkedIn	linkedin	✅Stable	May rate-limit after ~10 pages per IP
Glassdoor	glassdoor	⚠️ Unreliable	API changes, location parsing issues
ZipRecruiter	zip_recruiter	❌ Blocked	Cloudflare WAF 403 Forbidden
Google Jobs	google	⚠️ Unreliable	Requires specific google_search_term syntax
Bayt	bayt	❌ Blocked	403 Forbidden (anti-bot)
Naukri	naukri	❌ Blocked	Requires CAPTCHA
BDJobs	bdjobs	❌ Bug	Upstream bug: missing user_agent param

Recommended: Use ["indeed", "linkedin"] for reliable results.

Quick Start

Upload this folder to CoreClaw as a new Script
Configure parameters in the UI
Run the script

Examples

Example 1: Basic Search (Indeed, New York)

json

{
  "site_name": [{ "string": "indeed" }],
  "search_term": "Software Engineer",
  "location": "New York",
  "country_indeed": "usa",
  "results_wanted": 15
}

Example 2: Multi-Platform Remote Jobs with Salary Comparison

json

{
  "site_name": [
    { "string": "indeed" },
    { "string": "linkedin" }
  ],
  "search_term": "Data Scientist",
  "location": "San Francisco, CA",
  "country_indeed": "usa",
  "is_remote": true,
  "results_wanted": 30,
  "enforce_annual_salary": true,
  "description_format": "markdown"
}

Example 3: Recent Full-Time Jobs in London

json

{
  "site_name": [{ "string": "indeed" }, { "string": "linkedin" }],
  "search_term": "Product Manager",
  "location": "London",
  "country_indeed": "uk",
  "job_type": "fulltime",
  "hours_old": 168,
  "results_wanted": 25
}

Example 4: Google Jobs with Custom Search Term

json

{
  "site_name": [{ "string": "google" }, { "string": "indeed" }],
  "search_term": "Machine Learning Engineer",
  "google_search_term": "ML Engineer AI jobs near Berlin since yesterday",
  "location": "Berlin",
  "country_indeed": "germany",
  "results_wanted": 20
}

Example 5: LinkedIn Company-Specific Search

json

{
  "site_name": [{ "string": "linkedin" }],
  "search_term": "Software Engineer",
  "location": "Singapore",
  "country_indeed": "singapore",
  "linkedin_fetch_description": true,
  "linkedin_company_ids": "1441,2382910",
  "easy_apply": true,
  "results_wanted": 10
}

Example 6: Full-Featured Search with Custom Proxy

json

{
  "site_name": [
    { "string": "indeed" },
    { "string": "linkedin" }
  ],
  "search_term": "DevOps Engineer",
  "location": "Tokyo",
  "country_indeed": "japan",
  "distance": 25,
  "is_remote": false,
  "job_type": "fulltime",
  "easy_apply": false,
  "results_wanted": 50,
  "description_format": "markdown",
  "enforce_annual_salary": true,
  "linkedin_fetch_description": false,
  "hours_old": 72,
  "offset": 0,
  "verbose": 1,
  "proxies": "socks5://user:pass@proxy.example.com:1080"
}

Parameters

Parameter	Type	Default	Description
`site_name`	stringList	`["indeed", "linkedin"]`	Job boards to search (multiple allowed).Concurrency split field
`search_term`	string	`"Software Engineer"`	Job title or keyword
`location`	string	`"New York"`	City, state, or country
`country_indeed`	select	`"usa"`	Country for Indeed/Glassdoor (21 options)
`distance`	integer	`50`	Search radius in miles
`is_remote`	boolean	`false`	Filter remote-only jobs
`job_type`	select	`""`	Employment type (fulltime, parttime, contract, internship, etc.)
`results_wanted`	integer	`50`	Results per site
`description_format`	select	`"markdown"`	Job description format (markdown / html / plain)
`enforce_annual_salary`	boolean	`true`	Convert all salaries to annual
`linkedin_fetch_description`	boolean	`false`	Fetch full LinkedIn descriptions (slower)
`hours_old`	integer	`0`	Only jobs posted within N hours (0 = no filter)
`offset`	integer	`0`	Skip N results (pagination)
`google_search_term`	string	`""`	Separate search term for Google Jobs
`easy_apply`	boolean	`false`	Filter one-click apply jobs (Indeed/LinkedIn)
`linkedin_company_ids`	string	`""`	Comma-separated LinkedIn company IDs
`user_agent`	string	`""`	Custom User-Agent header
`verbose`	select	`1`	Log level (0=Errors, 1=Warnings, 2=Info)
`proxies`	string	`""`	Proxy URL (leave empty for platform built-in)

Output

Each row contains 35 fields:

Category	Fields
Job Identity	id, site, job_url, job_url_direct
Job Info	title, company, location, date_posted, job_type, is_remote
Salary	salary_source, interval, min_amount, max_amount, currency
Job Details	job_level, job_function, listing_type, description
Company	company_industry, company_url, company_logo, company_url_direct, company_addresses, company_num_employees, company_revenue, company_description, company_rating, company_reviews_count
Skills & Experience	skills, experience_range, emails
Other	vacancy_count, work_from_home_type
Status	status, error

Proxy

Three proxy modes (in priority order):

User-defined — set the proxies parameter (e.g. socks5://user:pass@host:port)
Platform built-in — auto-detected via PROXY_AUTH environment variable
Direct — no proxy

Troubleshooting

No results from a platform?

Platform	Solution
ZipRecruiter	Blocked by Cloudflare WAF. No workaround available.
Glassdoor	Try different location format (e.g. "USA" instead of "New York")
Google Jobs	Use `google_search_term` with specific syntax from Google Jobs UI
Bayt	Blocked by anti-bot. No workaround available.
Naukri	Requires CAPTCHA. No workaround available.
BDJobs	Upstream bug. Wait for JobSpy update or exclude from search.

Rate Limited (429)?

Reduce results_wanted
Use rotating proxies via proxies parameter
Wait between requests

Project Structure

text

worker-jobspy/
├── main.py              # Entry point (CoreSDK + asyncio)
├── jobspy_worker.py     # Core logic (framework-independent)
├── input_schema.json    # UI parameter definitions
├── requirements.txt     # Python dependencies
├── sdk.py               # CoreClaw SDK
├── sdk_pb2.py           # gRPC protobuf
├── sdk_pb2_grpc.py      # gRPC stub
└── .gitignore

Dependencies

Package	Purpose
`python-jobspy`	Multi-platform job scraper
`pandas`	DataFrame processing
`PySocks`	SOCKS5 proxy support
`grpcio`	CoreClaw SDK gRPC communication
`protobuf`	Protocol Buffers runtime
`python-dateutil`	Date parsing

Pricing

Failed results don't count

Rating

5.0

Developer

Kael Odin

Worker Stats

34 Total runs

Success rate: 88.23%

Last updated: Apr 22, 2026

Twitter Post Scraper (by Profile URL)

by mn06pz6r

Twitter Post Scraper (by Profile URL) extracts recent tweets from any public Twitter/X profile. Provide one or more profile URLs and it will fetch the latest posts along with engagement metrics and author details.

5.0

5 runs

From $0.6/1,000 results

Twitter Profile Scraper (by Username)

by mn06pz6r

Twitter Profile Scraper (by Username) extracts a complete Twitter/X profile and its recent tweets using just a username. Provide one or more usernames and it will return the full profile metadata along with an array of recent posts.

5.0

11 runs

From $0.6/1,000 results

Twitter Profile Scraper (by Profile URL)

by mn06pz6r

Twitter Profile Scraper (by Profile URL) extracts a complete Twitter/X profile and its recent tweets in a single run. Provide one or more profile URLs and it will return the full profile metadata along with an array of recent posts.

5.0

9 runs

From $0.6/1,000 results

Twitter Post Scraper (by Post URL)

by mn06pz6r

Twitter Post Scraper (by Post URL) extracts a single Twitter/X post with all its details by post URL. Provide one or more tweet URLs and it will return the full post data including engagement metrics, media, hashtags, and author information.

5.0

11 runs

From $0.6/1,000 results

View All Scrapers