twitter scraper

What Is Twitter Scraper?

Twitter Scraper helps you collect publicly available posts and profile information from X and Twitter and convert the results into structured data.

Instead of manually searching X and copying tweets one by one, define the audience, topic, account, hashtag, or search criteria you need and let CoreClaw process the results.

You can use it to:

• Search tweets by keyword
• Scrape tweets from a profile
• Search hashtags
• Find mentions
• Filter by date range
• Filter by language
• Filter verified accounts
• Find image or video posts
• Filter by engagement
• Search by geographic criteria
• Collect detailed author information
• Export results to CSV or JSON

Common Use Cases

Social Listening

Track public conversations around brands, products, companies, competitors, industries, and events.

Sentiment Analysis

Collect tweet text at scale and send it into sentiment, topic classification, or language analysis workflows.

Brand Monitoring

Search for brand names, product names, URLs, hashtags, and account mentions to monitor public discussion.

Competitor Research

Track competitor accounts and related keywords to study messaging, launches, audience reaction, and engagement.

Market Research

Analyze conversations around markets, industries, technologies, and emerging trends.

Content Research

Find high-engagement tweets by keyword, hashtag, author, or media type to understand what content resonates.

Creator Research

Analyze posts from selected X accounts together with followers, verification information, and engagement metrics.

Academic Research

Build structured public tweet datasets for permitted research into communication, public discussion, and online behavior.

What Data Can You Extract From Tweets?

Available fields depend on the tweet and information X makes publicly available.

What Author Data Can You Extract?

Available author fields can include:

This allows you to analyze tweet content together with useful public author context.

How to Scrape Tweets

1. Create a CoreClaw account.
2. Open the Twitter Scraper.
3. Add search terms, usernames, or supported X URLs.
4. Add optional date, language, media, verification, or engagement filters.
5. Choose the result order.
6. Set the maximum number of tweets.
7. Start the Worker.
8. Review the structured records.
9. Export results as CSV or JSON.

Choose Result Sorting

Available search ordering may include:

• Latest
• Top
• Latest + Top

Use Latest when recency matters.

Use Top when engagement and relevance matter more.

Use Latest + Top when you want broader coverage and are prepared to deduplicate overlapping results.

Important Search Limits

The referenced Apify implementation is optimized for large-scale search and currently applies several specific restrictions.

It requires each query to return at least approximately 50 tweets.

It is not designed for:

• Fetching one individual tweet
• Scraping a single conversation
• Extracting complete reply threads
• Very small queries
• High-frequency monitoring of the same search

CoreClaw's exact limits depend on the current Worker implementation.

If your workflow requires single Tweet URLs or complete conversation threads, use a dedicated Tweet or conversation extraction Worker when available.

Why Some Tweets May Be Missing

X controls its own search results.

Some tweets may not appear because of:

• X search filtering
• Account visibility
• Content moderation
• Rate limiting or visibility filtering
• Search pagination changes
• Deleted content
• Query restrictions

A scraper cannot reliably return content that X itself does not expose through the corresponding search.

For important research, test your query directly on X before running a large collection task.

Twitter Scraper API

CoreClaw Twitter Scraper can also be integrated into automated workflows.

Connect tweet data with:

• Social listening platforms
• Brand monitoring systems
• Sentiment analysis
• AI classification pipelines
• Market research databases
• Content analytics dashboards
• Competitor monitoring systems
• Internal data pipelines

Why Use CoreClaw Twitter Scraper?

Search With Keywords or Profiles

Start with a topic, account, hashtag, mention, or X search rather than manually collecting Tweet URLs.

Use Advanced Search Filters

Combine author, date, language, media, verification, location, and engagement criteria.

Get Tweet and Author Data Together

Collect tweet text, engagement metrics, and public author information together.

Focus on Relevant Tweets

Use likes, reposts, replies, media, or verification filters to reduce irrelevant results.

Export Structured Results

Move X data into CSV, JSON, spreadsheets, databases, or analysis workflows.

Use It Without Building a Scraper

Run searches through CoreClaw while keeping API access available for automated workflows.

Frequently Asked Questions

Can I Scrape Tweets by Keyword?

Yes. Add keywords or advanced Twitter search queries.

Can I Scrape Tweets From a User?

Yes. Use a Twitter username or a query such as from:username.

Can I Scrape Tweets by Hashtag?

Yes. Add hashtags such as #AI or combine them with language and other search operators.

Can I Scrape Tweets by Date?

Yes. Date filters can be applied through advanced search syntax or supported date fields.

Can I Scrape Only Popular Tweets?

Yes. Set minimum likes, reposts, and reply counts, or sort by Top when supported.

Can I Scrape Verified Users?

Yes. You can restrict results to verified users when supported.