Scraping Social Media for Business Contacts and Leads

Social Media as a Data Source

Social media platforms contain vast amounts of publicly shared business information: company profiles, employee directories, contact details, industry affiliations, technology usage and business relationships. This data is valuable for lead generation, market research and competitive analysis.

This guide covers what data is available on each platform, how to access it, the legal and ethical boundaries, and alternatives to scraping.

Platform-by-Platform Overview

LinkedIn

Data available:

• Professional profiles (name, title, company, location, skills, experience).
• Company pages (industry, size, specialities, employee count, recent hires).
• Job postings (roles, requirements, technology stack).
• Content engagement (who comments on industry topics, who shares what).
• Group memberships (industry affiliations, interests).
• Connections and mutual connections.

Access methods:

LinkedIn's position: LinkedIn actively prohibits scraping and has pursued legal action against scrapers. The landmark hiQ v. LinkedIn case (2022) established that scraping publicly available data may not violate the CFAA, but LinkedIn has continued to enforce its ToS through account restrictions and legal threats.

Recommendation: Use LinkedIn Sales Navigator for research and prospecting. Use official data providers for contact data. Do not scrape LinkedIn directly.

Twitter/X

Data available:

• Public profiles (name, bio, location, links, follower count).
• Tweets and replies (content, engagement, timing).
• Followers and following lists.
• Lists (curated groups of users by topic).
• Spaces participation.

Access methods:

Business use cases:

• Find industry thought leaders and influencers.
• Monitor competitor activity and positioning.
• Identify prospects discussing relevant topics.
• Discover company employees and their interests.
• Research industry trends and conversations.

Facebook and Instagram

Data available:

• Business pages (company info, contact details, reviews).
• Groups (member lists on public groups, discussions).
• Ad library (competitor advertising).
• Instagram business profiles (contact info, category).

Access methods:

Business use cases:

• Competitor ad monitoring (Ad Library).
• Local business contact information.
• Customer sentiment from reviews and comments.
• Industry group discussions and member identification.

GitHub

Data available:

• Developer profiles (name, email, location, company, repositories).
• Organisation profiles (members, repositories, technology stack).
• Repository data (languages, dependencies, contributors).
• Issue and pull request discussions.

Access methods:

Business use cases:

• Developer recruiting (find developers by language, contribution level, location).
• Technology adoption tracking (which companies use which frameworks).
• Competitor engineering analysis (open-source activity, hiring, technology choices).
• Finding developer emails from commit history.

Note on emails from commits: Git commits often contain the committer's email address. These are publicly visible in open-source repositories. While technically public data, using scraped commit emails for cold outreach is generally unwelcome in the developer community.

Reddit

Data available:

• Subreddit discussions (topics, opinions, recommendations).
• User profiles (post history, subreddit participation).
• Awards and engagement data.

Access methods:

Business use cases:

• Market research (what do people say about your category?).
• Competitive intelligence (competitor complaints and praise).
• Product feedback (subreddit mentions of your product or competitors).
• Community identification (which subreddits discuss your industry?).

Reddit is generally poor for direct contact extraction but excellent for market research and competitive intelligence.

Data Extraction Techniques

API-first approach (recommended)

APIs are the preferred method for accessing social media data because:

• Access is explicitly permitted.
• Data is structured and clean.
• Rate limits are documented.
• No risk of account bans.
• Compliance with platform policies.

API workflow:

1. Register for API access on the platform.
2. Obtain API credentials (API key, OAuth tokens).
3. Build queries to extract the data you need.
4. Respect rate limits (implement backoff and retry logic).
5. Store data responsibly (comply with platform data use policies).

Browser extension approach

For platforms where API access is limited (especially LinkedIn), browser extensions can automate manual research:

These extensions automate what a user could do manually: view a profile and extract visible data. They operate in a grey area: faster than manual but slower than direct scraping.

Export approach

Many platforms allow you to export your own data:

Exporting your own LinkedIn connections is the most legitimate way to get contact data from LinkedIn. Go to Settings > Data Privacy > Get a copy of your data > Connections. The export includes name, email (if shared), company, title and connection date.

After exporting, upload the CSV to Email Extractor to extract and deduplicate the email addresses from the export files.

Data Enrichment

Social media data is often incomplete. Enrichment fills the gaps:

Workflow:

1. Extract or export base data from social media (name, company, title).
2. Use an email finder tool to discover email addresses. See Email Finder Tool Comparison.
3. Use a data provider for phone numbers, company data, technology data.
4. Consolidate and deduplicate. Upload all sources to Email Extractor.
5. Verify email addresses before outreach.

Enrichment providers:

Legal and Ethical Considerations

Terms of service

Every major social media platform prohibits scraping in its terms of service:

Legal landscape

• hiQ v. LinkedIn (2022): The Ninth Circuit ruled that scraping publicly available data may not violate the CFAA. However, this does not override ToS or other legal claims.
• GDPR (EU): Personal data scraped from social media is still personal data. Processing requires a legal basis (legitimate interest is the most commonly claimed basis for B2B).
• CCPA (California): Publicly available information is generally excluded from CCPA's definition of personal information, but the exclusion is narrow.
• Platform-specific data policies: Platforms may require that data obtained through their APIs is not used for certain purposes (e.g., advertising, reselling).

Ethical guidelines

Even where scraping may be technically legal:

• Respect opt-out signals (private accounts, do-not-contact requests).
• Do not contact individuals about sensitive information revealed on social media.
• Do not misrepresent how you obtained contact information.
• Rate limit requests to avoid impacting platform performance.
• Do not redistribute or resell scraped data.
• Consider whether the individual would expect to be contacted based on what they shared.

See Is Web Scraping Legal and Scraping Ethics Best Practices.

Alternatives to Scraping

Related Guides

• LinkedIn Prospecting Guide
• Scraping Job Boards for Leads
• Scraping Directories for Leads
• Building Targeted Prospect Lists
• Web Scraping for Market Research