Scraping Real Estate Listings for Business Intelligence and Leads

Real Estate Data Sources

Real estate generates large volumes of publicly available data. Understanding where data comes from and what access methods exist is essential for legal and effective data collection:

Legal and Ethical Considerations

What you can and cannot safely collect

Accessing Public Property Records

County assessor data

Most counties provide online access to property records. These are public records that include:

Python: Accessing county records APIs

Some counties and data aggregators provide APIs or bulk downloads:

import requests
import csv
import time

def fetch_property_records(county_api_url, params,
                            output_file):
    """Fetch property records from a county API endpoint."""
    results = []
    page = 1

    while True:
        params['page'] = page

        try:
            response = requests.get(
                county_api_url,
                params=params,
                timeout=30
            )
            response.raise_for_status()
            data = response.json()

            records = data.get('results', [])
            if not records:
                break

            results.extend(records)
            page += 1
            time.sleep(2)  # Polite rate limiting

        except requests.RequestException as e:
            print(f"Error on page {page}: {e}")
            break

    # Write to CSV
    if results:
        keys = results[0].keys()
        with open(output_file, 'w', newline='') as f:
            writer = csv.DictWriter(f, fieldnames=keys)
            writer.writeheader()
            writer.writerows(results)

    return len(results)

Property data APIs (commercial)

# Example: Using a property data API
import requests

def lookup_property(api_key, address, city, state):
    """Look up a property using a data API."""
    url = 'https://api.example.com/v1/property'
    headers = {
        'Authorization': f'Bearer {api_key}',
        'Accept': 'application/json'
    }
    params = {
        'address': address,
        'city': city,
        'state': state
    }

    response = requests.get(url, headers=headers,
                             params=params, timeout=30)
    response.raise_for_status()
    return response.json()

Extracting Business Intelligence from Property Data

Market research applications

Lead generation applications

Monitoring property data

import hashlib
import json
import os

def monitor_property_changes(records, previous_file):
    """Compare current records to previous snapshot
    and identify changes."""
    changes = {
        'new': [],
        'price_changed': [],
        'status_changed': [],
        'removed': []
    }

    # Load previous records
    previous = {}
    if os.path.exists(previous_file):
        with open(previous_file, 'r') as f:
            for record in json.load(f):
                key = record.get('address', '')
                previous[key] = record

    # Compare
    current_keys = set()
    for record in records:
        key = record.get('address', '')
        current_keys.add(key)

        if key not in previous:
            changes['new'].append(record)
        else:
            prev = previous[key]
            if record.get('price') != prev.get('price'):
                changes['price_changed'].append({
                    'address': key,
                    'old_price': prev.get('price'),
                    'new_price': record.get('price')
                })
            if record.get('status') != prev.get('status'):
                changes['status_changed'].append({
                    'address': key,
                    'old_status': prev.get('status'),
                    'new_status': record.get('status')
                })

    # Find removed listings
    for key in previous:
        if key not in current_keys:
            changes['removed'].append(previous[key])

    # Save current as new baseline
    with open(previous_file, 'w') as f:
        json.dump(records, f)

    return changes

Agent and Brokerage Directories

Real estate agent directories on brokerage websites are a source of contact information for B2B prospecting (selling services to agents, not for consumer outreach):

Ethical guidelines for agent data

Processing Real Estate Data

When working with property data downloads (CSV from county assessor sites, HTML from public record searches, PDF from tax documents), upload to Email Extractor to extract email addresses. The tool handles CSV, HTML, PDF and other formats commonly used for property and agent data. This is useful for consolidating agent contact lists from multiple brokerage sites or extracting owner contact information from public record documents.

Related Guides

• Email Extraction for Real Estate Agents
• Scraping Directories for Leads
• Web Scraping for Market Research
• Scraping for Lead Generation
• Lead Generation for Real Estate