Data Cleaning for Recruitment Agency Candidate Databases, ATS Systems, Talent Pools, Staffing Firm Contact Lists and Executive Search Firm Research Files
On this page
The Recruitment Data Problem
Recruitment agencies accumulate candidate data faster than almost any other business type. Every job board application, LinkedIn import, referral submission, career fair contact card, resume received by email, and manual entry creates a new record. A mid-size staffing firm with 10 recruiters may add 500-2,000 new candidate records per week. Over 3-5 years, the ATS database grows to 100,000-500,000+ records, and without active maintenance, 30-50% of those records become unusable:
| Data quality issue | How it happens | Scale | Impact on recruiting operations |
|---|---|---|---|
| Duplicate candidate records | Same candidate applies through multiple job boards (Indeed, LinkedIn, ZipRecruiter); recruiter imports a candidate from LinkedIn who already exists from a prior application; candidate reapplies after 6 months and creates a new record; ATS migration from a previous system creates duplicates | 15-30% of records in a typical recruiting database are duplicates | Recruiter contacts the same candidate twice for the same role (unprofessional); candidate history is split across records (one record shows the interview from 2024, another shows the placement from 2025); reporting is inflated (500,000 "candidates" is really 375,000 unique people) |
| Outdated contact information | Candidates change jobs (and lose their work email), change phone numbers, move to new cities; email addresses provided 2-3 years ago may no longer be active | 20-40% of candidate email addresses become invalid within 2 years; 30-50% of phone numbers become invalid within 3 years | Recruiters waste time trying to reach unreachable candidates; voicemails are never returned because the number belongs to someone else; emails bounce; the best candidate for a role is in your database but you cannot reach them |
| Incomplete records | Candidate submitted a resume but no contact details were parsed correctly; recruiter manually entered a name and phone number but no email; LinkedIn import captured name and headline but no email or phone | 10-25% of records are missing email address; 15-30% are missing phone number; 20-40% are missing current employer/title | Incomplete records are unsearchable for future roles; a candidate with no email cannot receive job alerts or marketing; a candidate with no current title cannot be matched to relevant positions |
| Outdated skills and experience | Candidate's record reflects their 2022 resume; they have since earned a certification, changed industries, been promoted, or learned new technologies | Nearly 100% of records older than 12 months have some outdated information | Boolean searches return outdated results; a candidate who was a "Junior Developer" in 2022 is now a "Senior Developer" but your search for "Senior Developer" does not find them; skills-based matching produces poor results |
| Consent and compliance gaps | GDPR requires a legal basis for processing candidate data; some records predate GDPR implementation; consent records were not maintained during ATS migrations; candidates who withdrew consent were not properly suppressed | Varies by agency; agencies operating in the EU or processing EU candidate data may have 10-40% of records with unclear consent status | Processing candidate data without proper consent risks GDPR fines (up to 4% of annual global turnover or 20 million euros, whichever is higher); reputational damage; candidate complaints to data protection authorities |
Systems That Hold Candidate Data
| System | What it stores | Common platforms |
|---|---|---|
| Applicant Tracking System (ATS) | Primary candidate database: contact information, resume/CV, application history, interview notes, placement history, skills, certifications, availability, rate/salary, recruiter notes, communication history | Bullhorn, JobAdder, Vincere, Tracker, Crelate, Loxo, PCRecruiter, CATS, Zoho Recruit, Greenhouse (more employer-side), Lever (more employer-side), iCIMS, Workday Recruiting |
| Job board accounts | Candidate profiles and applications received through job boards; may include resume, cover letter, screening question responses | Indeed, LinkedIn Recruiter, ZipRecruiter, Monster, CareerBuilder, Dice (tech), Glassdoor |
| Email (recruiter inboxes) | Resumes received by email; candidate correspondence; referral introductions; interview scheduling | Gmail, Outlook, corporate email |
| LinkedIn Recruiter | LinkedIn profiles saved, InMail history, project/pipeline assignments, tags and notes | LinkedIn Recruiter, LinkedIn Recruiter Lite |
| Spreadsheets and personal files | Recruiter-maintained candidate lists; often for specific searches, niches, or VIP candidates; may contain more current information than the ATS | Excel, Google Sheets, personal CRM tools |
| Career fair and event contacts | Business cards, sign-up sheets, badge scans from career fairs, networking events, industry conferences | Often paper-based or in event-specific apps; data must be manually entered or imported into the ATS |
| Legacy ATS (post-migration) | Candidate data from a previous ATS that was migrated to the current system; migration may have introduced duplicates, data loss, or formatting issues | Any previous ATS the agency used |
Cleaning Workflows
Full database audit
| Step | Action | Details |
|---|---|---|
| 1. Export from all systems | Export candidate records from: ATS (CSV), LinkedIn Recruiter (CSV export of saved profiles), job board accounts (CSV where available), recruiter spreadsheets (XLSX, CSV), event contact lists (XLSX, CSV) | Include: candidate ID (ATS-assigned), first name, last name, email (all known), phone (all known), current title, current employer, city/state, skills, last activity date, source, recruiter owner |
| 2. Extract and deduplicate emails | Upload all exported files to Email Extractor to extract and deduplicate email addresses across all systems | The deduplicated email list reveals the true unique candidate count; source-tagged results show which system has each candidate's email and which systems are missing it; candidates with multiple email addresses (personal and work) are identified |
| 3. Identify duplicate records | Match on: email address (most reliable); first name + last name + city; first name + last name + employer; phone number match across systems | Create a review queue for potential duplicates; automated matching will produce false positives (two "John Smith" candidates who are different people); human review is essential |
| 4. Merge duplicates in ATS | Merge duplicate records preserving: all application history, all interview notes, all placement history, most current contact information, all skills and certifications from both records | Most ATS platforms (Bullhorn, JobAdder, Vincere) have built-in duplicate detection and merge tools; use them to merge within the ATS rather than exporting, deduplicating, and reimporting |
| 5. Verify email addresses | Run the deduplicated email list through a verification service (ZeroBounce, NeverBounce, Clearout) | Results: valid (safe to email), invalid (need new email), catch-all (monitor), disposable (flag for review -- candidates using disposable emails may not want to be contacted) |
| 6. Enrich incomplete records | For candidates missing email or phone: search LinkedIn, company websites, professional directories; for candidates with outdated titles: check LinkedIn for current position | Prioritise enrichment by candidate value: recently placed candidates and highly skilled candidates are worth more enrichment effort than cold records from 3 years ago |
| 7. Update ATS | Import cleaned, deduplicated, verified and enriched data back to the ATS; update contact information, skills, titles; flag invalid email addresses; suppress or archive records with no valid contact method | Configure the ATS to track the cleaning date and source of each update for compliance documentation |
Ongoing maintenance
| Frequency | Action | Why |
|---|---|---|
| Daily (automated) | ATS duplicate detection on new records: flag potential duplicates at the point of entry (when a new application or import occurs) | Preventing duplicates at entry is 10x easier than finding and merging them later; most modern ATS platforms can flag "this candidate may already exist" when a new record is created |
| Weekly | Process email bounces from outreach campaigns; update candidate records with bounce information; investigate and replace bounced email addresses | Recruiters who send weekly job alerts or marketing emails should process bounces weekly to maintain list health and sender reputation |
| Monthly | Review and merge newly flagged duplicates; update records for recently placed candidates (confirm start date, title, salary); archive records with no activity in 18-24 months (do not delete; archive for compliance) | Monthly maintenance prevents data quality from degrading; recently placed candidates are high-value records that should be accurate for redeployment |
| Quarterly | Email verification of active candidate pool (candidates contacted in the last 6 months or flagged as available); skills and title refresh for top candidates | Quarterly verification catches email decay before it impacts outreach campaigns; skills refresh ensures search results are current |
| Annually | Full database audit (steps 1-7 above); GDPR/consent review (are all records compliant?); archive or delete records per retention policy; report on database health metrics | Annual audit is the comprehensive reset; compliance review is essential for agencies processing EU candidate data |
ATS-Specific Workflows
| ATS | Export method | Duplicate detection | Merge capability | Bulk update |
|---|---|---|---|---|
| Bullhorn | Admin > Data Management > Export; REST API for programmatic export | Built-in duplicate detection on import (matches on email, name + phone); configurable matching rules | Built-in merge tool; preserves notes, activities, placements from both records | CSV import with field mapping; API for programmatic updates |
| JobAdder | Reports > Custom export; API access | Duplicate detection on candidate creation; flags potential matches | Built-in merge (select primary record; merge secondary into it) | CSV import; API |
| Vincere | Data management > Export; API | Duplicate detection on record creation | Built-in merge | CSV import; API |
| Zoho Recruit | Module > Export (CSV/XLS); API | Duplicate detection rules (configurable: email, phone, name) | Built-in merge; select master record | CSV import; API |
| Greenhouse | Requires API access or support export; no self-service bulk export in standard plans | Limited duplicate detection (primarily email match) | Manual merge by support or admin | API; some bulk actions available in admin |
Metrics
| Metric | Before cleaning (typical) | After cleaning | Impact |
|---|---|---|---|
| Duplicate rate | 15-30% of records | Under 5% | Accurate candidate count; no duplicate outreach; complete candidate history in one record |
| Email deliverability | 60-75% (recruiting databases have high bounce rates due to work email churn) | 85-95% (after verification and enrichment) | More candidates receive job opportunities; higher response rates; better sender reputation for email campaigns |
| Candidate reachability (has valid email or phone) | 55-70% | 80-90% (after enrichment) | Larger reachable candidate pool; more candidates available for each search |
| Search result relevance | Low (outdated titles and skills produce inaccurate boolean search results) | High (current titles and skills improve matching) | Recruiters find the right candidates faster; reduce time-to-fill; improve candidate quality for submissions |
| Time-to-fill (average) | Longer (recruiters spend time on unreachable candidates, duplicate outreach, manual deduplication) | Shorter (clean data means faster sourcing, fewer dead ends) | Faster placements mean faster revenue; improved client satisfaction |
| Redeployment rate (placing previously placed candidates in new roles) | 15-25% (many prior placements are unreachable due to outdated contact information) | 25-40% (accurate contact information enables redeployment outreach) | Redeployment is the most profitable placement type: no sourcing cost, known candidate quality, existing relationship |
Related Guides
- Cold Email for Recruiting Agencies
- Data Cleaning for Association Membership Databases
- Data Cleaning for Alumni Databases
- Email Verification for Recruiting and Staffing Firms
- Email Extraction from CRM Exports
