Data Cleaning for Healthcare Provider Networks, Physician Practice Databases, Hospital Staff Directories, Credentialing Systems and Medical Group Contact Lists
On this page
The Provider Data Problem
Healthcare provider networks maintain some of the most complex contact databases of any industry. A single physician may appear in a hospital's credentialing system, a health plan's provider directory, a practice management system, the NPI registry, a state medical board database, multiple hospital medical staff offices, a physician recruitment database, a continuing medical education system and several referral management platforms. Each system stores the provider's information independently, and the records drift apart as the provider changes practice locations, joins or leaves medical groups, updates their phone number in one system but not others, or acquires new certifications:
| System | What it stores | How it gets stale |
|---|---|---|
| Credentialing system (Cactus, Modio, CredentialStream, VerityStream) | Provider demographics, medical education, training, board certifications, licensure, DEA registration, malpractice history, work history, references, privilege delineation | Credentialing cycles are every 2-3 years; between cycles, address changes, new certifications, new privileges and practice location changes may not be captured |
| Practice management system / EHR (Epic, Cerner, Athenahealth, eClinicalWorks) | Provider name, NPI, specialty, practice location, scheduling information, patient panel | Provider profiles are maintained by individual practices; when a provider leaves a practice, the record may be deactivated but not updated with the new practice location |
| NPI registry (NPPES) | Provider name, NPI number, taxonomy (specialty), practice address, mailing address | Providers are supposed to update NPPES within 30 days of a change but many do not; NPI records are frequently outdated by 6-12+ months |
| Health plan provider directory | Provider name, NPI, specialty, accepting new patients status, practice address, phone, languages, hospital affiliations | CMS and state regulations require directory accuracy (No Surprises Act requires quarterly verification); despite requirements, 40-50% of provider directory entries contain at least one inaccuracy (CMS studies) |
| Medical staff office records | Credentials, privileges, committee assignments, call schedules, peer review participation, contact information | Medical staff offices manage data for their hospital; a provider with privileges at 3 hospitals has 3 separate records that must be kept consistent |
| State medical board | Licence status, licence number, disciplinary actions, address of record | Address updates depend on the provider notifying the board; many providers update their licence address only at renewal time |
| Referral management platform | Referring provider contact information, specialty, preferred communication method, fax number, secure messaging address | Referral databases are built over years of accumulated contact information; old fax numbers, closed practices and retired providers persist |
Common Data Quality Issues
| Issue | Prevalence | Impact | Example |
|---|---|---|---|
| Duplicate provider records | 10-20% of records in large provider databases are duplicates | Duplicate records cause confusion in credentialing (is this provider credentialed?), directory errors (multiple listings), and reporting inaccuracies (provider appears twice in panel counts) | Dr. Smith, Jane vs. Jane Smith, MD vs. J. Smith MD -- all the same provider with 3 separate records |
| Outdated practice address | 15-30% of provider addresses are incorrect (based on CMS provider directory audits) | Patients cannot find the provider; referral correspondence goes to the wrong address; credentialing communications are not received | Provider moved from Main Street Medical to University Health Partners 8 months ago; 4 of 6 systems still show the old address |
| Incorrect phone/fax numbers | 10-25% of phone and fax numbers are outdated | Referral faxes fail; patient calls go to disconnected numbers; appointment scheduling fails | Practice changed its phone system; new number in the practice management system but old number in the health plan directory and referral platform |
| Mismatched NPI/name combinations | 5-10% of records have NPI-name mismatches or incorrect NPI associations | Billing errors (wrong NPI on claims); credentialing confusion; regulatory compliance issues | Two providers with similar names at the same practice; their NPIs are swapped in one system |
| Stale credentialing status | 5-15% of credentialing records do not reflect current licence/certification status | Provider with an expired licence may still appear as credentialed; provider who completed re-credentialing may not be updated in all systems | Board certification expired 3 months ago but credentialing system still shows "board certified" |
| Missing email addresses | 20-40% of provider records lack an email address | Cannot communicate electronically; paper-based processes persist; slower credentialing; missed regulatory notifications | Provider was added to the system 10 years ago when fax was the primary communication method; no email was collected |
| Inconsistent specialty coding | 10-20% of records have specialty mismatches across systems | Provider directory shows wrong specialty; patients are matched to inappropriate providers; referral patterns are distorted | Cardiologist coded as "Internal Medicine" in one system and "Cardiovascular Disease" in another; correct taxonomy code: 207RC0000X |
Cleaning Workflows
Step 1: Export and consolidate
| Source system | Export format | Key fields to export |
|---|---|---|
| Credentialing system | CSV | Provider name (first, middle, last, suffix), NPI, date of birth, medical school, graduation year, specialty, board certifications, licence numbers, licence states, licence expiration dates, DEA number, practice address, mailing address, phone, fax, email |
| Practice management / EHR | CSV, XLSX | Provider name, NPI, specialty (taxonomy code), practice locations (all), phone, fax, email, accepting new patients, panel status |
| NPI registry (NPPES) | CSV (downloadable from CMS) | NPI, provider name, credential, taxonomy code (specialty), practice address, mailing address, phone, fax, authorised official |
| Health plan provider directory | CSV, XLSX | Provider name, NPI, specialty, network status, practice address, phone, fax, accepting new patients, languages, hospital affiliations |
| Medical staff office | CSV, XLSX | Provider name, medical staff category (active, courtesy, consulting), privileges, committee assignments, contact information |
Upload all exported files to Email Extractor to extract and deduplicate email addresses across all systems. The deduplicated email list reveals how many unique providers exist across all systems (compared to the total record count, which includes duplicates) and identifies providers whose email appears in some systems but not others.
Step 2: Create a master provider index
| Field | Source of truth | Validation method |
|---|---|---|
| NPI | NPPES registry | Validate each NPI against the NPPES API; confirm NPI-name match; flag mismatches |
| Legal name | State medical board | Cross-reference credentialing system name against state board licence records |
| Specialty (taxonomy) | NPPES taxonomy code + board certification | Primary taxonomy from NPPES; validate against board certification records; flag mismatches for review |
| Licence status | State medical board | Query state board database for current licence status; flag expired, suspended or revoked licences |
| Board certification | ABMS (for physicians) or relevant certification body | Verify through ABMS certification verification or specialty board websites |
| Practice address | Provider self-attestation (most recent) | Use the most recently updated address from any system; for health plan directories, CMS requires provider attestation of address accuracy |
| Most recent verified email from any system | After deduplication, use the most recently verified email; prioritise institutional email over personal | |
| Phone/fax | Practice management system (most operationally current) | Call the number to verify it reaches the correct practice (for high-priority providers) |
Step 3: Reconcile duplicates
| Matching criteria | How to identify | Resolution |
|---|---|---|
| Exact NPI match | Same NPI in multiple systems or multiple records within one system | Merge records; use a single NPI as the unique identifier; consolidate contact information from all sources |
| Name + date of birth match | Same name and DOB but different NPIs or missing NPI | Verify if this is the same provider (could be a data entry error, name change, or genuinely different providers with the same name); confirm NPI assignment |
| Name + specialty + address match | Same name, specialty and practice address but different records | Likely duplicates; confirm by NPI or licence number; merge if confirmed |
| Similar name + same practice | Similar but not identical names at the same practice address (e.g., "Catherine Smith" vs "Cathy Smith") | May be the same provider; verify by NPI, licence number or date of birth |
Regulatory Requirements for Directory Accuracy
| Regulation | Requirement | Penalty for non-compliance |
|---|---|---|
| No Surprises Act (federal, effective 2022) | Health plans must verify and update provider directory information every 90 days; directory must reflect within 2 business days when a provider leaves the network | Plans may be liable for out-of-network costs if a patient relied on an inaccurate directory and received out-of-network care |
| CMS Medicare Advantage / Medicaid managed care | Provider directories must be updated monthly; CMS audits directory accuracy; "secret shopper" calls to verify phone numbers and accepting-new-patients status | CMS may impose civil monetary penalties; require corrective action plans; affect star ratings |
| State insurance regulations | Most states require health plan provider directories to be updated quarterly or more frequently; many states require online directories to be updated within 5-30 days of a change | State regulatory action; fines; required consumer notification |
| Joint Commission (hospital accreditation) | Medical staff credentialing and privileging must be current; provider credential files must be complete and verified | Accreditation risk; hospitals must demonstrate current credentialing for all providers with privileges |
| NCQA (health plan accreditation) | Credentialing and re-credentialing standards require verified provider information; directory accuracy is assessed | NCQA accreditation affects health plan competitiveness; Medicare Advantage plans may need NCQA accreditation |
Maintenance Schedule
| Frequency | Activity | Systems involved |
|---|---|---|
| Daily | Process incoming provider updates (address changes, new hires, terminations); propagate changes to all systems | Credentialing system, provider directory, practice management system |
| Weekly | Review bounce-back mail and returned faxes; update contact information for failed deliveries | All systems with provider contact information |
| Monthly | Reconcile provider directory against credentialing system; identify providers whose credentials are expiring in the next 90 days; update health plan directory per CMS requirements | Credentialing system, provider directory |
| Quarterly | Full provider directory attestation (outreach to providers to confirm address, phone, specialty, accepting-new-patients status); NPPES data refresh; state board licence status check | Provider directory, NPPES, state medical boards |
| At re-credentialing (every 2-3 years) | Complete credential verification including licence, board certification, DEA, malpractice history, education, training; update all demographic information | Credentialing system; all downstream systems |
