Using Excel Power Query to Clean and Transform Email Data
On this page
What Power Query Does for Email Data
Power Query is a data transformation tool built into Excel (Windows and Mac, Microsoft 365) and Google Sheets (via Apps Script or add-ons). It lets you connect to data sources, clean and reshape the data, and load the results into a spreadsheet. For email list management, Power Query handles tasks that are tedious or error-prone with formulas:
| Task | Formulas approach | Power Query approach |
|---|---|---|
| Lowercase all emails | =LOWER(A2) dragged down | Transform > Lowercase, applied to entire column |
| Remove duplicates | Remove Duplicates button (destructive) | Remove Rows > Remove Duplicates (non-destructive; refresh anytime) |
| Split name and email from "Name <email>" | Complex formula with FIND, MID, SUBSTITUTE | Split Column > By Delimiter |
| Merge two email lists | Manual copy-paste and dedup | Append Queries; then Remove Duplicates |
| Fix common domain typos | Nested SUBSTITUTE formulas | Replace Values or custom function |
| Trim whitespace | =TRIM(A2) | Transform > Trim |
| Filter out role-based addresses | Complex IF/OR formula | Filter Rows with text conditions |
| Standardise format across sources | Manual per-source adjustment | One query per source; standardise in transformation steps |
Why Power Query instead of formulas
| Advantage | Explanation |
|---|---|
| Non-destructive | Original data is never changed; transformations are applied on refresh |
| Repeatable | Refresh the query to re-apply all steps to updated source data |
| Auditable | Every transformation step is listed in the Applied Steps pane |
| Scalable | Handles hundreds of thousands of rows without slowing down |
| Combinable | Merge and append multiple data sources into one clean output |
| No formula errors | Point-and-click interface; no risk of dragging formulas incorrectly |
Getting Started with Power Query for Email Data
Opening Power Query
| Excel version | How to access |
|---|---|
| Excel for Microsoft 365 (Windows) | Data tab > Get Data > From File or From Table/Range |
| Excel 2019/2021 (Windows) | Data tab > Get Data > From File or From Table/Range |
| Excel for Mac (Microsoft 365) | Data tab > Get Data > From File or From Table/Range (limited features) |
| Excel 2016 (Windows) | Data tab > Get & Transform section |
Loading email data into Power Query
| Source format | How to load | Steps |
|---|---|---|
| CSV file | Data > Get Data > From File > From Text/CSV | Select file; Power Query opens a preview |
| Excel table | Select data range > Data > From Table/Range | Data loads into Power Query editor |
| Excel workbook | Data > Get Data > From File > From Workbook | Select workbook; choose sheet or named range |
| Multiple CSV files from a folder | Data > Get Data > From File > From Folder | Select folder; Power Query combines all CSVs |
| Web page (table) | Data > Get Data > From Other Sources > From Web | Enter URL; select the table on the page |
Common Email Cleaning Transformations
Step 1: Trim and lowercase
Whitespace and case inconsistencies are the most common data quality issues:
| Transformation | How to apply | M code |
|---|---|---|
| Trim whitespace | Transform > Format > Trim | Text.Trim([Email]) |
| Lowercase | Transform > Format > lowercase | Text.Lower([Email]) |
| Combined | Add Custom Column | Text.Lower(Text.Trim([Email])) |
Power Query M formula (Add Custom Column):
Text.Lower(Text.Trim([Email]))
Step 2: Validate email format
Filter out rows that are not valid email addresses:
Keep only rows containing @ with text on both sides:
// Custom column: IsValidEmail
Text.Contains([Email], "@") and
Text.Length(Text.BeforeDelimiter([Email], "@")) > 0 and
Text.Length(Text.AfterDelimiter([Email], "@")) > 0 and
Text.Contains(Text.AfterDelimiter([Email], "@"), ".")
Then filter to keep only rows where IsValidEmail is TRUE.
Step 3: Extract domain from email
Add Custom Column:
Text.AfterDelimiter([Email], "@")
This creates a Domain column useful for analysis, deduplication by company or filtering.
Step 4: Remove duplicates
| Method | Steps |
|---|---|
| Remove duplicate emails | Select Email column > Home > Remove Rows > Remove Duplicates |
| Keep first occurrence | Duplicates are removed in the order the data appears; first row is kept |
| Case-insensitive dedup | Lowercase first (Step 1), then remove duplicates |
Step 5: Fix common domain typos
Replace Values (Home > Replace Values) for each common typo:
| Find | Replace with |
|---|---|
| gmial.com | gmail.com |
| gmal.com | gmail.com |
| gamil.com | gmail.com |
| yaho.com | yahoo.com |
| hotmal.com | hotmail.com |
| outlok.com | outlook.com |
For many replacements, create a custom function:
// Create a function query called FixDomain
(domain as text) as text =>
let
fixes = [
gmial.com = "gmail.com",
gmal.com = "gmail.com",
gamil.com = "gmail.com",
yaho.com = "yahoo.com",
hotmal.com = "hotmail.com",
outlok.com = "outlook.com"
],
result = try Record.Field(fixes, domain)
otherwise domain
in
result
Step 6: Filter out unwanted addresses
| Filter type | Condition | Purpose |
|---|---|---|
| Role-based addresses | Email starts with info@, admin@, support@, sales@, etc. | Remove non-personal addresses |
| Disposable domains | Domain is in a list of known disposable email providers | Remove temporary addresses |
| Internal addresses | Domain is your own company domain | Remove internal contacts from prospect lists |
| No-reply addresses | Email starts with noreply@ or no-reply@ | Remove unmonitored addresses |
Filter role-based addresses (Custom Column + Filter):
// Add custom column: IsRoleBased
let
local = Text.BeforeDelimiter(Text.Lower([Email]), "@"),
rolePrefixes = {"info", "admin", "support", "sales",
"contact", "help", "billing",
"noreply", "no-reply", "webmaster",
"postmaster", "abuse"},
isRole = List.AnyTrue(
List.Transform(
rolePrefixes,
each Text.StartsWith(local, _)
)
)
in
isRole
Then filter to keep only rows where IsRoleBased is FALSE.
Advanced Transformations
Splitting "Name <email>" format
Some data sources combine name and email in a single field:
John Smith <john.smith@example.com>
Jane Doe <jane.doe@example.org>
Steps:
- Split Column > By Delimiter >
<> At each occurrence - The second column contains the email with a trailing
> - Replace Values: find
>, replace with empty string - Rename columns to Name and Email
- Trim both columns
Merging multiple email lists
| Step | Action |
|---|---|
| 1 | Load each source file as a separate query |
| 2 | Standardise column names (rename to match across queries) |
| 3 | Home > Append Queries > Append Queries as New |
| 4 | Select all source queries |
| 5 | Remove duplicates on the combined query |
| 6 | Add a Source column to track origin (if not already present) |
Adding a source identifier before appending:
Add a Custom Column to each query before appending:
// In Query 1:
"conference-list" as source column
// In Query 2:
"website-scrape" as source column
Merging email list with company data
Use Merge Queries to join an email list with a company lookup table:
| Step | Action |
|---|---|
| 1 | Load both tables (email list and company data) |
| 2 | Extract domain from email in the email list |
| 3 | Home > Merge Queries |
| 4 | Select Domain column from email list and Domain column from company data |
| 5 | Choose join type (Left Outer to keep all emails; Inner to keep only matches) |
| 6 | Expand the merged company columns you need |
Deduplicating by company (keep one email per domain)
| Step | Action |
|---|---|
| 1 | Extract domain from email |
| 2 | Sort by a priority column (e.g., seniority, data freshness) |
| 3 | Remove Duplicates on the Domain column |
| Result | One email per company, prioritised by your sort order |
Working with Large Email Datasets
Performance tips
| Tip | Why it helps |
|---|---|
| Filter early | Remove unwanted rows as the first step; fewer rows means faster processing |
| Remove unnecessary columns | Drop columns you do not need; reduces memory usage |
| Use buffered tables for merges | Table.Buffer() loads the table into memory for faster joins |
| Disable background refresh during editing | Prevents unnecessary re-processing while you build the query |
| Use From Folder for many files | More efficient than opening each file individually |
Handling common issues
| Issue | Cause | Fix |
|---|---|---|
| "Query exceeded memory limit" | Dataset too large | Filter early; remove unnecessary columns; increase Excel memory |
| Slow refresh | Many transformation steps or large dataset | Simplify steps; use Table.Buffer for merges |
| Encoding issues (garbled characters) | Wrong encoding (UTF-8 vs ANSI) | Specify encoding when loading: File Origin setting |
| Merged cells in source | Excel source has merged cells | Unmerge in source or use Fill Down in Power Query |
| Mixed data types in column | Numbers and text mixed | Set column type explicitly; handle errors |
Automating Recurring Workflows
Setting up automatic refresh
| Method | How to configure |
|---|---|
| Manual refresh | Data > Refresh All (or right-click query > Refresh) |
| Refresh on file open | Query Properties > Refresh data when opening the file |
| Scheduled refresh (Power BI) | Publish to Power BI service; set scheduled refresh |
| VBA-triggered refresh | Macro to refresh queries on a schedule or button click |
Building a reusable cleaning template
| Step | Action |
|---|---|
| 1 | Create a workbook with all your cleaning queries |
| 2 | Point the source query at a specific file path or folder |
| 3 | When you have new data, place the file at that path |
| 4 | Refresh the query; all transformations apply automatically |
| 5 | Output goes to a designated output sheet |
Pre-Processing with Email Extractor
Before loading email data into Power Query, you may need to extract email addresses from unstructured sources. Upload files to Email Extractor to extract email addresses from PDF, HTML, DOCX, XLSX, CSV and other formats. Download the results as CSV, which loads directly into Power Query for further cleaning and transformation. This two-step workflow handles extraction (Email Extractor) and transformation (Power Query) separately, using each tool for what it does best.
Quick Reference: Common M Functions for Email Data
| Function | What it does | Example |
|---|---|---|
Text.Lower(text) |
Converts to lowercase | Text.Lower("John@Example.COM") returns "john@example.com" |
Text.Trim(text) |
Removes leading/trailing whitespace | Text.Trim(" email@example.com ") returns "email@example.com" |
Text.Contains(text, value) |
Checks if text contains a substring | Text.Contains("user@gmail.com", "@") returns true |
Text.BeforeDelimiter(text, delimiter) |
Returns text before the delimiter | Text.BeforeDelimiter("user@gmail.com", "@") returns "user" |
Text.AfterDelimiter(text, delimiter) |
Returns text after the delimiter | Text.AfterDelimiter("user@gmail.com", "@") returns "gmail.com" |
Text.StartsWith(text, value) |
Checks if text starts with a substring | Text.StartsWith("info@example.com", "info") returns true |
Text.Replace(text, old, new) |
Replaces occurrences | Text.Replace("gmial.com", "gmial", "gmail") returns "gmail.com" |
Text.Length(text) |
Returns character count | Text.Length("user@example.com") returns 16 |