How to Deduplicate Email Lists in Google Sheets: Formulas, Apps Script and Add-Ons
On this page
Why Deduplication Matters
Duplicate email addresses in your spreadsheets cause real problems:
| Problem | Impact | How duplicates cause it |
|---|---|---|
| Sending duplicate emails | Recipients receive the same message twice; unprofessional | Same email in list twice |
| Inflated list metrics | Subscriber count looks higher than it actually is | Counting the same person multiple times |
| Wasted sending costs | Paying ESP for sends to the same person twice | Extra sends with no extra reach |
| Skewed analytics | Open and click rates are diluted or artificially inflated | Duplicate records distort calculations |
| CRM data quality issues | Duplicate contacts create confusion in sales pipeline | Imported lists carry duplicates into CRM |
| Compliance risk | Unsubscribed contact may still receive email from a duplicate record | Suppression list misses the duplicate |
Method 1: Remove Duplicates (Built-In Feature)
Google Sheets has a built-in tool for removing duplicates. This is the simplest method:
| Step | Action | Notes |
|---|---|---|
| 1 | Select the column (or range) containing email addresses | Click the column header to select the entire column |
| 2 | Go to Data > Data cleanup > Remove duplicates | Opens the Remove duplicates dialog |
| 3 | Check "Data has header row" if your first row is a label | Prevents the header from being treated as data |
| 4 | Select which columns to check for duplicates | Select only the email column for email-only deduplication |
| 5 | Click "Remove duplicates" | Shows how many duplicates were found and removed |
| Pros | Cons |
|---|---|
| Simple; no formulas needed | Destructive (permanently removes rows) |
| Fast for small to medium lists | No preview of what will be removed |
| Built into Google Sheets | Removes the second (and later) occurrence; keeps the first |
| Case-insensitive by default | Does not handle near-duplicates (e.g. spaces, formatting) |
Method 2: Highlight Duplicates with Conditional Formatting
If you want to see duplicates before removing them:
| Step | Action |
|---|---|
| 1 | Select the email column (e.g. A2:A1000) |
| 2 | Format > Conditional formatting |
| 3 | Set "Format rules" to "Custom formula is" |
| 4 | Enter: =COUNTIF($A$2:$A$1000, $A2) > 1 |
| 5 | Choose a fill colour (e.g. red or yellow) |
| 6 | Click "Done" |
This highlights all cells that appear more than once. You can then manually review and delete the duplicates you want to remove.
Method 3: UNIQUE Function
The UNIQUE function extracts a deduplicated list into a new column or sheet:
=UNIQUE(A2:A1000)
| Variation | Formula | What it does |
|---|---|---|
| Basic unique | =UNIQUE(A2:A1000) |
Returns unique values from column A |
| Case-insensitive unique | =UNIQUE(LOWER(A2:A1000)) |
Normalises to lowercase first, then deduplicates |
| Unique across multiple columns | =UNIQUE(A2:C1000) |
Deduplicates based on all columns (A, B, C must all match) |
| Unique with sorting | =SORT(UNIQUE(A2:A1000)) |
Returns unique values, sorted alphabetically |
| Pros | Cons |
|---|---|
| Non-destructive (original data preserved) | Output is in a separate column/range |
| Dynamic (updates as source data changes) | Cannot be edited directly (array formula) |
| Can be combined with other functions | Does not carry over associated data (names, etc.) |
Method 4: COUNTIF to Flag Duplicates
Use COUNTIF in a helper column to identify which rows are duplicates:
In cell B2 (assuming emails are in column A):
=COUNTIF($A$2:$A2, A2)
| Value in helper column | Meaning | Action |
|---|---|---|
| 1 | First occurrence | Keep |
| 2 | Second occurrence (duplicate) | Remove |
| 3+ | Third+ occurrence | Remove |
Then filter column B to show only values greater than 1, and delete those rows.
Checking for duplicates across two lists
If you have one list in Sheet1!A:A and want to check for duplicates against Sheet2!A:A:
In Sheet1, cell B2:
=COUNTIF(Sheet2!$A:$A, A2)
| Value | Meaning |
|---|---|
| 0 | Not in Sheet2 (unique to Sheet1) |
| 1+ | Also exists in Sheet2 (duplicate) |
Method 5: QUERY Function
For more control over deduplication, use QUERY:
=QUERY(A2:C1000,
"SELECT A, B, C
GROUP BY A, B, C", 1)
This groups by all columns and returns unique rows.
QUERY for email-only deduplication keeping first occurrence data
=QUERY(A2:C1000,
"SELECT A, MIN(B), MIN(C)
WHERE A IS NOT NULL
GROUP BY A
LABEL MIN(B) 'Name', MIN(C) 'Source'", 1)
Method 6: Apps Script for Advanced Deduplication
For large lists or complex deduplication logic, use Google Apps Script:
function deduplicateEmails() {
var sheet = SpreadsheetApp.getActiveSheet();
var data = sheet.getDataRange().getValues();
var header = data[0];
var emailCol = 0; // Column A (0-indexed)
// Track seen emails (case-insensitive)
var seen = {};
var uniqueRows = [header];
var duplicateCount = 0;
for (var i = 1; i < data.length; i++) {
var email = String(data[i][emailCol])
.toLowerCase()
.trim();
if (email === '' || email === 'undefined') {
continue;
}
if (!seen[email]) {
seen[email] = true;
uniqueRows.push(data[i]);
} else {
duplicateCount++;
}
}
// Write results to a new sheet
var newSheet = SpreadsheetApp.getActiveSpreadsheet()
.insertSheet('Deduplicated');
newSheet
.getRange(1, 1, uniqueRows.length, header.length)
.setValues(uniqueRows);
SpreadsheetApp.getUi().alert(
'Removed ' + duplicateCount + ' duplicates. ' +
'Results in "Deduplicated" sheet.'
);
}
How to use
| Step | Action |
|---|---|
| 1 | Extensions > Apps Script |
| 2 | Paste the code above |
| 3 | Save (Ctrl/Cmd + S) |
| 4 | Click Run |
| 5 | Authorise the script when prompted |
| 6 | Check the new "Deduplicated" sheet |
Advanced: normalise before deduplication
function normaliseAndDeduplicate() {
var sheet = SpreadsheetApp.getActiveSheet();
var data = sheet.getDataRange().getValues();
var header = data[0];
var emailCol = 0;
var seen = {};
var uniqueRows = [header];
var duplicateCount = 0;
for (var i = 1; i < data.length; i++) {
var rawEmail = String(data[i][emailCol]);
var email = rawEmail
.toLowerCase()
.trim()
.replace(/\s+/g, ''); // Remove spaces
// Skip empty
if (!email || email === 'undefined') {
continue;
}
// Basic format check
if (email.indexOf('@') === -1) {
continue;
}
// Normalise the row's email value
data[i][emailCol] = email;
if (!seen[email]) {
seen[email] = true;
uniqueRows.push(data[i]);
} else {
duplicateCount++;
}
}
var newSheet = SpreadsheetApp.getActiveSpreadsheet()
.insertSheet('Normalised');
newSheet
.getRange(1, 1, uniqueRows.length, header.length)
.setValues(uniqueRows);
SpreadsheetApp.getUi().alert(
'Normalised and removed ' + duplicateCount +
' duplicates.'
);
}
Method 7: Google Sheets Add-Ons
| Add-on | Features | Pricing |
|---|---|---|
| Remove Duplicates by Ablebits | Find, highlight, remove, move duplicates; compare sheets | Free basic; $30/year premium |
| Power Tools | Deduplication + many other data cleaning tools | Free trial; $30/year |
| Flookup | Compare and merge sheets; deduplication | Free basic; paid premium |
Method Comparison
| Method | Best for | Skill level | Handles case differences | Non-destructive | Scales to large lists |
|---|---|---|---|---|---|
| Remove Duplicates (built-in) | Quick cleanup | Beginner | Yes | No | Medium (up to ~50K) |
| Conditional formatting | Visual review before deletion | Beginner | Yes | Yes (highlighting only) | Medium |
| UNIQUE function | Dynamic deduplication | Intermediate | No (wrap in LOWER) | Yes | Medium |
| COUNTIF helper column | Flagging and reviewing | Intermediate | No (wrap in LOWER) | Yes | Medium |
| QUERY function | Complex deduplication with grouping | Intermediate-advanced | Partial | Yes | Medium |
| Apps Script | Large lists; advanced normalisation | Advanced | Yes (customisable) | Yes (writes to new sheet) | Large (100K+) |
| Add-ons | Non-technical users; complex rules | Beginner | Yes | Configurable | Medium-large |
When to Use a Dedicated Tool
Google Sheets works well for deduplicating email lists up to around 50,000-100,000 rows. For larger lists, or when you need to extract emails from non-spreadsheet formats (PDF, DOCX, HTML, EML), upload your files to Email Extractor. It handles case-insensitive deduplication automatically across all uploaded files and works with 19 file types, not just spreadsheets.