CSV Deduplicator

How this reads your output

Rows identical across every field are collapsed to one. Separately, rows sharing a value in the first column but differing elsewhere are counted and listed, and deliberately not removed. That second category is the one that causes real damage: two records claiming to be the same customer with different addresses will not be noticed by anyone, whereas an exact repeat is obvious once found.

What the results mean

Exact duplicates removed

Rows matching in every field. Safe to remove, because no information is lost.

Conflicting values

The same key appearing with different data. Left in place, because deciding which version is correct needs knowledge no tool has.

Rows kept

What remains after deduplication, in the original order with the first occurrence of each row retained.

Common problems and fixes

Obvious duplicates are not being removed

They differ somewhere, usually in trailing whitespace, capitalisation or a formatting difference such as a phone number written two ways. Normalise those fields first, then deduplicate. Doing it in that order is important, since normalising afterwards leaves you deciding which of two surviving rows to keep.

I need to deduplicate on one column rather than the whole row

The conflicting-values list is exactly that, computed on the first column. Reorder your columns so the key you care about comes first, and the report will show every record sharing that key with different data behind it.

Frequently asked questions

More in Data

• CSV SplitterSplit a long CSV into parts that each keep the header.
• CSV ViewerView a CSV as a table — and find the rows that do not line up.
• CSV to Markdown TableTurn a CSV into a Markdown table, with pipes escaped properly.