Remove Duplicate Lines
Remove duplicate lines — ignoring case, spaces or punctuation — keep the first or last copy, or list only the duplicates, the unique lines or how often each line appears.
- Your data never leaves your device
- 100% free
- No sign-up
- Works offline after first load
How to remove duplicate lines
- Paste your list or open a text, CSV or Word file.
- Choose how lines are compared and what to keep; sort the result if you like.
- Copy or download the list without duplicates.
About Remove Duplicate Lines
Clean up lists of emails, keywords, names, URLs or log lines by removing repeated lines. Choose whether upper and lower case, extra spaces or punctuation make lines different, keep the first or last copy, and sort the result.
Or analyse a list instead: show only the lines that are repeated, only the ones that appear once, or how many times every line appears. Big lists are handled in a moment, in your browser.
Good for
- Cleaning email and contact lists
- Deduplicating keyword and URL lists
- Finding repeated entries in logs and data
Tip: Use “Count each line” with “Most frequent first” to see which entries repeat most.
Frequently asked questions
Can it ignore upper and lower case?
Yes, “Apple” and “apple” count as the same line when “Ignore upper / lower case” is on — and you can also ignore extra spaces and punctuation.
Which copy is kept?
The first one by default, in its original position; you can keep the last one instead.
Can I see which lines were duplicated?
Yes — “Only duplicates” lists them, “Only lines that appear once” shows the rest, and “Count each line” shows how often every line appears.
Does it work with big lists?
Yes, lists with hundreds of thousands of lines are processed in a moment in your browser.
Can I open files?
Yes — open or drop TXT, Markdown, CSV, HTML, JSON, subtitles, Word (.docx), OpenDocument (.odt), RTF and PDF files, even several at once.
Is my list uploaded?
No. Duplicates are removed in your browser, so your list stays on your device. A draft is kept only in this browser so you don't lose it.