How to remove duplicate lines
- Paste your list. Paste the text in the Text or list box, with one item on each line.
- Set the options. Tick Ignore case if capital and small letters should count as the same. Tick Remove empty lines to drop blank lines.
- Copy the result. The list without duplicates appears in the Lines without duplicates box, with the number of lines removed. Press Copy.
What is a duplicate line?
A duplicate line is a line that has the same text as a line above it. Duplicates appear when lists are merged, for example two lists of emails, keywords or names.
This tool keeps the first time a line appears and removes every later copy. The lines that remain stay in their original order.
Features
- Keeps the first copy of each line and the original order.
- Ignore case option for lists where Apple and apple are the same item.
- Ignores spaces at the start and end of a line when it compares.
- Counts the duplicates removed and the lines left.
- Copy, select or download the result as a .txt file.
The options
| Option | Off | On |
|---|---|---|
| Ignore case | Apple and apple are two different lines | Apple and apple are the same line. The first one stays |
| Ignore spaces at the start and end | A line with a space at the end is a different line | Spaces at both ends are ignored when lines are compared |
| Remove empty lines | Empty lines stay where they are | Empty lines are removed |
Empty lines are never counted as duplicates. They are removed only when you tick Remove empty lines.
Why the result may not look right
- Two lines look the same but both stay. They differ in a detail you cannot see easily, such as a capital letter, a double space inside the line or a different kind of quote. Tick Ignore case, or clean the spaces first.
- Items on one line are not separated. The tool compares whole lines. Put each item on its own line first.
- The text stops growing. The box takes up to 500,000 characters.
Related tools
Three other tools on this site work on lists. The sort text tool puts the lines in alphabetical order. The tool to remove extra spaces cleans spaces that make two equal lines look different. The text compare tool shows which lines differ between two lists.
Is it private?
Yes. The list is processed inside your browser. It is not sent to our server and is not saved by us.
Frequently asked questions
- How do I remove duplicate lines from a list?
- Paste the list with one item per line. The duplicates are removed at once and the unique lines appear in the result box.
- Which copy of a duplicate is kept?
- The first one. Every later copy of the same line is removed.
- Does the order of my lines change?
- No. The lines that remain keep their original order.
- Can it treat Apple and apple as the same line?
- Yes. Tick Ignore case.
- Is the Remove Duplicate Lines tool free?
- Yes. It is free and you do not need an account.