Remove Duplicate Lines is a free online tool that deletes repeated lines from a list or text, keeps the first copy of each line in its original order and shows how many it removed. Options let you ignore case, ignore spaces at the line ends and drop empty lines, for up to 500,000 characters, with no sign-up.
How to remove duplicate lines
- Paste your list. Paste the text in the Text or list box, with one item on each line.
- Set the options. Tick Ignore case if capital and small letters should count as the same. Tick Remove empty lines to drop blank lines.
- Copy the result. The list without duplicates appears in the Lines without duplicates box, with the number of lines removed. Press Copy.
What is a duplicate line?
A duplicate line is a line that has the same text as a line above it, and to remove duplicate lines is to keep one copy and delete the rest. Duplicates appear when lists are merged, for example two lists of emails, keywords or names.
This tool keeps the first time a line appears and removes every later copy. The lines that remain stay in their original order.
Features
- Keeps the first copy of each line and the original order.
- Ignore case option for lists where Apple and apple are the same item.
- Ignores spaces at the start and end of a line when it compares.
- Counts the duplicates removed and the lines left.
- Copy, select or download the result as a .txt file.
The options
| Option | Off | On |
|---|---|---|
| Ignore case | Apple and apple are two different lines | Apple and apple are the same line. The first one stays |
| Ignore spaces at the start and end | A line with a space at the end is a different line | Spaces at both ends are ignored when lines are compared |
| Remove empty lines | Empty lines stay where they are | Empty lines are removed |
Empty lines are never counted as duplicates. They are removed only when you tick Remove empty lines.
Is it private?
Yes. The list is processed inside your browser. It is not sent to our server and is not saved by us.
Tips for removing duplicate lines
- Put one item on each line. The tool compares whole lines, so items that share a line count as one item.
- Tick Ignore case for names and words. Apple and apple then count as the same line, and the first one stays.
- Clean the spaces inside lines first. A capital letter, a double space inside a line or a different kind of quote makes two lines that look the same count as different. Clean the spaces, or tick Ignore case, then remove the duplicates.
- Keep Ignore spaces at the start and end on. It is on from the start, so a line with a space at the end still counts as a copy of the same line without it.
- Paste up to 500,000 characters at a time. For a bigger list, run it in parts, join the results and run the joined list once more to catch copies across the parts.
Related tools
Three other tools on this site work on lists. The sort text tool puts the lines in alphabetical order. The tool to remove extra spaces cleans spaces that make two equal lines look different. The text compare tool shows which lines differ between two lists.
Frequently asked questions
- How do I remove duplicate lines from a list?
- Paste the list with one item per line. The duplicates are removed at once and the unique lines appear in the result box.
- Which copy of a duplicate is kept?
- The first one. Every later copy of the same line is removed.
- Does the order of my lines change?
- No. The lines that remain keep their original order.
- Can it treat Apple and apple as the same line?
- Yes. Tick Ignore case.
- Is the Remove Duplicate Lines tool free?
- Yes. It is free and you do not need an account.