Orpheus

Sort and Remove Duplicate Lines

Clean up a list: sort it, drop repeats, trim the whitespace.

Options
Remove duplicates
Treat case as the same
Trim whitespace and drop empty lines
Result

Everything is processed in this tab. Nothing you paste is sent anywhere.

Every tool runs entirely in your browser. Your files are never uploaded to a server.

Sorting orders lines and deduplication removes repeats. The detail that matters is natural sorting: plain alphabetical order puts file10 before file2, because it compares character by character rather than reading the digits as a number.

How to use Sort & Deduplicate Lines

  1. Paste your list. One item per line — emails, filenames, IDs, anything.
  2. Choose how to sort. Natural order is what you want for anything numbered.
  3. Copy the cleaned list. Counts before and after are shown so you can see what was removed.

About sorting and deduplicating lines

List cleaning looks trivial and hides two genuine problems. The first is sort order. Comparing strings character by character is correct for words and wrong for anything containing numbers, which is why file10 lands before file2 and why a directory of invoices sorts into an order nobody wants. Natural sorting reads consecutive digits as a single number, and it is what almost everyone means by "sorted" when the items are numbered. There is a second layer beneath that: raw comparison uses character codes, so accented and non-Latin letters sort after Z rather than alongside their base letter, and uppercase sorts before all lowercase. Locale-aware collation fixes both and is what is used here. The second problem is invisible difference. Two lines that look identical frequently are not — a trailing space, a non-breaking space pasted from a web page, or a carriage return left by a Windows file will all defeat deduplication while showing nothing on screen. Trimming resolves most of it, and it is why the counts before and after are worth reading: a deduplication that removes nothing usually means the duplicates differ in a character you cannot see.

Frequently asked questions

Why does file10 come before file2 when I sort?
Because alphabetical sorting compares character by character, and "1" comes before "2". Natural sorting reads runs of digits as numbers instead, which produces the order a person expects. Use it for anything with a number in the name.
Does removing duplicates keep the first or last occurrence?
The first. That preserves the original ordering of whatever survives, which matters when you have chosen not to sort — the list stays in the sequence you supplied, minus the repeats.
Why do two lines that look identical not count as duplicates?
Almost always trailing whitespace, a non-breaking space pasted from a web page, or Windows line endings leaving a stray carriage return. Turning on trimming resolves the first two; the third is handled automatically.
Is sorting case-sensitive?
By default yes, using your locale’s collation rather than raw character codes — so accented letters sort next to their base letter rather than after Z. Turn on case-insensitivity to treat Apple and apple as the same item.

Last updated