WebTools

Text Tools

Remove Duplicate Lines

Paste a list and keep each line once, with options for case, spaces, blanks and sorting.

By Nitin Raturi · Last reviewed

Input

How to use Remove Duplicate Lines

Paste a list and keep only the first copy of each line.

  1. Paste your list, one item per line.
  2. Choose whether case and spaces matter, and whether to drop blank lines.
  3. Pick an order: original, A to Z or Z to A.
  4. Copy the cleaned list and check the counts and repeated lines.

Good to know

  • Your text never leaves your browser or goes into the link.
  • Leave the order as original when the sequence matters.

Keep the first copy of every line, with options for case, spaces, blank lines and sorting.

Runs in your browser. Nothing you type is uploaded or saved in the link.

Your lines

The list to clean: one item per line.

How to fill it in

  • Type or paste up to 5,00,000 characters.
  • Each line break starts a new item.

Example

The sample has 8 lines; 5 are left after removing 2 duplicates and 1 blank.

Ignore case

Treat lines that differ only in capital letters as the same.

How to fill it in

  • Tick it to merge “Asha” and “asha”. The first spelling seen is kept.

Example

With case ignored, the sample keeps 4 lines instead of 5.

Trim spaces

Ignore spaces and tabs at the start and end of each line, and remove them.

How to fill it in

  • Leave it on for pasted lists, where stray spaces are common.

Example

Without trimming or blank removal, “ Ravi ” stays separate and 7 lines are kept.

Remove blank lines

Drop empty lines instead of keeping one blank line.

How to fill it in

  • With trim on, lines of only spaces count as blank too.

Order

Keep the original order, or sort the remaining lines.

How to fill it in

  • Original keeps each line where it first appeared.
  • A to Z and Z to A ignore capitals when ordering; numbers sort as text, so 10 comes before 9.

Example

Sorted A to Z with case ignored: Asha, Kabir, Meera, Ravi.

Result

Unique lines

Asha
Ravi
asha
Meera
Kabir

Lines in
8
Lines kept
5
Duplicates removed
2
Blank lines removed
1

Most repeated

Ravi
3 times

ChangelogHow we test this toolEmbed this tool

How this calculator works

  1. Step 1Splitone per lineEvery line break starts a new item.
  2. Step 2Comparecase, spacesYour options decide which lines count as the same.
  3. Step 3Keep firstfirst copyLater copies are dropped; order is kept unless you sort.

The tool reads your text line by line and keeps the first copy of each one. Everything runs in your browser; the list is never uploaded or put in the link.

Worked example: a pasted list of names

Cleaning eight pasted lines
Defaults
trim spaces, remove blanks, case matters
  1. Pasted“Asha”, “Ravi”, “asha”, “ Ravi ”, (blank), “Meera”, “Ravi”, “Kabir”8 lines
  2. Blank removed−1
  3. Duplicates removedtwo more Ravi−2
  4. KeptAsha, Ravi, asha, Meera, Kabir5 lines

“ Ravi ” matches “Ravi” because trimming removes the stray spaces. “asha” stays, because case matters by default.

How the options change the count

  • No trim, keep blanks7 lines
  • Defaults5 lines
  • Ignore case too4 lines
Lines kept from the same sample

Each option merges more lines. Sorted A to Z with case ignored, the list becomes Asha, Kabir, Meera, Ravi.

Read the result

Lines kept + duplicates removed + blank lines removed always equals the lines you pasted.

The “Most repeated” list shows which lines appeared more than once, so you can spot a double entry before you send a list on.

Assumptions and limits

  • Exact by defaultEvery character counts unless you ignore case.
  • Spaces and tabsTrim removes only these, at each end.
  • Text sortNumbers sort as text: 10 comes before 9.
  • Not fuzzyNear-duplicates and typos are not merged.

Related text tools

Removing duplicates is one way to clean a list. Comparing and counting are others.

To see what changed between two lists, use the diff checker. The word counter counts lines and words, and the case converter makes entries consistent first.

Common mistakes

  • Mistake: Sorting a list whose order matters.

    Instead: Keep the original order.

  • Mistake: Ignoring case for codes or passwords.

    Instead: Leave case on when capitals carry meaning.

  • Mistake: Expecting “Ravi K” to match “Ravi”.

    Instead: Only identical lines merge; edit near-duplicates by hand.

Checking the method

A separate script re-works every golden value with list membership and Python's own sort, without a hash map.

Read how we test our calculators, or browse more everyday tools.

Frequently asked questions

Which copy of a duplicate line is kept?

The first one, in its original position and spelling. Later copies are removed.

Can it ignore capital letters?

Yes. Tick Ignore case and “Asha” and “asha” merge; the sample then keeps 4 lines instead of 5.

Does it remove lines that differ only by spaces?

With Trim spaces on, spaces and tabs at either end are ignored and removed, so they merge.

How does sorting work?

A to Z ignores capitals and compares character codes, so numbers sort as text. The sample sorts to Asha, Kabir, Meera, Ravi.

Is my text uploaded?

No. It is processed in your browser and never put in the link.

Sources and methodology

The formulas are standard time-value-of-money results. We check every figure against golden values worked out independently, by closed form and by month-by-month simulation; see the methodology.

Written and maintained by Nitin Raturi, a software engineer (about the author). Last reviewed 4 Nov 2026; next review due by . Found an error? Tell us.

Changelog

  • : Initial release.

  • Word & Character Counter

    Count words, characters, sentences, paragraphs and lines, with reading and speaking time. Works for Hindi too.

  • Case Converter

    Change text to upper, lower, title or sentence case, or to camelCase, snake_case and kebab-case.

  • Diff Checker

    Compare two texts or code files line by line, inline or side by side, with changed words highlighted.