What the result contains
What This Tool Provides
Remove repeated lines while retaining the first matching occurrence. Choose whether comparison trims outer whitespace, ignores letter case, or sorts the unique output, and review original, unique, and removed counts before copying.
- Unique lines as plain text
- Original, unique, and removed line totals
- First-occurrence preservation by default
- Optional trim, case-insensitive comparison, and sorting
How to Use This Tool
- Paste one item per lineAdd IDs, URLs, email addresses, SKUs, or other line-oriented values.
- Set comparison rulesChoose trim, case handling, and whether original order should remain.
- Review and copyConfirm the statistics and copy the unique output.
How It Works
CRLF and old CR line endings are normalized to LF. Each line receives the selected trim and case normalization for comparison, while the first retained value supplies the output spelling. A Set records keys already seen. Turning off Preserve order sorts the retained lines with locale-aware string comparison.
Worked Examples
Case-sensitive cleanup
- Input
Apple apple Banana Apple- Output
Apple apple Banana
Apple repeats exactly; lowercase apple remains distinct.
Ignore case
- Input
Apple apple Banana Apple- Output
Apple Banana
The first spelling is kept after all Apple variants share one comparison key.
Before and After the Trim
Duplicates hide behind formatting. Normalizing case first exposes entries that differ only in capitalization, and encoded lines will never match as text until they are decoded.
Afterwards, count what is left to confirm the list is the size you expected, and diff the two versions if you need to prove exactly which lines went.
What the Options Change
| Option | Enabled behavior | Disabled behavior |
|---|---|---|
| Trim whitespace | Remove outer spaces before comparing and output | Spaces remain significant |
| Ignore case | Apple and apple match | Letter case remains significant |
| Preserve order | Keep first-seen order | Sort unique output alphabetically |
Practical Uses
- Cleaning one-column email or ID lists
- Removing repeated URLs or log lines
- Deduplicating inventory SKUs
- Normalizing keyword lists before comparison
Important Edge Cases
- Blank lines are values; multiple blank lines collapse to one matching blank line.
- Whitespace inside a line is not changed by Trim whitespace.
- The tool is line-aware, not CSV-aware, so quoted multiline fields are not parsed.
- A line break at the very end of the text counts as one more, empty line, so a list pasted with a final newline reports one extra line in its totals.
- Ignore case lowercases letters but does not fold ß into ss, so STRASSE and straße stay as two separate lines.
- Trim whitespace strips tabs as well as spaces, so a tab-indented value matches the same value indented with spaces.
Limitations
- It does not validate email addresses, URLs, or IDs.
- Sorting is lexical rather than natural numeric sorting.
- Case folding follows browser Unicode behavior rather than database collation rules.
Input handling
Private Browser Processing
The list stays in the browser and is not submitted or placed in the URL.
Google Analytics records page visits and whether a tool finished or failed, but these tool implementations do not send field values, generated passwords, QR payloads, uploaded QR images, or camera frames to analytics or a Tiny Tiny Tools processing endpoint.
See the Privacy Policy for site-wide details.
Questions About This Tool
Which duplicate is preserved?
The first occurrence supplies the output line. Later matching lines are removed.
Does Ignore case change the output to lowercase?
No. Case is normalized only for comparison; the first retained line keeps its spelling.
Are Windows line endings supported?
Yes. CRLF and CR are normalized before lines are compared.
Can I use this with CSV?
Only for simple one-value-per-line data. It does not parse delimiters, quotes, or multiline CSV fields.