Remove duplicate words

Remove all repeats. Every word is kept the first time it appears and removed every time after that. Commas and spaces around removed words are tidied so you don't end up with , , or double spaces. Ideal for keyword lists and tag lists:

apple, pear, apple, plum, Pear, peach → apple, pear, plum, peach

Only back-to-back repeats. Removes a word only when it repeats immediately, the classic typo where the same word is typed twice: This this is a sentence where one word appears twice twice. → This is a sentence where one word appears twice. A repeat across a sentence boundary (…ends here. Here it…) is left alone, because that's usually intentional.

List of unique words. Outputs each distinct word once, one per line (or one line per input line, if you choose per-line scope). Punctuation is dropped. Good for building vocabulary lists or turning a paragraph into tags.

The note under the result shows how many repeats were removed, and a table lists each removed word with the number of times it was removed. The table can be downloaded as CSV.

• Where does a repeat count? Anywhere in the text finds repeats across the whole text; Within each line starts fresh on every line, so a word can appear once per line. Use per-line for lists where each line is a separate record, like product titles.
• Ignore case. On by default: Pear counts as a repeat of pear.
• Keep common words. Protects short function words like a, the, and, of, to, is so that normal sentences don't lose every second the. Turn it on when cleaning prose in “remove all repeats” mode.
• Keep words shorter than (letters). Another way to protect short words: set 4 and words of one to three letters are never removed.

Words include letters with accents and digits, and hyphenated words and contractions count as one word, like in the word counter.

Task	Mode	Options
Fix accidental doubles in an essay	Back-to-back	Ignore case on
Clean a keyword or tag list	Remove all repeats	Ignore case on
Deduplicate words in each product title	Remove all repeats	Within each line, keep common words
Build a vocabulary list from a text	Unique word list	Ignore case on

"Remove all repeats" on regular prose usually makes it unreadable, since natural sentences repeat words like the and is all the time. For prose, use back-to-back mode or protect common and short words.

• To remove repeated lines rather than words, use remove duplicate lines.
• To count how often each word appears without removing anything, use the word frequency counter.
• To clean up double spaces and stray punctuation afterwards, use remove extra spaces.

All processing happens in your browser; the text is not uploaded.