`uniq` Report or filter repeated lines

Collapses repeated lines into one, or counts them. It only compares lines that are next to each other, so it is almost always used after sort. Reach for it to count how often each value appears.

Synopsis

uniq [OPTION]... [INPUT [OUTPUT]]

Common options

Option or form	What it does
`-c`	Put the number of times each line occurred in front of it
`-d`	Show only lines that appear more than once, one copy of each
`-u`	Show only lines that appear exactly once
`-i`	Ignore case when comparing lines
`-f num`	Skip the first num whitespace-separated fields on each line when comparing
`-s chars`	Skip the first chars characters on each line when comparing

Examples

Remove repeated neighbouring lines

uniq words.txt

pear
apple
pear
fig
apple

Remove all duplicates (sort first)

sort words.txt | uniq

apple
fig
pear

Count how often each word appears

sort words.txt | uniq -c

Most common words first

sort words.txt | uniq -c | sort -rn

Show only the words that appear more than once

sort words.txt | uniq -d

apple
pear

Show only the words that appear once

sort words.txt | uniq -u

fig

Notes

uniq only collapses adjacent duplicates. Run it on unsorted input and non-adjacent repeats survive. In words.txt, pear appears twice but not side by side, so plain uniq keeps both.
sort -u gives the same result as sort | uniq, and is shorter. You still need sort | uniq when you want -c, -d or -u.
The counts from -c are padded with leading spaces. Pipe through sort -rn to rank them, or awk to reformat them.
The second positional argument is an output file. uniq in out writes to out. A typo in the arguments can overwrite a file.

uniq Report or filter repeated lines

Synopsis

Common options

Examples

Notes

See also

`uniq` Report or filter repeated lines