Counting Unique Words In Python

In direct, my code so far is this : from glob import glob pattern = 'D:\\report\\shakeall\\*.txt' filelist = glob(pattern) def countwords(fp): with open(fp) as fh: ret

Solution 1:

The best way to count objects in Python is to use collections.Counter class, which was created for that purposes. It acts like a Python dict but is a bit easier in use when counting. You can just pass a list of objects and it counts them for you automatically.

>>> from collections import Counter
>>> c = Counter(['hello', 'hello', 1])
>>> print c
Counter({'hello': 2, 1: 1})

Also Counter has some useful methods like most_common, visit documentation to learn more.

One method of Counter class that can also be very useful is update method. After you've instantiated Counter by passing a list of objects, you can do the same using update method and it will continue counting without dropping old counters for objects:

>>> from collections import Counter
>>> c = Counter(['hello', 'hello', 1])
>>> print c
Counter({'hello': 2, 1: 1})
>>> c.update(['hello'])
>>> print c
Counter({'hello': 3, 1: 1})

Solution 2:

print len(set(w.lower() for w in open('filename.dat').read().split()))

Reads the entire file into memory, splits it into words using whitespace, converts each word to lower case, creates a (unique) set from the lowercase words, counts them and prints the output

Solution 3:

If you want to get count of each unique word, then use dicts:

words = ['Hello', 'world', 'world']
count = {}
for word in words :
   if word in count :
      count[word] += 1
   else:
      count[word] = 1

And you will get dict

{'Hello': 1, 'world': 2}