python - 使用python仅计算文本文件中的每个单词一次

Question

我有一个小的 python 脚本，我正在为类作业编写。该脚本读取一个文件并打印 10 个最常用和最不常用的单词及其频率。对于此作业，一个单词定义为 2 个或更多字母。我的词频工作得很好，但是作业的第三部分是打印文档中独特 词的总数。独特的单词意味着计算文档中的每个单词，只计算一次。

如何在不对当前脚本进行过多更改的情况下，将文档中的所有单词只统计一次？

附注我使用的是 Python 2.6，所以请不要提及 collections.Counter 的使用

from string import punctuation
from collections import defaultdict
import re

number = 10
words = {}
total_unique = 0
words_only = re.compile(r'^[a-z]{2,}$')
counter = defaultdict(int)


"""Define words as 2+ letters"""
def count_unique(s):
    count = 0
    if word in line:
        if len(word) >= 2:
            count += 1
    return count


"""Open text document, read it, strip it, then filter it"""
txt_file = open('charactermask.txt', 'r')

for line in txt_file:
    for word in line.strip().split():
        word = word.strip(punctuation).lower()
        if words_only.match(word):
               counter[word] += 1


# Most Frequent Words
top_words = sorted(counter.iteritems(),
                    key=lambda(word, count): (-count, word))[:number] 

print "Most Frequent Words: "

for word, frequency in top_words:
    print "%s: %d" % (word, frequency)


# Least Frequent Words:
least_words = sorted(counter.iteritems(),
                    key=lambda (word, count): (count, word))[:number]

print " "
print "Least Frequent Words: "

for word, frequency in least_words:
    print "%s: %d" % (word, frequency)


# Total Unique Words:
print " "
print "Total Number of Unique Words: %s " % total_unique

Answer 1

计算 counter 字典中 key 的数量:

total_unique = len(counter.keys())

或者更简单地说:

total_unique = len(counter)