python - 从 NLTK 中删除停用词

标签 python nltk stop-words

我正在尝试读取一个文本文件 (foo1.txt),删除所有 nltk 定义的停用词并写入另一个文件 (foo2.txt)。代码如下: 需要导入:从nltk.corpus导入停用词

def stop_words_removal(): 
    with open("foo1.txt") as f:
            reading_file_line = f.readlines() #entire content, return  list 
            #print reading_file_line #list
            reading_file_info = [item.rstrip('\n') for item in reading_file_line]
            #print reading_file_info #List and strip \n
            #print ' '.join(reading_file_info)
            '''-----------------------------------------'''
            #Filtering & converting to lower letter
            for i in reading_file_info:
                words_filtered = [e.lower() for e in i.split() if len(e) >= 4]                
                print words_filtered

            '''-----------------------------------------'''
            '''removing the strop words from the file'''
            word_list = words_filtered[:] 
            #print word_list
            for word in words_filtered:
                        if word in nltk.corpus.stopwords.words('english'): 
                            print word
                            print word_list.remove(word)

            '''-----------------------------------------'''
            '''write the output in a file'''
            z = ' '.join(words_filtered)
            out_file = open("foo2.txt", "w")
            out_file.write(z)
            out_file.close()  

问题是代码的第二部分“从文件中删除斜杠词”不起作用。任何建议将不胜感激。谢谢。

Example Input File: 
'I a Love this car there', 'positive',
'This a view is amazing there', 'positive',
'He is my best friend there', 'negative'

Example Output:
['love', "car',", "'positive',"]
['view', "amazing',", "'positive',"]
['best', "friend',", "'negative'"]

我按照link中的建议进行了尝试,但它们都不起作用

最佳答案

这就是我在你的函数中要做的:

with open('input.txt','r') as inFile, open('output.txt','w') as outFile:
    for line in inFile:
        print(''.join([word for word in line.lower().translate(None, string.punctuation).split() 
              if len(word) >=4 and word not in stopwords.words('english')]), file=outFile)

不要忘记添加:

from __future__ import print_function                   

如果您使用的是 Python 2.x。

关于python - 从 NLTK 中删除停用词,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/16613751/

相关文章:

python - 使用python将日志文件的内容打印到curses屏幕

python - 如何训练大型数据集进行分类

java - java中的停用词和词干分析器

regex - 我可以在Elasticsearch中的停止分析器的停用词中指定regexp吗?

python - 属性错误 : 'int' object has no attribute 'id' - Odoo v9 community

python - 如何根据行组将行重新排列为 Pandas 中的列

python - 当包含有意义的空格时,如何编写与 re.VERBOSE 一起使用的模式?

python - 如何从句子中提取主题?

Python Pandas NLTK 从 Dataframe 'join() argument' 错误的文本字段中提取常用短语 (ngrams)

python - 如何向 NLTK 中的停用词添加更多语言?