python之从 Python 字符串中去除标点符号

我似乎在从 Python 字符串中剥离标点符号时遇到了一些问题。在这里，我得到了一个文本文件(特别是来自 Project Gutenberg 的一本书)和一个停用词列表。我想返回 10 个最常用单词的字典。不幸的是，我总是在我归还的字典中打嗝。

import sys 
import collections 
from string import punctuation 
import operator 
 
#should return a string without punctuation 
def strip_punc(s): 
    return ''.join(c for c in s if c not in punctuation) 
 
def word_cloud(infile, stopwordsfile): 
 
    wordcount = {} 
 
    #Reads the stopwords into a list 
    stopwords = [x.strip() for x in open(stopwordsfile, 'r').readlines()] 
 
 
    #reads data from the text file into a list 
    lines = [] 
    with open(infile) as f: 
        lines = f.readlines() 
        lines = [line.split() for line in lines] 
 
    #does the wordcount 
    for line in lines: 
        for word in line: 
            word = strip_punc(word).lower() 
            if word not in stopwords: 
                if word not in wordcount: 
                    wordcount[word] = 1 
                else: 
                    wordcount[word] += 1 
 
    #sorts the dictionary, grabs 10 most common words 
    output = dict(sorted(wordcount.items(), 
                  key=operator.itemgetter(1), reverse=True)[:10]) 
 
    print(output) 
 
 
if __name__=='__main__': 
 
    try: 
 
        word_cloud(sys.argv[1], sys.argv[2]) 
 
    except Exception as e: 
 
        print('An exception has occured:') 
        print(e) 
        print('Try running as python3 word_cloud.py <input-text> <stopwords>') 

这将打印出来

{'said': 659, 'mr': 606, 'one': 418, '“i': 416, 'lorry': 322, 'upon': 288, 'will': 276, 'defarge': 268, 'man': 264, 'little': 263} 

“我不应该在那里。我不明白为什么它没有在我的辅助函数中消除。

提前致谢。

请您参考如下方法:

字符“不是”。

string.punctuation 仅包含以下 ASCII 字符:

In [1]: import string 
 
In [2]: string.punctuation 
Out[2]: '!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~' 

因此您需要扩充要删除的字符列表。

像下面这样的东西应该可以完成你所需要的:

extended_punc = punctuation + '“' #  and any other characters you need to strip 
 
def strip_punc(s): 
    return ''.join(c for c in s if c not in extended_punc) 

或者，您可以使用包 unidecode以 ASCII 编码您的文本，而不必担心创建您可能需要处理的 unicode 字符列表:

from unidecode import unidecode 
 
def strip_punc(s): 
    s = unidecode(s.decode('utf-8')) 
    return ''.join(c for c in s if c not in punctuation).encode('utf-8')