Skip to main content
 首页 » 编程设计

python之BeautifulSoup4 stripped_strings 给我字节对象

2025年05月04日27jyk

我正在尝试从如下所示的 block 引用中获取文本:

<blockquote class="postcontent restore "> 
    01 Oyasumi 
    <br></br> 
    02 DanSin' 
    <br></br> 
    03 w.t.s. 
    <br></br> 
    04 Lovism 
    <br></br> 
    05 NoName 
    <br></br> 
    06 Gakkou 
    <br></br> 
    07 Happy☆Day 
    <br></br> 
    08 Endless End. 
</blockquote> 

我想做的是在 python 2.7 中(它无法解码 ☆ 字符,这就是我尝试使用编码的原因):

soup = BeautifulSoup(r.text, "html5lib") #r is from a requests get request 
content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings 
for line in content: 
    print(line.encode("utf-8")) 

这就是我得到的:

b'01 Oyasumi' 
b"02 DanSin'" 
b'03 w.t.s.' 
b'04 Lovism' 
b'05 NoName' 
b'06 Gakkou' 
b'07 Happy\xe2\x98\x86Day' 
b'08 Endless End.' 

我做错了什么?

请您参考如下方法:

问题是,如果未使用名为 Unicode, Dammit 的子库指定 from_encoding,Be​​autiful Soup 会将原始编码转换为 Unicode。 .更多信息在 Encodings文档中的部分。

>>> from bs4 import BeautifulSoup 
>>> doc = '''<blockquote class="postcontent restore "> 
...     01 Oyasumi 
...     <br></br> 
...     02 DanSin' 
...     <br></br> 
...     03 w.t.s. 
...     <br></br> 
...     04 Lovism 
...     <br></br> 
...     05 NoName 
...     <br></br> 
...     06 Gakkou 
...     <br></br> 
...     07 Happy☆Day 
...     <br></br> 
...     08 Endless End. 
... </blockquote>''' 
>>> soup = BeautifulSoup(doc, 'html5lib') 
>>> soup.original_encoding  
u'windows-1252' 
>>> content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings 
>>> for line in content: 
...     print(line) 
...  
01 Oyasumi 
02 DanSin' 
03 w.t.s. 
04 Lovism 
05 NoName 
06 Gakkou 
07 Happy☆Day 
08 Endless End. 

要解决这个问题,您有两个选择:

  1. 通过传递正确的 from_encoding 参数或排除 Dammit 猜测的错误编码。一个问题是并非所有解析器都支持 exclude_encodings 参数。例如 html5lib 树构建器不支持 exclude_encoding

    >>> soup = BeautifulSoup(doc, 'html5lib', from_encoding='utf-8') 
    >>> content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings 
    >>> for line in content: 
    ...     print(line) 
    ...  
    01 Oyasumi 
    02 DanSin' 
    03 w.t.s. 
    04 Lovism 
    05 NoName 
    06 Gakkou 
    07 Happy☆Day 
    08 Endless End. 
    >>>  
    
  2. 使用 lxml解析器

    >>> soup = BS(doc, 'lxml') 
    >>> soup.original_encoding 
    'utf-8' 
    >>> content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings 
    >>> for line in content: 
    ...     print(line) 
    ...  
    01 Oyasumi 
    02 DanSin' 
    03 w.t.s. 
    04 Lovism 
    05 NoName 
    06 Gakkou 
    07 Happy☆Day 
    08 Endless End.