我正在尝试从如下所示的 block 引用中获取文本:
<blockquote class="postcontent restore ">
01 Oyasumi
<br></br>
02 DanSin'
<br></br>
03 w.t.s.
<br></br>
04 Lovism
<br></br>
05 NoName
<br></br>
06 Gakkou
<br></br>
07 Happy☆Day
<br></br>
08 Endless End.
</blockquote>
我想做的是在 python 2.7 中(它无法解码 ☆ 字符,这就是我尝试使用编码的原因):
soup = BeautifulSoup(r.text, "html5lib") #r is from a requests get request
content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings
for line in content:
print(line.encode("utf-8"))
这就是我得到的:
b'01 Oyasumi'
b"02 DanSin'"
b'03 w.t.s.'
b'04 Lovism'
b'05 NoName'
b'06 Gakkou'
b'07 Happy\xe2\x98\x86Day'
b'08 Endless End.'
我做错了什么?
请您参考如下方法:
问题是,如果未使用名为 Unicode, Dammit 的子库指定 from_encoding,Beautiful Soup 会将原始编码转换为 Unicode。 .更多信息在 Encodings文档中的部分。
>>> from bs4 import BeautifulSoup
>>> doc = '''<blockquote class="postcontent restore ">
... 01 Oyasumi
... <br></br>
... 02 DanSin'
... <br></br>
... 03 w.t.s.
... <br></br>
... 04 Lovism
... <br></br>
... 05 NoName
... <br></br>
... 06 Gakkou
... <br></br>
... 07 Happy☆Day
... <br></br>
... 08 Endless End.
... </blockquote>'''
>>> soup = BeautifulSoup(doc, 'html5lib')
>>> soup.original_encoding
u'windows-1252'
>>> content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings
>>> for line in content:
... print(line)
...
01 Oyasumi
02 DanSin'
03 w.t.s.
04 Lovism
05 NoName
06 Gakkou
07 Happy☆Day
08 Endless End.
要解决这个问题,您有两个选择:
通过传递正确的
from_encoding参数或排除 Dammit 猜测的错误编码。一个问题是并非所有解析器都支持exclude_encodings参数。例如html5lib树构建器不支持exclude_encoding>>> soup = BeautifulSoup(doc, 'html5lib', from_encoding='utf-8') >>> content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings >>> for line in content: ... print(line) ... 01 Oyasumi 02 DanSin' 03 w.t.s. 04 Lovism 05 NoName 06 Gakkou 07 Happy☆Day 08 Endless End. >>>使用
lxml解析器>>> soup = BS(doc, 'lxml') >>> soup.original_encoding 'utf-8' >>> content = soup.find("blockquote", {"class": "postcontent restore "}).stripped_strings >>> for line in content: ... print(line) ... 01 Oyasumi 02 DanSin' 03 w.t.s. 04 Lovism 05 NoName 06 Gakkou 07 Happy☆Day 08 Endless End.
