为了从网站下载数据,我复制了一些 Python 代码。这是我的具体网站: https://www.codot.gov/business/bidding/bid-tab-archives/bid-tabs-2017-1
这是我复制的代码:
import requests
from bs4 import BeautifulSoup
def _getUrls_(res):
hrefs = []
soup = BeautifulSoup(res.text, 'lxml')
main_content = soup.find('div',{'id' : 'content-core'})
table = main_content.find("table")
for a in table.findAll('a', href=True):
hrefs.append(a['href'])
return(hrefs)
bidurl = 'https://www.codot.gov/business/bidding/bid-tab-archives/bid-tabs-2017-1'
r = requests.get(bidurl)
hrefs = _getUrls_(r)
def _getPdfs_(hrefs, basedir):
for i in range(len(hrefs)):
print(hrefs[i])
respdf = requests.get(hrefs[i])
pdffile = basedir + "/pdf_dot/" + hrefs[i].split("/")[-1] + ".pdf"
try:
with open(pdffile, 'wb') as p:
p.write(respdf.content)
p.close()
except FileNotFoundError:
print("No PDF produced")
basedir= "/Users/ABC/Desktop"
_getPdfs_(hrefs, basedir)
代码运行成功,但它根本没有下载任何东西,即使显然没有Filenotfounderror。
我尝试了以下两个 URL:
https://www.codot.gov/business/bidding/bid-tab-archives/bid-tabs-2017/aqc-088a-035-20360
https://www.codot.gov/business/bidding/bid-tab-archives/bid-tabs-2017/aqc-r100-258-21125
然而,这两个 URL 都返回 >>> No PDF produced。
问题是代码对其他人有效并成功下载,但我却不行。
请您参考如下方法:
我刚刚测试过您的代码有效。你需要确保 basedir 存在,你想将它添加到你的代码中:
if not os.path.exists(basedir):
os.makedirs(basedir)
