python之如何使用请求或其他模块从 url 不变的页面获取数据

我目前正在使用 selenium 转到一个页面:

https://www.nseindia.com/products/content/derivatives/equities/historical_fo.htm 

然后选择相关选项并点击Get Data按钮。

然后检索使用 BeautifulSoup 生成的表。

在这种情况下有没有办法使用请求？如果是这样，是否有人可以向我指出教程？

请您参考如下方法:

当您选择选项时，您或多或少只是为获取数据按钮设置参数以向其后端发出请求。如果您模仿此 curl 中的请求:

curl 'https://www.nseindia.com/products/dynaContent/common/productsSymbolMapping.jsp?instrumentType=FUTIDX&symbol=NIFTYMID50&expiryDate=31-12-2020&optionType=select&strikePrice=&dateRange=day&fromDate=&toDate=&segmentLink=9&symbolCount=' -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36' -H 'Accept: */*' -H 'Referer: https://www.nseindia.com/products/content/derivatives/equities/historical_fo.htm' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'Cache-Control: no-cache' --compressed 

然后你可以在请求中做同样的事情:

import requests 
 
headers = { 
    'Pragma': 'no-cache', 
    'Accept-Encoding': 'gzip, deflate, br', 
    'Accept-Language': 'en-US,en;q=0.9', 
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36', 
    'Accept': '*/*', 
    'Referer': 'https://www.nseindia.com/products/content/derivatives/equities/historical_fo.htm', 
    'X-Requested-With': 'XMLHttpRequest', 
    'Connection': 'keep-alive', 
    'Cache-Control': 'no-cache', 
} 
 
params = ( 
    ('instrumentType', 'FUTIDX'), 
    ('symbol', 'NIFTYMID50'), 
    ('expiryDate', '31-12-2020'), 
    ('optionType', 'select'), 
    ('strikePrice', ''), 
    ('dateRange', 'day'), 
    ('fromDate', ''), 
    ('toDate', ''), 
    ('segmentLink', '9'), 
    ('symbolCount', ''), 
) 
 
response = requests.get('https://www.nseindia.com/products/dynaContent/common/productsSymbolMapping.jsp', headers=headers, params=params) 

学习如何做到这一点的好网站:

https://curl.trillworks.com/