Skip to main content
 首页 » 编程设计

python之Scrapyd 错误 : exceptions. AttributeError: 'dict' object has no attribute 'fields'

2025年05月04日14落叶

我最近向 scrapyd 发布了一个工作抓取。我在运行抓取时收到以下错误消息。

我查看了这个已关闭的问题:https://github.com/scrapy/scrapy/issues/86并根据文档实现了建议的修复:http://scrapyd.readthedocs.org/en/latest/config.html

我的配置文件在这里:/etc/scrapyd/conf.d/000-default

我似乎无法通过将 items_dir 设置为 none 来禁用 Feedexporter 扩展。在 scrapyd 中是否有禁用 feedexporter 的新方法?

配置文件:

[scrapyd] 
http_port  = 6800 
debug      = off 
#max_proc  = 1 
eggs_dir   = /var/lib/scrapyd/eggs 
dbs_dir    = /var/lib/scrapyd/dbs 
items_dir  = 
logs_dir   = /var/log/scrapyd 

错误信息:

2014-10-12 06:29:15-0500 [jsc] ERROR: Error caught on signal handler: <bound method ?.item_scraped of <scrapy.contrib.feedexport.FeedExporter object at 0x7fc879ff82d0>> 
Traceback (most recent call last): 
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks 
    current.result = callback(current.result, *args, **kw) 
  File "/usr/lib/pymodules/python2.7/scrapy/core/scraper.py", line 215, in _itemproc_finished 
    item=output, response=response, spider=spider) 
  File "/usr/lib/pymodules/python2.7/scrapy/signalmanager.py", line 23, in send_catch_log_deferred 
    return signal.send_catch_log_deferred(*a, **kw) 
  File "/usr/lib/pymodules/python2.7/scrapy/utils/signal.py", line 53, in send_catch_log_deferred 
    *arguments, **named) 
--- <exception caught here> --- 
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 139, in maybeDeferred 
    result = f(*args, **kw) 
  File "/usr/lib/pymodules/python2.7/scrapy/xlib/pydispatch/robustapply.py", line 54, in robustApply 
    return receiver(*arguments, **named) 
  File "/usr/lib/pymodules/python2.7/scrapy/contrib/feedexport.py", line 190, in item_scraped 
    slot.exporter.export_item(item) 
  File "/usr/lib/pymodules/python2.7/scrapy/contrib/exporter/__init__.py", line 87, in export_item 
    itemdict = dict(self._get_serialized_fields(item)) 
  File "/usr/lib/pymodules/python2.7/scrapy/contrib/exporter/__init__.py", line 71, in _get_serialized_fields 
    field = item.fields[field_name] 
exceptions.AttributeError: 'dict' object has no attribute 'fields' 

请您参考如下方法:

我可以通过将以下行添加到 settings.py 来禁用 scrapyd 中的 FeedExporter

EXTENSIONS = {'scrapy.contrib.feedexport.FeedExporter': None} 

禁用 FeedExporter 解决了问题。