python之kafka-python 提高 kafka.errors.ConsumerFetchSizeTooSmall

我在 python 2.7 中编写了一个简单的代码，它正在使用来自 apache kafka 主题的消息。代码如下:

from kafka import SimpleConsumer,KafkaClient 
group = "my_group_test" 
client = KafkaClient('localhost:9092') 
cons = SimpleConsumer(client, group, "my_topic") 
messages = cons.get_messages(count=1000,block=False) 

但是引发了这个异常:

Traceback (most recent call last): 
  File "<stdin>", line 1, in <module> 
  File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/simple.py", line 285, in get_messages 
    update_offset=False) 
  File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/simple.py", line 320, in _get_message 
    self._fetch() 
  File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/simple.py", line 425, in _fetch 
    raise ConsumerFetchSizeTooSmall() 
kafka.errors.ConsumerFetchSizeTooSmall  

我如何修改此参数 (ConsumerFetchSize) 才能使此代码正常工作？

请您参考如下方法:

我找到了一个解决方案， 使用参数 max_buffer_size在 SimpleConsumer 中。

工作代码是:

#the size is adherent with my need but is totally arbitrary 
cons = SimpleConsumer(client, group, "my_topic",max_buffer_size=9932768)