Skip to main content
 首页 » 编程设计

python之读取 CSV 并将数据上传到 Elasticsearch

2025年05月04日24cmt

我正在逐行迭代一个 csv 文件,我想将它插入到 es 中。我是 python 和 Elasticsearch 的新手。如何转换一个 csv 行并将其一一插入到 es 中

import csv 
import json 
 
from elasticsearch import Elasticsearch 
 
es = Elasticsearch( 
  [{'host': 'localhost', 'port': 9200}]) 
 print(es) 
 
 
def csv_reader(file_obj, delimiter=','): 
   reader = csv.reader(file_obj) 
   i = 1 
   results = [] 
   for row in reader: 
    print(row) 
    es.index(index='product', doc_type='prod', id=i,  
   body=json.dump([row for row in reader], file_obj)) 
    i = i + 1 
    results.append(row) 
    print(row) 
 
 
 if __name__ == "__main__": 
  with open("/home/Documents/csv/acsv.csv") as f_obj: 
    csv_reader(f_obj) 

但是我收到了这个错误:

Traceback (most recent call last):

File "/home/PycharmProjects/CsvReaderForSyncEs/csvReader.py", line 25, in csv_reader(f_obj)

File "/home/PycharmProjects/CsvReaderForSyncEs/csvReader.py", line 17, in csv_reader

es.index(index='product', doc_type='prod', id=i, body=json.dump([row for row in reader], file_obj))

File "/usr/lib/python2.7/json/init.py", line 190, in dump fp.write(chunk)

IOError: File not open for writing

请您参考如下方法:

尝试批量 API。

import csv 
from elasticsearch import helpers, Elasticsearch 
 
def csv_reader(file_name): 
    es = Elasticsearch([{'host': 'localhost', 'port': 9200}]) 
    with open(file_name, 'r') as outfile: 
        reader = csv.DictReader(outfile) 
        helpers.bulk(es, reader, index="index_name", doc_type="type") 

有关批量 API 的更多信息 https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html