python之python中spacy的多线程训练

我试图找到一种在 spacy 上使用多线程来训练 NER 模型的方法。看起来在我的工作计算机(Ubuntu 16.04 Python3.5)上默认使用多线程，但在我的服务器上不使用。

知道为什么吗？

关于服务器上 spaCy 和 env 的信息

Platform           Linux-3.14.32-xxxx-grs-ipv6-64-x86_64-with-Debian-8 
Python version     3.4.2           
Location           /home/nlp/.env/lib/python3.4/site-packages/spacy 
Models             fr, fr_core_news_md 
spaCy version      2.0.5 

尝试过程:

安装

python3 -m venv .env 
source .env/bin/activate 
pip install -U spacy 
pip3 install pip --upgrade 
python -m spacy download fr 
python -m spacy validate 

脚本python3

import spacy 
import random 
 
ITERATION_NBR = 100 
DROP_RATE = 0.5 
 
TRAIN_DATA = [ 
    ('Who is Shaka Khan?', { 
        'entities': [(7, 17, 'PERSON')] 
    }), 
    ('I like London and Berlin.', { 
        'entities': [(7, 13, 'LOC'), (18, 24, 'LOC')] 
    }) 
] 
 
def main(): 
    try: 
        nlp = spacy.load("fr") 
    except: 
        nlp = spacy.load("fr_core_news_sm") 
    if 'ner' not in nlp.pipe_names: 
        ner = nlp.create_pipe('ner') 
        nlp.add_pipe(ner, last=True) 
    else: 
        ner = nlp.get_pipe('ner') 
    for _, annotations in TRAIN_DATA: 
        for ent in annotations.get('entities'): 
            ner.add_label(ent[2]) 
    other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'ner'] 
    with nlp.disable_pipes(*other_pipes): 
        optimizer = nlp.begin_training() 
        for itn in range(ITERATION_NBR): 
            random.shuffle(TRAIN_DATA) 
            losses = {} 
            for text, annotations in TRAIN_DATA: 
                nlp.update( 
                    [text], 
                    [annotations], 
                    drop=DROP_RATE, 
                    sgd=optimizer, 
                    losses=losses) 

执行

python3 <scriptName>.py 

请您参考如下方法:

需要python >= 3.5 多线程在训练spacy时默认工作