python之与 Python 并行解析和保存主机名

我正在尝试解析主机名列表。问题是当我访问一个不存在的域时，它会减慢整个过程。代码是一个简单的 for 循环:

for domain in domains: 
    try: 
        if socket.gethostbyname(domain.split('@')[1]): 
            file1.write(domain) 
        else: 
            file2.write(domain) 
    except socket.gaierror: 
        pass 

我想知道是否有一种简单的方法可以并行化 for 循环中的内容。

请您参考如下方法:

您可以使用 Gevent 中的示例之一 - dns_mass_resolve.py .还可以为所有查询设置超时。

from __future__ import with_statement 
import sys 
import gevent 
from gevent import socket 
from gevent.pool import Pool 
 
N = 1000 
# limit ourselves to max 10 simultaneous outstanding requests 
pool = Pool(10) 
finished = 0 
 
 
def job(url): 
    global finished 
    try: 
        try: 
            ip = socket.gethostbyname(url) 
            print ('%s = %s' % (url, ip)) 
        except socket.gaierror: 
            ex = sys.exc_info()[1] 
            print ('%s failed with %s' % (url, ex)) 
    finally: 
        finished += 1 
 
with gevent.Timeout(2, False): 
    for x in xrange(10, 10 + N): 
        pool.spawn(job, '%s.com' % x) 
    pool.join() 
 
print ('finished within 2 seconds: %s/%s' % (finished, N))