Skip to main content
 首页 » 编程设计

python之在大数据集中搜索

2025年05月04日31JustinYoung

我有一个 user:friends (50,000) 列表和一个事件参与者列表(25,000 个事件和每个事件的参与者列表)。我想找到与用户一起参加事件的前 k 个 friend 。这需要为每个用户完成。

我试过遍历列表,但计算量很大。我也在尝试通过创建加权图来做到这一点。(Python)

如果有任何其他方法,请告诉我。

请您参考如下方法:

Python 的集合对象(字典、集合和 collections.Counter)简化了这项任务:

from collections import Counter 
 
def top_k_friends(friends, events, k=2): 
    '''Given a dictionary users mapped to their set of friends 
    and a dictionary of events mapped to a set of their attendees, 
    find the top k friends with whom the user goes to the event. 
    Do this for each user. 
 
    ''' 
    for user, users_friends in friends.iteritems(): 
        c = Counter() 
        for event, attendees in events.iteritems(): 
            if user in attendees: 
                c.update(users_friends.intersection(attendees)) 
        print user, '-->', c.most_common(k) 
 
if __name__ == '__main__': 
 
    friends = { 
        'robert' : {'mary', 'marty', 'maggie', 'john'}, 
        'paul' : {'marty', 'mary', 'amber', 'susan'} 
    } 
 
    events = { 
        'derby': {'amber', 'mary', 'robert'}, 
        'pageant': {'maggie', 'paul', 'amber', 'marty', 'john'}, 
        'fireworks': {'susan', 'robert', 'marty', 'paul', 'robert'} 
    } 
 
    top_k_friends(friends, events)