python之确定有序子列表是否在大型列表列表中的最快方法

假设我有一个 my_huge_list_of_lists，其中包含 2,000,000 个列表，每个列表的长度约为 50。

我想通过丢弃序列中不包含两个元素的子列表来缩短 2,000,000 个 my_huge_list_of_lists。

到目前为止我有:

# https://stackoverflow.com/questions/3313590/check-for-presence-of-a-sliced-list-in-python 
def check_if_list_is_sublist(lst, sublst): 
    #checks if a list appears in order in another larger list. 
    n = len(sublst) 
    return any((sublst == lst[i:i + n]) for i in xrange(len(lst) - n + 1)) 
 
my_huge_list_of_lists = [x for x in my_huge_list_of_lists 
                            if not check_if_list_is_sublist(x, [a,b])] 
my_huge_list_of_lists = [x for x in my_huge_list_of_lists 
                            if not check_if_list_is_sublist(x, [b,a])] 

搜索词 [a,b] 或 [b,a] 的连续性很重要，因此我不能使用 set.issubset()。

我觉得这很慢。我想加快速度。我考虑了一些选项，例如使用“提前退出”和声明:

my_huge_list_of_lists = [x for x in my_huge_list_of_lists 
                            if (a in x and not check_if_list_is_sublist(x, [a,b]))] 

在 for 循环中使用 or 语句的次数更少:

my_huge_list_of_lists = [x for x in my_huge_list_of_lists 
                            if not (check_if_list_is_sublist(x, [a,b]) 
                                    or check_if_list_is_sublist(x, [b,a]))] 

并致力于加速功能(WIP)

# https://stackoverflow.com/questions/48232080/the-fastest-way-to-check-if-the-sub-list-exists-on-the-large-list 
def check_if_list_is_sublist(lst, sublst): 
        checks if a list appears in order in another larger list. 
        set_of_sublists = {tuple(sublst) for sublist in lst} 

并在 Stack Overflow 上进行了一些搜索；但想不出办法，因为 check_if_list_is_sublist() 被调用的次数是 len(my_huge_list) * 2。

编辑:按要求添加一些用户数据

from random import randint 
from string import ascii_lowercase 
my_huge_list_of_lists = [[ascii_lowercase[randint(0, 25)] for x in range(50)] for y in range(2000000)] 
my_neighbor_search_fwd = [i,c] 
my_neighbor_search_rev = my_neighbor_search_fwd.reverse() 

请您参考如下方法:

将 n 大小的子序列中的项目解包为 n 个变量。然后编写列表推导来过滤列表，检查子列表中的 a、b 或 b、a。例如

a, b = sublst 
 
def checklst(lst, a, b): 
    l = len(lst) 
    start = 0 
    while True: 
        try: 
            a_index = lst.index(a, start) 
        except ValueError: 
            return False 
        try: 
            return a_index > -1 and lst[a_index+1] == b 
        except IndexError: 
            try: 
                return a_index > -1 and lst[a_index-1] == b 
            except IndexError: 
                start = a_index + 1 
                if start == l: 
                    return False 
                continue # keep looking at the next a 
 
%timeit found = [l for l in lst if checklst(l, a, b)] 
1.88 s ± 31.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) 
 
%timeit found = [x for x in lst if (a in x and not check_if_list_is_sublist(x, [a,b]))] 
22.1 s ± 1.67 s per loop (mean ± std. dev. of 7 runs, 1 loop each)