python - 计算 pandas 数据框中的匹配组合

Question

我需要为以下问题找到更有效的解决方案:

给定的是一个数据框，每行有 4 个变量。我需要找到 8 个元素的列表，其中包含最大行数中每行的所有变量。

一个有效但非常慢的解决方案是创建第二个数据帧，其中包含所有可能的组合(基本上是没有重复的排列)。然后循环遍历每个组合并将其与初始数据帧进行比较。计算解决方案的数量并将其添加到第二个数据帧中。

import numpy as np
import pandas as pd
from itertools import combinations


df = pd.DataFrame(np.random.randint(0,20,size=(100, 4)), columns=list('ABCD'))
df = 'x' + df.astype(str)
listofvalues = df['A'].tolist()
listofvalues.extend(df['B'].tolist())
listofvalues.extend(df['C'].tolist())
listofvalues.extend(df['D'].tolist())
listofvalues = list(dict.fromkeys(listofvalues))
possiblecombinations = list(combinations(listofvalues, 6))
dfcombi = pd.DataFrame(possiblecombinations, columns = ['M','N','O','P','Q','R'])
dfcombi['List'] = dfcombi.M.map(str) + ',' + dfcombi.N.map(str) + ',' + dfcombi.O.map(str) + ',' + dfcombi.P.map(str) + ',' + dfcombi.Q.map(str) + ',' + dfcombi.R.map(str)
dfcombi['Count'] = ''
for x, row in dfcombi.iterrows():
        comparelist =  row['List'].split(',')
        pointercounter = df.index[(df['A'].isin(comparelist) == True) & (df['B'].isin(comparelist) == True) & (df['C'].isin(comparelist) == True) & (df['D'].isin(comparelist) == True)].tolist()
        row['Count'] = len(pointercounter)

我认为必须有一种方法可以避免 for - 循环并用一些指针替换它，我只是不知道如何实现。

谢谢!

Answer 1

您的代码可以重写为:

# working with integers are much better than strings
enums, codes = df.stack().factorize()

# encodings of df
s = [set(x) for x in enums.reshape(-1,4)]

# possible combinations
from itertools import combinations, product
possiblecombinations = np.array([set(x) for x in combinations(range(len(codes)), 6)])

# count the combination with issubset
ret = [0]*len(possiblecombinations)
for a, (i,b) in product(s, enumerate(possiblecombinations)):
    ret[i] += a.issubset(b)

# the combination with maximum count
max_combination = possiblecombinations[np.argmax(ret)]
# in code {0, 3, 4, 5, 17, 18}

# and in values: 
codes[list(max_combination)]
# Index(['x5', 'x15', 'x12', 'x8', 'x0', 'x6'], dtype='object')

所有这些花费了大约 2 秒，而您的代码花费了大约 1.5 分钟。