python - 计算两个字符串之间的唯一 ID 重叠

Question

我有一个包含两列的数据集。第一列包含唯一的用户 ID，第二列包含与这些 ID 相关的属性。

例如:

------------------------
User ID     Attribute
------------------------
1234        blond
1235        brunette
1236        blond   
1234        tall
1235        tall
1236        short
------------------------

我想知道的是属性之间的相关性。在上面的例子中，我想知道金发女郎的高度是多少倍。我想要的输出是:

------------------------------
Attr 1     Attr 2     Overlap
------------------------------
blond       tall         1
blond       short        1
brunette    tall         1
brunette    short        0
------------------------------

我尝试使用 pandas 来旋转数据并获得输出，但由于我的数据集有数百个属性，我目前的尝试是不可行的。

df = pandas.read_csv('myfile.csv')    

df.pivot_table(index='User ID', columns'Attribute', aggfunc=len, fill_value=0)

我当前的输出:

--------------------------------
Blond   Brunette   Short   Tall
--------------------------------
  0        1         0       1
  1        0         0       1
  1        0         1       0 
--------------------------------

有没有办法得到我想要的输出？提前致谢。

Answer 1

您可以使用 itertools product 找到每个可能的属性对，然后匹配行:

import pandas as pd
from itertools import product

# 1) creating pandas dataframe
df = [  ["1234"    ,    "blond"],
        ["1235"    ,    "brunette"],
        ["1236"    ,    "blond"   ],
        ["1234"    ,    "tall"],
        ["1235"    ,    "tall"],
        ["1236"    ,    "short"]]

df = pd.DataFrame(df)
df.columns = ["id", "attribute"]

#2) creating all the possible attributes binomes
attributs = set(df.attribute)
for attribut1, attribut2 in product(attributs, attributs):
    if attribut1!=attribut2:
        #3) selecting the rows for each attribut
        df1 = df[df.attribute == attribut1]["id"]
        df2 = df[df.attribute == attribut2]["id"]
        #4) finding the ids that are matching both attributs 
        intersection= len(set(df1).intersection(set(df2)))
        if intersection:
            #5) displaying the number of matches
            print attribut1, attribut2, intersection

给予:

tall brunette 1
tall blond 1
brunette tall 1
blond tall 1
blond short 1
short blond 1

编辑

然后很容易改进以获得您想要的输出:

import pandas as pd
from itertools import product

# 1) creating pandas dataframe
df = [  ["1234"    ,    "blond"],
        ["1235"    ,    "brunette"],
        ["1236"    ,    "blond"   ],
        ["1234"    ,    "tall"],
        ["1235"    ,    "tall"],
        ["1236"    ,    "short"]]

df = pd.DataFrame(df)
df.columns = ["id", "attribute"]

wanted_attribute_1 = ["blond", "brunette"]

#2) creating all the possible attributes binomes
attributs = set(df.attribute)
for attribut1, attribut2 in product(attributs, attributs):
    if attribut1 in wanted_attribute_1 and attribut2 not in wanted_attribute_1:
        if attribut1!=attribut2:
            #3) selecting the rows for each attribut
            df1 = df[df.attribute == attribut1]["id"]
            df2 = df[df.attribute == attribut2]["id"]
            #4) finding the ids that are matching both attributs 
            intersection= len(set(df1).intersection(set(df2)))
            #5) displaying the number of matches
            print attribut1, attribut2, intersection

给予:

brunette tall 1
brunette short 0
blond tall 1
blond short 1