python - 从同一项目 pandas 的不同数据框中获取值

Question

我有 3 个 df。

df1

id   val1   val2      
1    1.1     2.2
2    3.3     6.6

df2

id   val1   val2      
1    5.1     2.2
3    3.3     6.6
4    2.1     5.2

df3

id   val1   val2      
1    9.1     3.2
4    8.1     3.2
5    1.3     4.5

您可以注意到，对于相同的 id = 1,3,4，不同数据帧中的 val1 和 val2 值不同。

我正在寻找的是，这种多次出现的最终 df，其中每个 df 的值作为一列:

id   df1         df2         df3
 1  [1.1,2.2]   [5.1,2.2]   [9.1,3.2]
 4  [2.1,5.2]   [8.1,3.2]    NA

正在考虑:

df.groupby(['id']).apply(list)

这在 Pandas 中可能吗？

Answer 1

用途:

#list of all DataFrames
dfs = [df1, df2, df3]

#loop for set index and Series by constructor
L = []
for x in dfs:
    x = x.set_index('id')
    L.append(pd.Series(x.values.tolist(), index=x.index))

#all together   
df = pd.concat(L, axis=1, keys=('df1','df2','df3'))
print (df)
           df1         df2         df3
id                                    
1   [1.1, 2.2]  [5.1, 2.2]  [9.1, 3.2]
2   [3.3, 6.6]         NaN         NaN
3          NaN  [3.3, 6.6]         NaN
4          NaN  [2.1, 5.2]  [8.1, 3.2]
5          NaN         NaN  [1.3, 4.5]

#filter rows
df = df[df.count(axis=1) > 1]
print (df)
           df1         df2         df3
id                                    
1   [1.1, 2.2]  [5.1, 2.2]  [9.1, 3.2]
4          NaN  [2.1, 5.2]  [8.1, 3.2]

<小时/>

谢谢Arthur Gouveia供创意使用dropna :

df = df.dropna(thresh=2)
print (df)
           df1         df2         df3
id                                    
1   [1.1, 2.2]  [5.1, 2.2]  [9.1, 3.2]
4          NaN  [2.1, 5.2]  [8.1, 3.2]

编辑:

如果id列中的值不是唯一的，解决方案就简化了:

print (df3)
   id  val1  val2
0   1   9.1   3.2
1   4   8.1   3.2
2   1   1.3   4.5 <-change value to 1

dfs = [df1, df2, df3]
L = [x.groupby('id')['val1','val2'].apply(lambda x: x.values.ravel().tolist()) for x in dfs]
df = pd.concat(L, axis=1, keys=('df1','df2','df3'))
df = df[df.count(axis=1) > 1]
print (df)
           df1         df2                   df3
id                                              
1   [1.1, 2.2]  [5.1, 2.2]  [9.1, 3.2, 1.3, 4.5]
4          NaN  [2.1, 5.2]            [8.1, 3.2]