python - 基于多列更新数据框中的列

Question

我有一个名为“age”的列，其中有一些 NaN；推导年龄值的粗略逻辑是使用 2 个关键分类变量(工作、性别)找到年龄的平均值

df = pd.DataFrame([[1,2,1,2,3,4,11,12,13,12,11,1,10], [19,23,np.nan,29,np.nan,32,27,48,39,70,29,51,np.nan], 
            ['a','b','c','d','e','a','b','c','d','e','a','b','c'],['M','F','M','F','M','F','M','F','M','M','F','F','F']]).T
df.columns = ['col1','age','job','gender']

df = df.astype({"col1": int, "age": float})
df['job'] = df.job.astype('category')
df['gender'] = df.gender.astype('category')
df

col1    age job gender
0   1   19.0    a   M
1   2   23.0    b   F
2   1   NaN     c   M
3   2   29.0    d   F
4   3   NaN     e   M
5   4   32.0    a   F
6   11  27.0    b   M
7   12  48.0    c   F
8   13  39.0    d   M
9   12  70.0    e   M
10  11  29.0    a   F
11  1   51.0    b   F
12  10  NaN     c   M

df.groupby(['job','gender']).mean().reset_index()

   job  gender  col1    age
0   a   F   7.500000    30.5
1   a   M   1.000000    19.0
2   b   F   1.500000    37.0
3   b   M   11.000000   27.0
4   c   F   NaN NaN
5   c   M   7.666667    48.0
6   d   F   7.500000    34.0
7   d   M   NaN NaN
8   e   F   NaN NaN
9   e   M   7.500000    70.0

我想将年龄更新为上面的派生值。最佳的做法是什么？我应该将其存储在另一个数据帧中并循环进行更新吗？

结果输出应如下所示:

col1    age job gender
0   1   19.0    a   M
1   2   23.0    b   F
2   1   48.0    c   M
3   2   29.0    d   F
4   3   70.0    e   M
5   4   32.0    a   F
6   11  27.0    b   M
7   12  48.0    c   F
8   13  39.0    d   M
9   12  70.0    e   M
10  11  29.0    a   F
11  1   51.0    b   F
12  10  70.0    c   M

谢谢。

Answer 1

使用Series.fillna与 GroupBy.transform ，但因为示例数据中不是组合 c, M 的数据，因此存在 NaN:

df['age'] = df['age'].fillna(df.groupby(['job','gender'])['age'].transform('mean'))
print (df)
    col1   age job gender
0      1  19.0   a      M
1      2  23.0   b      F
2      1   NaN   c      M
3      2  29.0   d      F
4      3  70.0   e      M
5      4  32.0   a      F
6     11  27.0   b      M
7     12  48.0   c      F
8     13  39.0   d      M
9     12  70.0   e      M
10    11  29.0   a      F
11     1  51.0   b      F
12    10  48.0   c      F

如果需要，还可以通过仅通过 id 查找来替换 NaN，添加另一个 fillna:

avg1 = df.groupby(['job','gender'])['age'].transform('mean')
avg2 = df.groupby('job')['age'].transform('mean')

df['age'] = df['age'].fillna(avg1).fillna(avg2)
print (df)
    col1   age job gender
0      1  19.0   a      M
1      2  23.0   b      F
2      1  48.0   c      M
3      2  29.0   d      F
4      3  70.0   e      M
5      4  32.0   a      F
6     11  27.0   b      M
7     12  48.0   c      F
8     13  39.0   d      M
9     12  70.0   e      M
10    11  29.0   a      F
11     1  51.0   b      F
12    10  48.0   c      F