python - Pandas dataframe 使用 pd.concat 将字符串替换为 NaN

Question

我有一个由字符串组成的 pandas 数据框，即“P1”、“P2”、“P3”、...、null。

当我尝试将此数据框与另一个数据框连接时，所有字符串都被替换为“NaN”。

请参阅下面的代码:

descriptions = pd.read_json('https://raw.githubusercontent.com/ansymo/msr2013-bug_dataset/master/data/v02/eclipse/short_desc.json')
descriptions = descriptions.reset_index(drop=1)
descriptions['desc'] = descriptions.short_desc.apply(operator.itemgetter(0)).apply(operator.itemgetter('what'))
f1=pd.DataFrame(descriptions['desc'])

bugPrior = pd.read_json('https://raw.githubusercontent.com/ansymo/msr2013-bug_dataset/master/data/v02/eclipse/priority.json')
bugPrior = bugPrior.reset_index(drop=1)
bugPrior['priority'] = bugPrior.priority.apply(operator.itemgetter(0)).apply(operator.itemgetter('what'))
f2=pd.DataFrame(bugPrior['priority'])

df = pd.concat([f1,f2])
print(df.head())

输出如下:

              desc                                     priority
0    Usability issue with external editors (1GE6IRL)      NaN
1             API - VCM event notification (1G8G6RR)      NaN
2  Would like a way to take a write lock on a tea...      NaN
3  getter/setter code generation drops "F" in ".....      NaN
4  Create Help Index Fails with seemingly incorre...      NaN

关于如何阻止这种情况发生有什么想法吗？

最终，我的目标是将所有内容都放在一个数据框中，这样我就可以删除所有具有“空”值的行。它还有助于稍后编写代码。

谢谢。

Answer 1

假设您想要水平连接这些列，您需要将 axis=1 传递给 pd.concat，因为默认情况下，连接是垂直的。

df = pd.concat([f1,f2], axis=1)

---

要删除那些 NaN 行，您应该能够使用 df.dropna。之后调用 df.reset_index。

df = pd.concat([f1, f2], 1)
df = df.dropna().reset_index(drop=True)
print(df.head(10))
                                                desc priority
0  Create Help Index Fails with seemingly incorre...       P3
1  Internal compiler error when compiling switch ...       P3
2  Default text sizes in org.eclipse.jface.resour...       P3
3  [Presentations] [ViewMgmt] Holding mouse down ...       P3
4  Parsing of function declarations in stdio.h is...       P2
5  CCE in RenameResourceAction while renaming ele...       P3
6  Option to prevent cursor from moving off end o...       P3
7        Tasks section in the user doc is very stale       P3
8  Importing existing project with different case...       P3
9  Workspace in use --> choose new workspace but ...       P3

打印出 df.priority.unique()，我们看到有 5 个独特的优先级:

print(df.priority.unique())
array(['P3', 'P2', 'P4', 'P1', 'P5'], dtype=object)