python - 删除 pandas 中的 nan 后出现 keyerror

Question

我正在使用 pd.read_csv 读取文件，并删除所有 -1 值。这是代码

import pandas as pd
import numpy as np

columns = ['A', 'B', 'C', 'D']
catalog = pd.read_csv('data.txt', sep='\s+', names=columns, skiprows=1)

a = cataog['A']
b = cataog['B']
c = cataog['C']
d = cataog['D']

print len(b) # answer is 700

# remove rows that are -1 in column b
idx = np.where(b != -1)[0]
a = a[idx]
b = b[idx]
c = c[idx]
d = d[idx]

print len(b) # answer is 612

因此，我假设我已成功删除 b 列中值为 -1 的所有行。

为了测试这一点，我正在执行以下简单的方法:

for i in range(len(b)):
    print i, a[i], b[i]

它会打印出这些值，直到到达应该被过滤掉的行。但现在它给出了一个KeyError。

Answer 1

您可以按 boolean indexing 进行过滤:

catalog = catalog[catalog['B'] != -1]

a = cataog['A']
b = cataog['B']
c = cataog['C']
d = cataog['D']

预计您会收到KeyError，因为索引值不匹配，因为过滤。

一种可能的解决方案是将Series转换为list:

for i in range(len(b)):
    print i, list(a)[i], list(b)[i]

示例:

catalog = pd.DataFrame({'A':list('abcdef'),
                   'B':[-1,5,4,5,-1,4],
                   'C':[7,8,9,4,2,3],
                   'D':[1,3,5,7,1,0]})

print (catalog)
   A  B  C  D
0  a -1  7  1
1  b  5  8  3
2  c  4  9  5
3  d  5  4  7
4  e -1  2  1

#filtered DataFrame have no index 0, 4
catalog = catalog[catalog['B'] != -1]
print (catalog)
   A  B  C  D
1  b  5  8  3
2  c  4  9  5
3  d  5  4  7
5  f  4  3  0

---

a = catalog['A']
b = catalog['B']
c = catalog['C']
d = catalog['D']

print (b)
1    5
2    4
3    5
5    4
Name: B, dtype: int64

#a[i] in first loop want match index value 0 (a[0]) what does not exist, so KeyError,
#same problem for b[0]
for i in range(len(b)):
    print (i, a[i], b[i])

KeyError: 0

---

#convert Series to list, so list(a)[0] return first value of list - there is no Series index
for i in range(len(b)):
    print (i, list(a)[i], list(b)[i])
0 b 5
1 c 4
2 d 5
3 f 4

另一个解决方案应该是通过 reset_index 创建默认索引 0,1,...与drop=True:

catalog = catalog[catalog['B'] != -1].reset_index(drop=True)    
print (catalog)
   A  B  C  D
0  b  5  8  3
1  c  4  9  5
2  d  5  4  7
3  f  4  3  0

a = catalog['A']
b = catalog['B']
c = catalog['C']
d = catalog['D']

#default index values match a[0] and a[b]
for i in range(len(b)):
    print (i, a[i], b[i])

0 b 5
1 c 4
2 d 5
3 f 4