python - 如何从未堆叠的 Pandas 数据框中选择特定列？

Question

我正在将文本文件中的一些数据读入 Pandas。读入的txt的简化版本是:

idx_level1|idx_level2|idx_level3|idx_level4|START_NODE|END_NODE|OtherData...
353386066294006|1142|2018-09-20T07:57:26Z|1|18260004567689|18260005575180|...
353386066294006|1142|2018-09-20T07:57:26Z|2|18260004567689|18260004240718|...
353386066294006|1142|2018-09-20T07:57:26Z|3|18260005359901|18260004567689|...
353386066294006|1142|2018-09-20T07:57:31Z|1|18260004567689|18260005575180|...
353386066294006|1142|2018-09-20T07:57:31Z|2|18260004567689|18260004240718|...
353386066294006|1142|2018-09-20T07:57:31Z|3|18260005359901|18260004567689|...
353386066294006|1142|2018-09-20T07:57:36Z|1|18260004567689|18260005575180|...
353386066294006|1142|2018-09-20T07:57:36Z|2|18260004567689|18260004240718|...
353386066294006|1142|2018-09-20T07:57:36Z|3|18260005359901|18260004567689|...
353386066736543|22|2018-04-17T07:08:23Z||||...
353386066736543|22|2018-04-17T07:08:24Z||||...
353386066736543|22|2018-04-17T07:08:25Z||||...
353386066736543|22|2018-04-17T07:08:26Z||||...
353386066736543|403|2018-07-02T16:55:07Z|1|18260004580350|18260005235340|...
... 

我用来读入的代码如下:

mydata = pd.read_csv('/myloc/my_simple_data.txt', sep='|', 
 dtype={'idx_level1': 'int',
        'idx_level2': 'int',
        'idx_level3': 'str',
        'idx_level4': 'float',
        'START_NODE': 'str',
        'END_NODE': 'str',
        'OtherData...': 'str'},
parse_dates = ['idx_level3'],
index_col=['idx_level1','idx_level2','idx_level3','idx_level4'])

在某些时候我会解压这些数据:

temp_df = mydata.loc[(slice(None)),['START_NODE', 'END_NODE', 'OtherData...']].unstack()

我的数据现在看起来像

                                                START_NODE                                 ...  OtherData...
idx_level4                                                1.0             2.0             3.0  ...      25.0 26.0 27.0 28.0 29.0 30.0 31.0 32.0
idx_level1      idx_level2 idx_level3                                                          ...
353386066294006 1033       2018-09-03 14:52:27  18260004553260  18260005729143  18260004553259 ...       NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN
                           2018-09-03 14:52:32  18260004553260  18260005729143  18260004553259 ...       NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN
                           2018-09-03 14:52:37  18260004553260  18260005729143  18260004553259 ...       NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN
                           2018-09-03 14:52:42  18260004553260  18260005729143  18260004553259 ...       NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN
                           2018-09-03 14:52:47  18260004553260  18260005729143  18260004553259 ...       NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN
                           2018-09-03 14:52:52  18260004553260  18260005729143  18260004553259 ...       NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN
                           2018-09-03 14:52:57  18260004553260  18260005729143  18260004553259 ...       NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN
...

现在有没有一种方法可以让我选择特定列来应用某些操作 - 假设我想在 idx_level4 = 1.0 的“START_NODE”列上shift(1)？

Answer 1

您可以按元组选择:

s = df[('START_NODE', 4.0)].shift(1)

编辑:

对于多个 Multiindex 列，请使用 boolean indexing使用 loc 通过掩码选择列:

mux = pd.MultiIndex.from_product([['START_NODE','END_NODE'], range(1, 5)])
df = pd.DataFrame([[1] * 8], columns=mux)
print (df)
  START_NODE          END_NODE         
           1  2  3  4        1  2  3  4
0          1  1  1  1        1  1  1  1

v = [('START_NODE', 4.0), ('END_NODE', 3.0)]
df1 = df.loc[:,  df.columns.isin(v)]
print (df1)
  START_NODE END_NODE
           4        3
0          1        1