python - 在Python中删除CSR格式矩阵的列

Question

我有一个 csr 格式的稀疏矩阵 (22000x97482)，我想删除一些列(列号的索引存储在列表中)

Answer 1

如果您有非常多的列，那么生成完整的列索引集可能会变得相当昂贵。一种稍微快一点的替代方法是暂时转换为 COO format :

import numpy as np
from scipy import sparse

def dropcols_fancy(M, idx_to_drop):
    idx_to_drop = np.unique(idx_to_drop)
    keep = ~np.in1d(np.arange(M.shape[1]), idx_to_drop, assume_unique=True)
    return M[:, np.where(keep)[0]]

def dropcols_coo(M, idx_to_drop):
    idx_to_drop = np.unique(idx_to_drop)
    C = M.tocoo()
    keep = ~np.in1d(C.col, idx_to_drop)
    C.data, C.row, C.col = C.data[keep], C.row[keep], C.col[keep]
    C.col -= idx_to_drop.searchsorted(C.col)    # decrement column indices
    C._shape = (C.shape[0], C.shape[1] - len(idx_to_drop))
    return C.tocsr()

检查等价性:

m, n, d = 1000, 2000, 20

M = sparse.rand(m, n, format='csr')
idx_to_drop = np.random.randint(0, n, d)

M_drop1 = dropcols_fancy(M, idx_to_drop)
M_drop2 = dropcols_coo(M, idx_to_drop)

print(np.all(M_drop1.A == M_drop2.A))
# True

基准:

In [1]: m, n = 1000, 1000000

In [2]: %%timeit M = sparse.rand(m, n, format='csr')
   ...: dropcols_fancy(M, idx_to_drop)
   ...: 
1 loops, best of 3: 1.11 s per loop

In [3]: %%timeit M = sparse.rand(m, n, format='csr')
   ...: dropcols_coo(M, idx_to_drop)
   ...: 
1 loops, best of 3: 365 ms per loop