python - 使用 KNN 在 python 中缺失值插补

标签 python scikit-learn knn

我有一个看起来像这样的数据集

1908    January 5.0 -1.4
1908    February    7.3 1.9
1908    March   6.2 0.3
1908    April   NaN   2.1
1908    May NaN   7.7
1908    June    17.7    8.7
1908    July    NaN   11.0
1908    August  17.5    9.7
1908    September   16.3    8.4
1908    October 14.6    8.0
1908    November    9.6 3.4
1908    December    5.8 NaN
1909    January 5.0 0.1
1909    February    5.5 -0.3
1909    March   5.6 -0.3
1909    April   12.2    3.3
1909    May 14.7    4.8
1909    June    15.0    7.5
1909    July    17.3    10.8
1909    August  18.8    10.7  

我想使用 KNN 作为方法替换 NaN。我查找了 sklearnImputer 类,但它仅支持均值、中位数和众数插补。有一个功能请求 here但我认为到目前为止还没有实现。关于如何使用 KNN 替换最后两列中的 NaN 有什么想法吗?

编辑: 由于我需要在另一个环境中运行代码,所以我没有安装包的奢侈。 Sklearn、pandas、numpy 和其他标准包是我唯一可以使用的包。

最佳答案

fancyimpute package使用以下 API 支持此类插补:

from fancyimpute import KNN    
# X is the complete data matrix
# X_incomplete has the same values as X except a subset have been replace with NaN

# Use 3 nearest rows which have a feature to fill in each row's missing features
X_filled_knn = KNN(k=3).complete(X_incomplete)

这里是这个包支持的插补:

•SimpleFill: Replaces missing entries with the mean or median of each column.

•KNN: Nearest neighbor imputations which weights samples using the mean squared difference on features for which two rows both have observed data.

•SoftImpute: Matrix completion by iterative soft thresholding of SVD decompositions. Inspired by the softImpute package for R, which is based on Spectral Regularization Algorithms for Learning Large Incomplete Matrices by Mazumder et. al.

•IterativeSVD: Matrix completion by iterative low-rank SVD decomposition. Should be similar to SVDimpute from Missing value estimation methods for DNA microarrays by Troyanskaya et. al.

•MICE: Reimplementation of Multiple Imputation by Chained Equations.

•MatrixFactorization: Direct factorization of the incomplete matrix into low-rank U and V, with an L1 sparsity penalty on the elements of U and an L2 penalty on the elements of V. Solved by gradient descent.

•NuclearNormMinimization: Simple implementation of Exact Matrix Completion via Convex Optimization by Emmanuel Candes and Benjamin Recht using cvxpy. Too slow for large matrices.

•BiScaler: Iterative estimation of row/column means and standard deviations to get doubly normalized matrix. Not guaranteed to converge but works well in practice. Taken from Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares.

关于python - 使用 KNN 在 python 中缺失值插补,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/45321406/

相关文章:

python - 除了 p-norm 之外,在 scikit-learn 中使用 KNN 中的其他成对距离度量

matlab - 如何在matlab中确定矩阵的k最近邻算法的k值

scikit-learn - Scikit-learn 中 KNN 分类器中的网格搜索参数和交叉验证数据集

python - tensorflow rnn nan 错误

python - 将旧 SIGNAL 和 SLOT 转换为新样式的正确方法?

scikit-learn - 如何克服classlib方法上的joblib的 "TypeError: can' t泡菜实例方法对象?

scikit-learn - 同时预测

python pandas 读取空格分隔的数据

python - 解决超定系统最小二乘法的最快方法

python-2.7 - 如何比较使用 scikit-learn 库 load_svmlight_file 存储的 2 个稀疏矩阵?