python - 使用 numpy 和 matplotlib 进行 13 维矩阵的 KDE(核密度估计)

Question

我不断收到这些错误:

Traceback (most recent call last):   File "D:/Dropbox/Public/Data Processor/src/dP.py", line 69, in <module>
    gkde = stats.gaussian_kde(kdeData)   File "D:\Python27\lib\site-packages\scipy\stats\kde.py", line 86, in
__init__
    self._compute_covariance()   File "D:\Python27\lib\site-packages\scipy\stats\kde.py", line 339, in
_compute_covariance
    self.inv_cov = linalg.inv(self.covariance)   File "D:\Python27\lib\site-packages\scipy\linalg\basic.py", line 327, in inv
    raise LinAlgError("singular matrix") numpy.linalg.linalg.LinAlgError: singular matrix

我不确定这如何适用于我的数据。这是一堵巨大的文本墙，但如果它至少有助于了解代码在这里应用的上下文，那就是 http://pastebin.com/Myx5TpYy 。每个矩阵都有 12 个数据点，说实话，我不确定是否需要所有数据点，但我认为了解这里出了什么问题将帮助我解决问题。 这是我一直试图开始工作的代码

from decimal import *  
import csv  
import numpy as np  
from scipy import stats  
import matplotlib.pylab as plt

matrix = [] 
col1 = [] 
col2 = [] 
col3 = [] 
col4 = [] 
col5 = [] 
col6 = [] 
col7 = [] 
col8 = [] 
col9 = [] 
col10 = [] 
col11 = [] 
col12 = []


for line in open("data.txt", "r"):
    col_1, col_2, col_3, col_4, col_5, col_6, col_7, col_8, col_9, col_10, col_11, col_12 = line.split()

    col_1_val = col_1[:]
    col_2_val = col_2[:]
    col_3_val = col_3[:]
    col_4_val = col_4[:]
    col_5_val = col_5[:]
    col_6_val = col_6[:]
    col_7_val = col_7[:]
    col_8_val = col_8[:]
    col_9_val = col_9[:]
    col_10_val = col_10[:]
    col_11_val = col_11[:]
    col_12_val = col_12[:]

    matrix.append([Decimal(col_1_val), Decimal(col_2_val), Decimal(col_3_val), Decimal(col_4_val), Decimal(col_5_val), Decimal(col_6_val), Decimal(col_7_val), Decimal(col_8_val), Decimal(col_8_val), Decimal(col_9_val), Decimal(col_10_val), Decimal(col_11_val), Decimal(col_12_val)])

kdeData = np.array(matrix).T  
print kdeData
gkde = stats.gaussian_kde(kdeData)
ind = np.linspace(-13,13,512) 
kdepdf = gkde.evaluate(matrix)
plt.figure()
plt.hist(xn, bins=20, normed=1)
plt.plot(ind, stats.norm.pdf(ind), color="r", label='DGP normal')
plt.plot(in, kdepdf, label='kde', color="g") plt.title('Kernel Density Estimation')
plt.legend()
plt.show()

Answer 1

输入矩阵中似乎有两个完全为零的列。这会在 gaussian_kde 计算的内部协方差矩阵中产生大量零，使其成为奇异矩阵并导致例程失败。

如果我像这样重写你的示例:

import numpy as np  
from scipy import stats  
import matplotlib.pylab as plt

valid=[0,1,2,3,4,5,6,7,10,11]
matrix=np.loadtxt('data.txt',skiprows=1,usecols=valid)
kdeData = np.array(matrix).T  
print kdeData
gkde = stats.gaussian_kde(kdeData)
ind = np.linspace(-13,13,512)
kdepdf = gkde.evaluate(kdeData)
plt.figure()
plt.plot(ind, stats.norm.pdf(ind), color="r", label='DGP normal')
plt.plot(ind, kdepdf, label='kde', color="g")
plt.title('Kernel Density Estimation')
plt.legend()
plt.show()

它有效: