python - 具有 Softmax 输出的神经网络

Question

我目前正在学习多层感知编码。对于这个 MLP，我尝试使用逻辑 sigmoidal 作为我的隐藏层，使用 Softmax 作为我的输出，并假设有两个类标签。

import theano
from theano import tensor as T
import numpy as np
import matplotlib.pyplot as plt


alpha = 0.1
#Alpha value
alpha = 2*alpha/2
no_iters = 1 #Trying to get 1 iteration to work first.

#Weight matrix to hidden layer (2 input into 2 neuron)
w_h = np.array([ [1.0, 2.0],
              [-2.0, 0.0] ])

#Bias to hidden layer need ( 2 Hidden Layer neurons)
b_h = np.array([3.0, -1])

#Weight matrix to output layer (2 input into 1 neuron)
w_o = np.array([[1.0],
                [1.0]])

#Bias to output layer (Only 1 bias for one output neuron)
b_o = np.array([-2.0])

# X Input Array (No of data rows, No of inputs)
x = np.array([[1.0, 2.0],
              [-2.0, 3.0]])

#Desired Outputs(2 data row = 2 desired output (Rows))
d = np.array([[0.0],
             [1.0]])

#Assume 2 class labels for the 2 data rows
k = np.array([[1.0, 0.0],
             [0.0, 1.0]])

for iter in range(no_iters):
    #Hidden Layer Functions
    s = np.dot(x,w_h)+ b_h
    z = 1.0/(1 + np.exp(-s))


    #Output Layer Functions (Softmax)
    u = np.dot(z, w_o)+b_o
    u_max = np.max(u, axis=1, keepdims=True)
    p = np.exp(u-u_max)/np.sum(np.exp(u-u_max), axis=1, keepdims=True)
    y = np.argmax(p, axis=1)

    #SoftMax Delta O
    delta_o = k - p

    #Delta for input layer (DZ = differentiation of function)
    dz = z*(1-z)
    delta_h = np.dot(delta_o, np.transpose(w_o))*dz

    #Assign new weight and bias to output layer
    dw = -np.dot(np.transpose(z),delta_o)
    db = -np.sum(delta_o, axis=0)
    w_o = w_o - dw * alpha
    b_o = b_o - db * alpha

    #Assign new weight and bias to hidden layer
    w_h = w_h + alpha*np.dot(np.transpose(x), delta_h)
    b_h = b_h + alpha*np.sum(np.transpose(delta_h), axis=1)

    print(z)
    print(y)

执行代码时，delta_h = np.dot(delta_o, np.transpose(w_o))*dz 的矩阵点积将会出现问题。由于 delta_o 是一个 2x2 矩阵，而 transpose(w_o) 是一个 1x2 矩阵。

我是否使用了错误的公式来解决这个问题？

Answer 1

你不能将两个不同大小的张量相乘。您可以做的是获得所获得的误差向量的平均值，并对权重进行逐元素修改。这不会影响性能，并且会解决我希望的错误。