python - Tensorflow RNN 多对多二进制标签的时间序列

Question

我正在尝试训练一个基本的递归神经网络(多对多)。输入数据具有单一特征(sin 函数)并且标签是二进制的:(1 和 2)

我可以使用 MSE 损失函数对其进行训练，但是当我尝试将其替换为熵时，我遇到了一些问题。

这是我目前拥有的适用于非离散标签的代码:

import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np

n_steps = 20
n_inputs = 1
n_neurons = 100
n_outputs = 1
learning_rate = 0.001

# Create data 
fs = 1000 # sample rate 
f = 2 # the frequency of the signal
x = np.arange(fs) # the points on the x axis for plotting

# training features
dfX = np.array([ np.sin(2*np.pi*f * (i/fs)) for i in x]) 

#labels
dfX2 = np.array([ np.sin(2*np.pi*f * (i/fs)) for i in x]) 
dfX2[dfX2 < 0] = 1
dfX2[dfX2 > 0] = 2

# RNN

X = tf.placeholder(tf.float32, [None, n_steps, n_inputs])
y = tf.placeholder(tf.float32, [None, n_steps, n_outputs])

cell = tf.contrib.rnn.OutputProjectionWrapper( tf.contrib.rnn.BasicRNNCell(num_units=n_neurons, activation=tf.nn.relu),output_size=n_outputs)
outputs, states = tf.nn.dynamic_rnn(cell, X, dtype=tf.float32)


loss = tf.reduce_mean(tf.square(outputs - y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
training_op = optimizer.minimize(loss)
init = tf.global_variables_initializer()

n_iterations = 1000
batch_size = 200
n_epouchs=10

with tf.Session() as sess:
  init.run()
  for epouch in range(n_epouchs):
    for iteration in range(n_iterations-200):
      X_batch= dfX[iteration:iteration+batch_size]
      X_batch= X_batch.reshape(-1,n_steps,n_inputs)
      y_batch= dfX2[(iteration+1):(iteration+batch_size+1)]
      y_batch= y_batch.reshape(-1,n_steps,n_outputs)
      sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
      if iteration % 100 == 0:
    mse = loss.eval(feed_dict={X: X_batch, y: y_batch})
    print(iteration,"--",epouch, "\tMSE:", mse)
  X_new1= dfX[37:37+batch_size]
  X_new1= X_new1.reshape(-1,n_steps,n_inputs)
  y_pred1 = sess.run(outputs, feed_dict={X: X_new1})

我正在尝试将损失函数更改为类似这样的东西(因为我想预测离散标签):

xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=outputs)
loss = tf.reduce_mean(xentropy)

但是没有效果

Answer 1

代码必须进行以下更改才能适用于 cross_entropy_loss:

1:一个热门标签:它们应该取 0 或 1。因此将您的代码更改为:

dfX2[dfX2 < 0] = 0
dfX2[dfX2 > 0] = 1

2:对于2类问题，网络输出层应该是2，所以你的RNN应该是:

cell = tf.contrib.rnn.OutputProjectionWrapper( tf.contrib.rnn.BasicRNNCell(
          num_units=n_neurons, activation=tf.nn.relu),
        output_size=2) #Output size is set to 2.

3:由于输入不是one-hot 编码，您需要将它们转换为one-hot 用于交叉熵损失:

 xentropy =  tf.nn.softmax_cross_entropy_with_logits_v2(
               labels=tf.one_hot(tf.cast(y, tf.int32),2), logits=outputs)

进行上述更改将获得类似的 MSE 分数:

0 -- 0  MSE: 0.60017127
100 -- 0    MSE: 0.13623504
200 -- 0    MSE: 0.07625882
300 -- 0    MSE: 0.006987947