python - 用于对 2d 点进行二元分类的深度神经网络模型的训练和测试准确率达到 50%

Question

我有一个带有 2 个隐藏层的深度神经网络。我正在尝试使用神经网络将 2d 数据点分类为 0 或 1。我生成的数据如下所示:

我想做的也看起来像这样 tensorflow classification .

import random
import torch.optim
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from torch import nn
from sklearn.datasets import make_circles

def accuracy_fn(y_true, y_pred):
    correct = torch.eq(y_true, y_pred).sum().item() # torch.eq() calculates where two tensors are equal
    acc = (correct / len(y_pred)) * 100
    return acc


n_samples = 1000

X, y = make_circles(n_samples, noise=0.03, random_state=42)

X = torch.from_numpy(X).type(torch.float)
y = torch.from_numpy(y).type(torch.float).unsqueeze(1)

X_train, X_test, y_train, y_test = train_test_split(X,
                                                    y,
                                                    test_size=0.2, # 20% test, 80% train
                                                    random_state=42)

model = nn.Sequential(
    nn.Linear(2, 8),
    nn.ReLU(),
    nn.Linear(8, 8),
    nn.ReLU(),
    nn.Linear(8, 1),
)

print(model.state_dict())

criterion = nn.BCEWithLogitsLoss()

optimizer = torch.optim.SGD(model.parameters(), lr=0.1)

Loss = []
epochs = 500

for epoch in range(epochs):
    y_logit = model(X_train)
    loss = criterion(y_logit, y_train)
    y_pred = torch.round(torch.sigmoid(y_logit))
    acc = accuracy_fn(y_true=y_train,
                      y_pred=y_pred)

    if epoch % 100 == 0:
        Loss.append(loss.item())

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    model.eval()
    with torch.inference_mode():
        # 1. Forward pass
        test_logits = model(X_test)
        test_pred = torch.round(torch.sigmoid(test_logits))  # logits -> prediction probabilities -> prediction labels
        # 2. Calcuate loss and accuracy
        test_loss = criterion(test_logits, y_test)
        test_acc = accuracy_fn(y_true=y_test,
                               y_pred=test_pred)

    # Print out what's happening
    if epoch % 100 == 0:
        print(
            f"Epoch: {epoch} | Loss: {loss:.5f}, Accuracy: {acc:.2f}% | Test Loss: {test_loss:.5f}, Test Accuracy: {test_acc:.2f}%")

我不太确定我在这里做错了什么。我尝试了各种学习率和时期，但我的最终模型通常具有 50% 的准确率，并且损失永远不会下降。有时，如果我重新运行程序足够多次，准确度会达到 50 秒甚至 60 秒。我有点引用this link为了了解我应该获得的数据，作者的准确度在 70 多岁左右。他的损失也大幅缩小。

我没有完全复制他的代码，但我也看了其他类似的例子，我的代码似乎并没有太大的不同。

Answer 1

for epoch in range(epochs):
################################################################
    optimizer.zero_grad()
################################################################
    y_logit = model(X_train)
    loss = criterion(y_logit, y_train)
    y_pred = torch.round(torch.sigmoid(y_logit))
    acc = accuracy_fn(y_true=y_train,
                      y_pred=y_pred)

    if epoch % 100 == 0:
        Loss.append(loss.item())

    
    loss.backward()
    optimizer.step()

    model.eval()
    with torch.inference_mode():
        # 1. Forward pass
        test_logits = model(X_test)
        test_pred = torch.round(torch.sigmoid(test_logits))  # logits -> prediction probabilities -> prediction labels
        # 2. Calcuate loss and accuracy
        test_loss = criterion(test_logits, y_test)
        test_acc = accuracy_fn(y_true=y_test,
                               y_pred=test_pred)

    # Print out what's happening
    if epoch % 100 == 0:
        print(
            f"Epoch: {epoch} | Loss: {loss:.5f}, Accuracy: {acc:.2f}% | Test Loss: {test_loss:.5f}, Test Accuracy: {test_acc:.2f}%")

你必须将zero_grad放在循环的顶部。当您将 Optimizer.zero_grad() 放在训练循环的底部时，在计算当前批处理的梯度之前，您不会清除梯度。这可能会导致梯度在多个批处理中累积，从而导致模型参数的更新不正确且不一致。