简介
根据烤宽面条文档: “该层应插入到线性变换(例如 DenseLayer 或 Conv2DLayer)及其非线性之间。方便的函数batch_norm() 修改现有层以在其非线性之前插入批量归一化。”
不过千层面也有实用功能:
lasagne.layers.batch_norm
但是,由于我这边的实现,我无法使用该功能。
我的问题是:我应该如何以及在哪里添加 BatchNormLayer?
class lasagne.layers.BatchNormLayer(incoming, axes='auto', epsilon=1e-4, alpha=0.1, beta=lasagne.init.Constant(0), gamma=lasagne.init.Constant(1), mean=lasagne.init.Constant(0), inv_std=lasagne.init.Constant(1), **kwargs)
我可以在卷积层之后添加它吗?或者我应该在 maxpool 之后添加? 我是否必须手动消除图层的偏差?
使用的方法 我只是这样使用过:
try:
import lasagne
import theano
import theano.tensor as T
input_var = T.tensor4('inputs')
target_var = T.fmatrix('targets')
network = lasagne.layers.InputLayer(shape=(None, 1, height, width), input_var=input_var)
from lasagne.layers import BatchNormLayer
network = BatchNormLayer(network,
axes='auto',
epsilon=1e-4,
alpha=0.1,
beta=lasagne.init.Constant(0),
gamma=lasagne.init.Constant(1),
mean=lasagne.init.Constant(0),
inv_std=lasagne.init.Constant(1))
network = lasagne.layers.Conv2DLayer(
network, num_filters=60, filter_size=(3, 3), stride=1, pad=2,
nonlinearity=lasagne.nonlinearities.rectify,
W=lasagne.init.GlorotUniform())
network = lasagne.layers.Conv2DLayer(
network, num_filters=60, filter_size=(3, 3), stride=1, pad=1,
nonlinearity=lasagne.nonlinearities.rectify,
W=lasagne.init.GlorotUniform())
network = lasagne.layers.MaxPool2DLayer(incoming=network, pool_size=(2, 2), stride=None, pad=(0, 0),
ignore_border=True)
network = lasagne.layers.DenseLayer(
lasagne.layers.dropout(network, p=0.5),
num_units=32,
nonlinearity=lasagne.nonlinearities.rectify)
network = lasagne.layers.DenseLayer(
lasagne.layers.dropout(network, p=0.5),
num_units=1,
nonlinearity=lasagne.nonlinearities.sigmoid)
return network, input_var, target_var
引用文献:
https://github.com/Lasagne/Lasagne/blob/master/lasagne/layers/normalization.py#L120-L320
http://lasagne.readthedocs.io/en/latest/modules/layers/normalization.html
最佳答案
如果不使用batch_norm
:
- BatchNormLayer 应添加在密集层或卷积层之后、非线性之前。
- Maxpool 是一种非线性下采样,它将保留该层上的最高值。如果您在卷积/密集层之后添加 BatchNormLayer,采样值将被标准化。
- 如果不使用
batch_norm
,请手动删除偏差,因为它是多余的。
请测试下面的代码,并告诉我们它是否适合您想要实现的目标。 如果不起作用,您可以尝试调整 batch_norm code .
import lasagne
import theano
import theano.tensor as T
from lasagne.layers import batch_norm
input_var = T.tensor4('inputs')
target_var = T.fmatrix('targets')
network = lasagne.layers.InputLayer(shape=(None, 1, height, width), input_var=input_var)
network = lasagne.layers.Conv2DLayer(
network, num_filters=60, filter_size=(3, 3), stride=1, pad=2,
nonlinearity=lasagne.nonlinearities.rectify,
W=lasagne.init.GlorotUniform())
network = batch_norm(network)
network = lasagne.layers.Conv2DLayer(
network, num_filters=60, filter_size=(3, 3), stride=1, pad=1,
nonlinearity=lasagne.nonlinearities.rectify,
W=lasagne.init.GlorotUniform())
network = batch_norm(network)
network = lasagne.layers.MaxPool2DLayer(incoming=network, pool_size=(2, 2), stride=None, pad=(0, 0),
ignore_border=True)
network = lasagne.layers.DenseLayer(
lasagne.layers.dropout(network, p=0.5),
num_units=32,
nonlinearity=lasagne.nonlinearities.rectify)
network = batch_norm(network)
network = lasagne.layers.DenseLayer(
lasagne.layers.dropout(network, p=0.5),
num_units=1,
nonlinearity=lasagne.nonlinearities.sigmoid)
network = batch_norm(network)
当获取参数为您的更新方法创建图表时,请记住将 trainable 设置为 True:
params = lasagne.layers.get_all_params(l_out, trainable=True)
updates = lasagne.updates.adadelta($YOUR_LOSS_HERE, params)`
关于neural-network - 如何正确添加和使用BatchNormLayer?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/37624279/