python - tensorflow 中不同范围的子网权重共享

Question

使用 tensorflow，我试图在不同变量范围内共享来自相同网络的相同权重以节省内存。但是，似乎没有简单的方法可以做到这一点。我准备了一个小代码示例，以在较小的规模上说明我想对较大的子网执行的操作:

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    with tf.variable_scope("super_scope_one"):
        scope1 = tf.variable_scope("sub_scope_one")
        with scope1:
            number_one = tf.get_variable("number_one", shape=[1],
                                         initializer=tf.ones_initializer)
    with tf.variable_scope("super_scope_two"):
        with tf.variable_scope("sub_scope_one", reuse=True) as scope2:
            # Here is the problem.
            # scope1.reuse_variables() # this crashes too if reuse=None.
            number_one = tf.get_variable("number_one", shape=[1])
        with tf.variable_scope("sub_scope_two"):
            number_two = tf.get_variable("number_two", shape=[1],
                                         initializer=tf.ones_initializer)
        number_three = number_one + number_two

    init_op = tf.global_variables_initializer()

with tf.Session(graph=graph):
    init_op.run()
    print(number_three.eval())

有没有办法在不删除的情况下共享两个子作用域中的变量 上面的范围？如果不是，是否有充分的理由说明这将是一个坏主意？

Answer 1

您可以在 "super_scope_one" 中简单地定义一次 number_one 并在 "super_scope_two" 中使用它。

不同作用域的两个变量可以一起使用。见下文:

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    with tf.variable_scope("super_scope_one"):
        scope1 = tf.variable_scope("sub_scope_one")
        with scope1:
            number_one = tf.get_variable("number_one", shape=[1],
                                         initializer=tf.ones_initializer)
    with tf.variable_scope("super_scope_two"):
        with tf.variable_scope("sub_scope_two"):
            number_two = tf.get_variable("number_two", shape=[1],
                                         initializer=tf.ones_initializer)
        number_three = number_one + number_two

    init_op = tf.global_variables_initializer()

    with tf.Session(graph=graph):
        init_op.run()
        print(number_three.eval())

返回 [2]