python - 为什么tensorflow中的 `tf.nn.nce_loss`无法在GPU上运行？

Question

代码 tensorflow/examples/tutorials/word2vec/word2vec_basic.py 有注释 # Ops and variables pinned to the CPU because missing GPU implementation。我还发现操作tf.nn.nce_loss不能用GPU实现。那么为什么 tf.nn.nce_loss 不能在 GPU 上运行呢？

我已经使用 log_device_placement 查看了 MUST BE CPU 操作。这是结果:

nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0

但是，我仍然不知道是哪个特殊操作导致了这个问题。有没有GPU不能实现的特殊Ops？还是tensorflow本身没有实现？

Answer 1

您也可以使用 gpu 加速器执行此示例。
查看 tensorflow 支持的设备 https://www.tensorflow.org/guide/using_gpu#supported_devices
您必须添加以下代码行

with tf.device('/gpu:0'):

并删除 #with tf.device('/cpu:0'):

基本上在这一行用gpu替换cpu: https://github.com/tensorflow/tensorflow/blob/bd965956535b1026a0769d04c3f6419d020f5f9c/tensorflow/examples/tutorials/word2vec/word2vec_basic.py#L189

希望这对您有所帮助。