我正在尝试使用 pytorch 添加自定义 C++ 扩展。
我正在学习 https://pytorch.org/tutorials/advanced/cpp_extension.html 上的教程
我创建了两个文件。
project/
main.py
sigmoid.cpp
主.py
from torch.utils.cpp_extension import load
lltm_cpp = load(name=‘sigmoid’, sources=[‘sigmoid.cpp’], verbose=True)
乙状结肠.cpp
#include <torch/extension.h>
#include <iostream>
torch::Tensor d_sigmoid(torch::Tensor z) {
auto s = torch::sigmoid(z);
return (1 - s) * s;
}
PYBIND11_MODULE('sigmoid', m) {
m.def('d_sigmoid', &d_sigmoid, 'sigmoid');
}
当我运行 main.py 时出现以下错误
error: pasting “pybind11_init_” and “‘sigmoid’” does not give a valid preprocessing token
如何修复错误?
完整的错误报告
Using /tmp/torch_extensions as PyTorch extensions root…
Emitting ninja build file /tmp/torch_extensions/sigmoid/build.ninja…
Building extension module sigmoid…
[1/2] c++ -MMD -MF sigmoid.o.d -DTORCH_EXTENSION_NAME=sigmoid -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /home/wickrama/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/wickrama/projects/torch_cpp_ext/sigmoid.cpp -o sigmoid.o
FAILED: sigmoid.o
c++ -MMD -MF sigmoid.o.d -DTORCH_EXTENSION_NAME=sigmoid -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/TH -isystem /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/THC -isystem /home/wickrama/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/wickrama/projects/torch_cpp_ext/sigmoid.cpp -o sigmoid.o
In file included from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pytypes.h:12:0,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/cast.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/attr.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pybind11.h:44,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/utils/pybind.h:6,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/python.h:12,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/extension.h:6,
from /home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:1:
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: error: pasting “pybind11_init_” and “‘sigmoid’” does not give a valid preprocessing token
PYBIND11_MODULE(‘sigmoid’, m) {
^
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: warning: character constant too long for its type
PYBIND11_MODULE(‘sigmoid’, m) {
^~~~~~~~~~~~~~
In file included from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pytypes.h:12:0,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/cast.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/attr.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pybind11.h:44,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/utils/pybind.h:6,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/python.h:12,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/extension.h:6,
from /home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:1:
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: error: pasting “PyInit_” and “‘sigmoid’” does not give a valid preprocessing token
PYBIND11_MODULE(‘sigmoid’, m) {
^
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: warning: character constant too long for its type
PYBIND11_MODULE(‘sigmoid’, m) {
^~~~~~~
In file included from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pytypes.h:12:0,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/cast.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/attr.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pybind11.h:44,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/utils/pybind.h:6,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/python.h:12,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/extension.h:6,
from /home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:1:
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: error: pasting “pybind11_init_” and “‘sigmoid’” does not give a valid preprocessing token
PYBIND11_MODULE(‘sigmoid’, m) {
^
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: warning: character constant too long for its type
PYBIND11_MODULE(‘sigmoid’, m) {
^~~~~~~~~~~~~~
In file included from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pytypes.h:12:0,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/cast.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/attr.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pybind11.h:44,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/utils/pybind.h:6,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/python.h:12,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/extension.h:6,
from /home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:1:
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: error: pasting “pybind11_init_” and “‘sigmoid’” does not give a valid preprocessing token
PYBIND11_MODULE(‘sigmoid’, m) {
^
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:1: warning: character constant too long for its type
PYBIND11_MODULE(‘sigmoid’, m) {
^~~~~~~~~~~~~~
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:11:9: warning: character constant too long for its type
m.def(‘d_sigmoid’, &d_sigmoid, ‘sigmoid’);
^~~~~~~~~~~
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:11:34: warning: character constant too long for its type
m.def(‘d_sigmoid’, &d_sigmoid, ‘sigmoid’);
^~~~~~~~~
In file included from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pytypes.h:12:0,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/cast.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/attr.h:13,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/pybind11/pybind11.h:44,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/utils/pybind.h:6,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/python.h:12,
from /home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/include/torch/extension.h:6,
from /home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:1:
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:17: error: expected initializer before ‘\x6d6f6964’
PYBIND11_MODULE(‘sigmoid’, m) {
^
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:17: error: expected initializer before ‘\x6d6f6964’
PYBIND11_MODULE(‘sigmoid’, m) {
^
/home/wickrama/projects/torch_cpp_ext/sigmoid.cpp:10:17: error: expected initializer before ‘\x6d6f6964’
PYBIND11_MODULE(‘sigmoid’, m) {
^
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File “/home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py”, line 949, in _build_extension_module
check=True)
File “/home/wickrama/anaconda3/envs/pytorch/lib/python3.6/subprocess.py”, line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘ninja’, ‘-v’]’ returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/wickrama/projects/torch_cpp_ext/main.py”, line 3, in
lltm_cpp = load(name=‘sigmoid’, sources=[‘sigmoid.cpp’], verbose=True)
File “/home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py”, line 644, in load
is_python_module)
File “/home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py”, line 813, in _jit_compile
with_cuda=with_cuda)
File “/home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py”, line 866, in _write_ninja_file_and_build
_build_extension_module(name, build_directory, verbose)
File “/home/wickrama/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/cpp_extension.py”, line 962, in _build_extension_module
raise RuntimeError(message)
RuntimeError: Error building extension ‘sigmoid’
最佳答案
我的猜测:与 Python 不同,单引号 ('
) 与 C++ 中的双引号 ("
) 不等价。前者用于构造 char
字面量,所以它们只能包裹单个字符,后者用于构造字符串字面量(严格来说 const char *
字面量)。所以尝试替换
PYBIND11_MODULE('sigmoid', m) {
m.def('d_sigmoid', &d_sigmoid, 'sigmoid');
}
与
PYBIND11_MODULE("sigmoid", m) {
m.def("d_sigmoid", &d_sigmoid, "sigmoid");
}
看看是否有帮助。
关于c++ - 自定义 C++ 扩展错误 : pasting “pybind11_init_” and “‘sigmoid’ ” does not give a valid preprocessing token,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/58186907/