python - 段错误 - python C 扩展中的核心转储

Question

我正在为 python 编写一个 c 扩展。正如您在下面看到的，代码的目的是计算两个 vector 的欧氏距离。 第一个参数 n 是 vector 的维度， 第二个，第三个参数是float的两个列表。

我在 python 中这样调用函数:

import cutil
cutil.c_euclidean_dist(2,[1.0,1,0],[0,0])

它运行良好，返回正确的结果。 但是如果我这样做超过 100 次(维度是 1*1000)，它会导致段错误 - 核心转储:

#!/usr/bin/env python
#coding:utf-8
import cutil
import science
import time
a = []
b = []
d = 0.0 
for x in range(2500):
    a.append([float(i+x) for i in range(1000)])
    b.append([float(i-x) for i in range(1000)])

t1 = time.time()
for x in range(500):
    d += cutil.c_euclidean_dist(1000,a[x],b[x])
print time.time() - t1
print d

C代码在这里:

#include <python2.7/Python.h>
#include <math.h>

static PyObject* cutil_euclidean_dist(PyObject* self, PyObject* args) {
    PyObject *seq_a, *seq_b;
    int n;
    float * array_a,* array_b;
    PyObject *item;

    PyArg_ParseTuple(args,"iOO", &n , &seq_a, &seq_b);
    if (!PySequence_Check(seq_a) || !PySequence_Check(seq_b)) {
       PyErr_SetString(PyExc_TypeError, "expected sequence");
       return NULL;
    }

    array_a =(float *)malloc(sizeof(float)*n);
    array_b =(float *)malloc(sizeof(float)*n);  

    if (NULL == array_a || NULL == array_b){
        PyErr_SetString(PyExc_TypeError, "malloc failed!");
        Py_DECREF(seq_a);
        Py_DECREF(seq_b);
        return NULL;
    }

    int i;
    for(i=0;i<n;i++){
        item = PySequence_GetItem(seq_a,i);

        if (!PyFloat_Check(item)) {
            free(array_a);  /* free up the memory before leaving */
            free(array_b);
            Py_DECREF(seq_a);
            Py_DECREF(seq_b);
            Py_DECREF(item);
            PyErr_SetString(PyExc_TypeError, "expected sequence of float");
            return NULL;
        }
        array_a[i] = PyFloat_AsDouble(item);

        Py_DECREF(item);

        item = PySequence_GetItem(seq_b,i);
        if(!PyFloat_Check(item)) {
            free(array_a);
            free(array_b);
            Py_DECREF(seq_a);
            Py_DECREF(seq_b);
            Py_DECREF(item);
            PyErr_SetString(PyExc_TypeError, "expected sequence of float"); 
            return NULL;
        }
        array_b[i] = PyFloat_AsDouble(item);
        Py_DECREF(item);
    }

    double sum = 0;
    for(i=0;i<n;i++){
        double delta = array_a[i] - array_b[i];
        sum += delta * delta;
    }

    free(array_a);
    free(array_b);
    Py_DECREF(seq_a);
    Py_DECREF(seq_b);

    return Py_BuildValue("d",sqrt(sum));
}

static PyMethodDef cutil_methods[] = {
    {"c_euclidean_dist",(PyCFunction)cutil_euclidean_dist,METH_VARARGS,NULL},
    {NULL,NULL,0,NULL}
};

PyMODINIT_FUNC initcutil(void) {
    Py_InitModule3("cutil", cutil_methods, "liurui's c extension for python");
} 

错误信息:

segmentation fault - core dump:

c扩展被编译成cutil.so，不知道怎么看dump。 但是我多次查看我的C代码，没有发现任何问题..

可能是内存问题？

应该是很简单的一段C代码，有什么问题吗？ 我需要你的帮助~非常感谢!

这是 gdb/usr/bin/python2.7 ./core 的结果:

root@ubuntu:/home/rrg/workspace/opencvTest/test# gdb /usr/bin/python2.7 ./core 
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/python2.7...Reading symbols from /usr/lib/debug//usr/bin/python2.7...done.
done.

warning: core file may not match specified executable file.
[New LWP 13787]
[New LWP 13789]
[New LWP 13790]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `python py.py'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000000005398b3 in list_dealloc.16846 (op=0x7f688b2faa28) at ../Objects/listobject.c:309
309 ../Objects/listobject.c: no such file or directory
#0  0x00000000005398b3 in list_dealloc.16846 (op=0x7f688b2faa28) at ../Objects/listobject.c:309
#1  0x00000000004fdb96 in insertdict_by_entry (value=<optimized out>, ep=0x1777fa8, hash=<optimized out>, key='b', mp=0x7f68a8dbb168) at ../Objects/dictobject.c:519
#2  insertdict (value=<optimized out>, hash=<optimized out>, key='b', mp=0x7f68a8dbb168) at ../Objects/dictobject.c:556
#3  dict_set_item_by_hash_or_entry (value=<optimized out>, ep=0x0, hash=<optimized out>, key='b', 
    op={'a': None, 'x': None, 'c': None, 'b': None, 'd': <float at remote 0x4480b30>, '__builtins__': <module at remote 0x7f68a8de6b08>, 'science': <module at remote 0x7f68a8ce4088>, '__package__': None, 'i': 999, 'cutil': <module at remote 0x7f68a8cdfbb0>, 'time': <module at remote 0x7f68a640ea28>, '__name__': '__main__', 't1': <float at remote 0xd012708>, '__doc__': None}) at ../Objects/dictobject.c:765
#4  PyDict_SetItem (
    op=op@entry={'a': None, 'x': None, 'c': None, 'b': None, 'd': <float at remote 0x4480b30>, '__builtins__': <module at remote 0x7f68a8de6b08>, 'science': <module at remote 0x7f68a8ce4088>, '__package__': None, 'i': 999, 'cutil': <module at remote 0x7f68a8cdfbb0>, 'time': <module at remote 0x7f68a640ea28>, '__name__': '__main__', 't1': <float at remote 0xd012708>, '__doc__': None}, key=key@entry='b', 
    value=<optimized out>) at ../Objects/dictobject.c:818
#5  0x000000000055a9e1 in _PyModule_Clear (m=<optimized out>) at ../Objects/moduleobject.c:139
#6  0x00000000004f2ad4 in PyImport_Cleanup () at ../Python/import.c:473
#7  0x000000000042fa89 in Py_Finalize () at ../Python/pythonrun.c:459
#8  0x000000000046ac10 in Py_Main (argc=<optimized out>, argv=0x7fff3958d058) at ../Modules/main.c:665
#9  0x00007f68a8665ec5 in __libc_start_main (main=0x46ac3f <main>, argc=2, argv=0x7fff3958d058, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff3958d048)
    at libc-start.c:287
#10 0x000000000057497e in _start ()

编辑: 我在最后一个返回附近评论最后 2 句话:

Py_DECREF(seq_a);
Py_DECREF(seq_b);  

然后它似乎运行良好。我觉得非常非常奇怪... 这两句的目的是释放(或释放)两个pyobject，为什么没有我认为必要的两句它也能很好地工作？

Answer 1

The c-extension is compiled to cutil.so, I do not know how to see the dump.

为了解决这个问题，我将引用 GNU Radio's GDB/Python debugging mini-tutorial :

Luckily, there's a feature called core dumping that allows the state of your program to be stored in a file, allowing later analysis. Usually, that feature is disabled; you can enable it by:

ulimit -c unlimited

Note that this only works for processes spawned from the shell that you used ulimit in. What happens here is that the maximum size of a core dump is set to unlimited (the original value is 0 in most cases).

Now, the core dump file lays in the current execution directory of the program that crashed. In our case, that's build/python/, but since all core dumps should have a name like core., we can use a little find magic:

marcus> find -type f -cmin 5 -name 'core.[0-9]*'

./build/python/core.22608

because that will find all _f_iles, changed/created within the last _5 min_utes, having a name that matches.

Using GDB with a core dump

having found build/python/core.22608, we can now launch GDB:

gdb programname coredump

i.e.

gdb /usr/bin/python2 build/python/core.22608

A lot of information might scroll by.

At the end, you're greeted by the GDB prompt:

(gdb) 

Getting a backtrace

Typically, you'd just get a backtrace (or shorter, bt). A backtrace is simply the hierarchy of functions that were called.

 (gdb)bt

[...] 跳过，

Frame #2 and following definitely look like they're part of the Python implementation -- that sounds bad, because GDB doesn't itself know how to debug python, but luckily, there's an extension to do that. So we can try to use py-bt:

(gdb) py-bt

If we get a undefined command error, we must stop here and make sure that the python development package is installed (python-devel on Redhatoids, python2.7-dev on Debianoids); for some systems, you should append the content of /usr/share/doc/{python-devel,python2.7-dev}/gdbinit[.gz] to your ~/.gdbinit, and re-start gdb.

The output of py-bt now states clearly which python lines correspond to which stack frame (skipping those stack frames that are hidden to python, because they are in external libraries or python-implementation routines)

...