python - 具有 Ctypes 的独立 CDLL 库实例

Question

我正在尝试使用 ctypes 并两次加载相同的编译 Fortran 库，这样我就有了它的两个独立实例，这样库包含的任何模块变量都不会存储在相同的内存位置。描述的一般解决方案(例如，此处:https://mail.python.org/pipermail/python-list/2010-May/575368.html)是提供库的完整路径，而不仅仅是其名称。但是，我无法让它像这样工作。这是一个演示问题的最小工作示例:

测试.f90:

module test
    use iso_c_binding, only: c_int
    implicit none
    integer :: n
contains
    integer(c_int) function get() bind(c, name='get')
        get = n
    end function get

    subroutine set(new_n) bind(c, name='set')
        integer(c_int), intent(in) :: new_n
        n = new_n
    end subroutine set
end module test

测试.py:

import os
from ctypes import cdll, c_int, byref

if __name__ == '__main__':
    lib1 = cdll.LoadLibrary(os.path.abspath('test.so'))
    lib2 = cdll.LoadLibrary(os.path.abspath('test.so'))

    lib1.set(byref(c_int(0)))
    lib2.set(byref(c_int(1)))

    print(lib1.get())

Fortran 库使用以下命令编译:

gfortran -shared -fPIC -o test.so test.f90

当我运行 python test.py 时，我得到 1 作为输出，而我想得到 0。有谁知道如何进行这项工作？

Answer 1

[Python.Docs]: ctypes - A foreign function library for Python使用 DlOpen 加载库(在 Nix 上)。根据[Man7]: DLOPEN(3) :

If the same shared object is loaded again with dlopen(), the same object handle is returned. The dynamic linker maintains reference counts for object handles, so a dynamically loaded shared object is not deallocated until dlclose() has been called on it as many times as dlopen() has succeeded on it.

我准备了一个小例子。
在继续之前，检查[SO]: C function called from Python via ctypes returns incorrect value (@CristiFati's answer)有关使用 CTypes 时经常遇到的错误(也在问题中)的详细信息。

dll00.c:

#if defined(_WIN32)
#  define DLL00_EXPORT_API __declspec(dllexport)
#else
#  define DLL00_EXPORT_API
#endif


static int val = -1;


DLL00_EXPORT_API int get()
{
    return val;
}


DLL00_EXPORT_API void set(int i)
{
    val = i;
}

code00.py:

#!/usr/bin/env python

import ctypes as cts
import os
import shutil
import sys


def get_dll_funcs(dll):
    get_func = dll.get
    get_func.argtypes = ()
    get_func.restype = cts.c_int
    set_func = dll.set
    set_func.argtypes = (cts.c_int,)
    set_func.restype = None
    return get_func, set_func


def main(*argv):
    dll00 = "./dll00.so"
    dll01 = "./dll01.so"
    dir00 = "dir00"
    os.makedirs(dir00, exist_ok=True)
    shutil.copy(dll00, dir00)
    shutil.copy(dll00, dll01)

    dll_names = [dll00, os.path.abspath(dll00), os.path.join(dir00, dll00), dll01]
    dlls = [cts.CDLL(item) for item in dll_names]

    for idx, dll in enumerate(dlls):
        print("Item {:d} ({:s}) was loaded at {:08X}".format(idx, dll_names[idx], dll._handle))
        set_func = get_dll_funcs(dll)[1]
        set_func(idx * 10)

    for idx, dll in enumerate(dlls):
        get_func = get_dll_funcs(dll)[0]
        print("Item {:d} get() returned {: d}".format(idx, get_func()))


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

输出:

[cfati@cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q054243176]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[064bit prompt]>ls
code00.py  dll00.c
[064bit prompt]> gcc -fPIC -shared -o dll00.so dll00.c
[064bit prompt]> ls
code00.py  dll00.c  dll00.so
[064bit prompt]> python ./code.py
Python 3.5.2 (default, Nov 12 2018, 13:43:14) [GCC 5.4.0 20160609] 064bit on linux

Item 0 (./dll00.so) was loaded at 02437A80
Item 1 (/mnt/e/Work/Dev/StackOverflow/q054243176/dll00.so) was loaded at 02437A80
Item 2 (dir00/./dll00.so) was loaded at 02438690
Item 3 (./dll01.so) was loaded at 02438EF0
Item 0 get() returned  10
Item 1 get() returned  10
Item 2 get() returned  20
Item 3 get() returned  30

从输出中可以看出(还要注意 _handle 属性)，尝试多次加载相同的 .dll(通过其路径)(相同的行为在 Win 上):

• 如果位于同一路径(即使指定不同)，实际上不会再次加载它，它只会增加其RefCount

• 如果名称或位置不同，则重新加载

简而言之，回答您的问题:只需将其复制到不同的名称下并加载即可。