c - DPDK 函数在从 Rust 调用时与从 C 调用时具有不同的输出

Question

我正在编写一个与英特尔 DPDK 接口(interface)的 Rust 程序，但我遇到了一个我不太了解的问题。我目前可以在 Rust 程序中执行 DPDK 库中的函数，但它与用 C 编写的相同程序的结果不同。

Rust 程序输出 -

~/Dev/rust_eal_init_test$ sudo target/debug/rust_eal_init_test 
EAL: Detected 8 lcore(s)
EAL: No free hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
~/Dev/rust_eal_init_test$

C程序输出-

~/Dev/c_eal_init_test$ sudo build/c_eal_init_test 
EAL: Detected 8 lcore(s)
EAL: No free hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:00:19.0 on NUMA socket -1
EAL:   probe driver: 8086:153a rte_em_pmd
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL:   probe driver: 8086:1533 rte_igb_pmd
~/Dev/c_eal_init_test$

DPDK 已被编译为我在 Rust 中链接到的静态库。下面是 Rust 和 C 程序的代码。

Rust main.rs -

extern crate libc;

use std::env;
use std::process::exit;
use std::ffi::CString;
use libc::{c_int, c_char,};

extern {
    pub fn rte_eal_init(argc: c_int, argv: *mut *mut c_char) -> c_int;
}

// librte_eal
pub fn dpdk_rte_eal_init(argc: i32, argv: Vec<String>) -> i32 {
    let mut args: Vec<*mut c_char> = argv.iter().map(|x| CString::new(x.clone()).unwrap().into_raw()).collect();
    let retc: c_int = unsafe { rte_eal_init(argc as c_int, args.as_mut_ptr()) };
    let ret: i32 = retc as i32;
    ret
}

fn main() {
    let args: Vec<String> = env::args().collect();

    let ret: i32 = dpdk_rte_eal_init(args.len() as i32, args);
    if ret < 0 {
        exit(ret);
    }
}

C main.c -

#include <stdio.h>
#include <string.h>
#include <rte_eal.h>
#include <rte_debug.h>

int
main(int argc, char **argv)
{
    int ret;

    ret = rte_eal_init(argc, argv);
    if (ret < 0)
        rte_panic("Cannot init EAL\n");

    return 0;
}

我通过查看 DPDK 源代码并添加打印语句来查看不同之处，从而进行了一些调查。 rte_eal_init() 函数位于 eal.c 中并调用各种其他初始化函数。额外输出的触发器来自 eal_common_dev.c 中另一个名为 rte_eal_dev_init() 的函数。

rte_eal_dev_init() -

int
rte_eal_dev_init(void)
{
    struct rte_devargs *devargs;
    struct rte_driver *driver;

    /*
     * Note that the dev_driver_list is populated here
     * from calls made to rte_eal_driver_register from constructor functions
     * embedded into PMD modules via the PMD_REGISTER_DRIVER macro
     */

    /* call the init function for each virtual device */
    TAILQ_FOREACH(devargs, &devargs_list, next) {

        if (devargs->type != RTE_DEVTYPE_VIRTUAL)
            continue;

        if (rte_eal_vdev_init(devargs->virt.drv_name,
                    devargs->args)) {
            RTE_LOG(ERR, EAL, "failed to initialize %s device\n",
                    devargs->virt.drv_name);
            return -1;
        }
    }

    /* Once the vdevs are initalized, start calling all the pdev drivers */
    TAILQ_FOREACH(driver, &dev_driver_list, next) {
        if (driver->type != PMD_PDEV)
            continue;
        /* PDEV drivers don't get passed any parameters */
        driver->init(NULL, NULL);
    }
   return 0;
}

我发现 Rust 程序进入 rte_eal_dev_init() 函数，但从未进入 TAILQ_FOREACH() 宏的循环。如果我像这样直接在宏的上方和下方添加打印语句 -

printf("Test before TAILQ_FOREACH\n");  
TAILQ_FOREACH(driver, &dev_driver_list, next) {
    printf("Test in TAILQ_FOREACH\n");
    if (driver->type != PMD_PDEV)
        continue;
    /* PDEV drivers don't get passed any parameters */
    driver->init(NULL, NULL);
}

这在 Rust 中为我提供了以下输出 -

~/Dev/rust_eal_init_test$ sudo target/debug/rust_eal_init_test 
EAL: Detected 8 lcore(s)
EAL: No free hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
Test before TAILQ_FOREACH
~/Dev/rust_eal_init_test$

C 中的以下输出 -

~/Dev/c_eal_init_test$ sudo build/c_eal_init_test 
EAL: Detected 8 lcore(s)
EAL: No free hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
Test before TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
PMD: bnxt_rte_pmd_init() called for (null)
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
EAL: PCI device 0000:00:19.0 on NUMA socket -1
EAL:   probe driver: 8086:153a rte_em_pmd
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL:   probe driver: 8086:1533 rte_igb_pmd
~/Dev/c_eal_init_test$

我发现 TAILQ_FOREACH() 是在 sys/queue.h 中定义的。据我了解，链接到静态库不应导致库动态链接到其他内容时出现问题。这是正确的吗？

Answer 1

我最初认为我没有正确链接到为 sys/queue.h 提供信息的东西。在尝试更好地理解链接的同时，我更改了我的 build.rs 以从此链接到 DPDK -

fn main() {
    println!("cargo:rustc-link-lib=static=rte_eal");
    println!("cargo:rustc-link-search=native=<Path to RTE_SDK>/x86_64-native-linuxapp-gcc/lib");
}

对此——

fn main() {
    println!("cargo:rustc-link-lib=static=dpdk");
    println!("cargo:rustc-link-search=native=<Path to RTE_SDK>/x86_64-native-linuxapp-gcc/lib");
}

Rust 现在链接到所有已编译的 DPDK 库，而不仅仅是调用初始函数的 rte_eal。现在，当我执行我的 Rust 程序时，我得到与 C 程序相同的输出 -

~/Dev/rust_eal_init_test$ sudo target/debug/rust_eal_init_test 
EAL: Detected 8 lcore(s)
EAL: No free hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
Test before TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
PMD: bnxt_rte_pmd_init() called for (null)
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
Test in TAILQ_FOREACH
EAL: PCI device 0000:00:19.0 on NUMA socket -1
EAL:   probe driver: 8086:153a rte_em_pmd
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL:   probe driver: 8086:1533 rte_igb_pmd
~/Dev/rust_eal_init_test$

我没想到其中一个静态库需要另一个包含在可执行文件中才能运行。我还假设，如果我将 DPDK 编译为共享库，我就不会遇到这个问题。

也感谢 Shepmaster，虽然没有直接回答我的问题，但迫使我对其进行了简化，这使得以更可控的方式解决我的问题变得更加容易。