linux - 使用 libhugetlbfs 分配页面

Question

我正在编写一个需要使用大页面的程序。

我正在尝试为 ubuntu64 使用 libhugetlbfs 库。 我试图用安装的库的名称替换 LD_LIBRARY 变量，并将我的程序链接到它。

在我的代码中，我定期调用 malloc，如何确保我得到的是大页面而不是常规页面？

Answer 1

首先，您无需链接到 libhugetlbfs 即可使用大页面。 事实上，您不必修改代码，甚至不必重新链接二进制文件。

根据这个HOWTO

Blockquote

Using hugepages for malloc() (morecore)

This feature allows an existing (dynamically linked) binary executable to use hugepages for all its malloc() calls. To run a program using the automatic hugepage malloc() feature, you must set several environment variables:

1. Set LD_PRELOAD=libhugetlbfs.so This tells the dynamic linker to load the libhugetlbfs shared library, even though the program wasn't originally linked against it.

   Note: If the program is linked against libhugetlbfs, preloading the library may lead to application crashes. You should skip this step in that case.

2. Set LD_LIBRARY_PATH to the directory containing libhugetlbfs.so This is only necessary if you haven't installed libhugetlbfs.so to a system default path. If you set LD_LIBRARY_PATH, make sure the directory referenced contains the right version of the library (32-bit or 64-bit) as appropriate to the binary you want to run.

3. Set HUGETLB_MORECORE This enables the hugepage malloc() feature, instructing libhugetlbfs to override libc's normal morecore() function with a hugepage version and use it for malloc(). From this point all malloc()s should come from hugepage memory until it runs out. This option can be specified in two ways:

   To use the default huge page size: HUGETLB_MORECORE=yes

   To use a specific huge page size: HUGETLB_MORECORE=

您需要做的就是(如果您还没有链接到 libhugetlbfs): 运行你的程序作为

LD_PRELOAD=libhugetlbfs.so LD_LIBRARY_PATH=LIBHUGEFS_DIR:$LD_LIBRARY_PATH HUGETLB_MORECORE=yes your_binary

或者只是

LD_LIBRARY_PATH=LIBHUGEFS_DIR:$LD_LIBRARY_PATH HUGETLB_MORECORE=yes your_binary

如果您的二进制文件链接到 libhugetlbfs

其中 LIBHUGEFS_DIR 需要替换为 libhugetlbfs.so 所在的目录，your_binary 需要替换为您的程序名称。

将您的系统配置为支持大页面也很重要。这可以通过多种方式完成，最简单的方式是:

echo "2000000" > /proc/sys/vm/nr_overcommit_hugepages

有关详细信息，请参阅链接的 HOWTO。

重新链接到 libhugefs 还有其他好处 - 它允许将二进制/共享对象中的某些段重新映射到大页面。

请注意，如果您在代码中直接使用 mmap 调用或使用自定义分配器，问题会更加复杂。

另外请考虑只使用 Transparent Huge Pages这是默认启用的，是一种更简单的方法(但容易出现物理内存碎片问题)。