c++ - 将 shared_ptr 用于 private_key 时出现段错误

Question

Updates

[X] I discovered this happen when TLS::credentials creds is declared on global scope but if I declare it outside seg fault won't happen.

I need it to be global because it helps with caching certificates and that multiple threads can use certificates created by other threads without spending time on creating new certificates.

[X] I further reduced code from 200 lines approx. to 100 lines

我正在使用 Botan 创建一个 TLS 应用程序，但我的应用程序在应用程序结束时因 seg 错误而崩溃。
我试图用 Valgrind 调试它，但它无处可去。
这是来自 Valgrind 的堆栈跟踪，

==3841967== Invalid write of size 8
==3841967==    at 0x4842964: memset (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3841967==    by 0x566A82F: Botan::deallocate_memory(void*, unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x55E1A4D: ??? (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==    by 0x40EC29: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:730)
==3841967==    by 0x41112D: std::__shared_ptr<Botan::RSA_Public_Data const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1169)
==3841967==    by 0x411107: std::shared_ptr<Botan::RSA_Public_Data const>::~shared_ptr() (shared_ptr.h:103)
==3841967==    by 0x41109D: Botan::RSA_PublicKey::~RSA_PublicKey() (rsa.h:25)
==3841967==    by 0x410FC1: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410DC5: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410E8A: std::_Sp_counted_ptr<Botan::RSA_PrivateKey*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:377)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==  Address 0x9419080 is not stack'd, malloc'd or (recently) free'd
==3841967== 
==3841967== 
==3841967== Process terminating with default action of signal 11 (SIGSEGV)
==3841967==  Access not within mapped region at address 0x9419080
==3841967==    at 0x4842964: memset (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3841967==    by 0x566A82F: Botan::deallocate_memory(void*, unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x55E1A4D: ??? (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==    by 0x40EC29: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:730)
==3841967==    by 0x41112D: std::__shared_ptr<Botan::RSA_Public_Data const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1169)
==3841967==    by 0x411107: std::shared_ptr<Botan::RSA_Public_Data const>::~shared_ptr() (shared_ptr.h:103)
==3841967==    by 0x41109D: Botan::RSA_PublicKey::~RSA_PublicKey() (rsa.h:25)
==3841967==    by 0x410FC1: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410DC5: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410E8A: std::_Sp_counted_ptr<Botan::RSA_PrivateKey*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:377)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==  If you believe this happened as a result of a stack
==3841967==  overflow in your program's main thread (unlikely but
==3841967==  possible), you can try to increase the size of the
==3841967==  main thread stack using the --main-stacksize= flag.
==3841967==  The main thread stack size used in this run was 8388608.
==3841967== 
==3841967== HEAP SUMMARY:
==3841967==     in use at exit: 149,626 bytes in 1,143 blocks
==3841967==   total heap usage: 211,782 allocs, 210,639 frees, 90,582,963 bytes allocated
==3841967== 
==3841967== LEAK SUMMARY:
==3841967==    definitely lost: 0 bytes in 0 blocks
==3841967==    indirectly lost: 0 bytes in 0 blocks
==3841967==      possibly lost: 1,352 bytes in 18 blocks
==3841967==    still reachable: 148,274 bytes in 1,125 blocks
==3841967==                       of which reachable via heuristic:
==3841967==                         newarray           : 1,536 bytes in 16 blocks
==3841967==         suppressed: 0 bytes in 0 blocks
==3841967== Rerun with --leak-check=full to see details of leaked memory
==3841967== 
==3841967== For lists of detected and suppressed errors, rerun with: -s
==3841967== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

您可以通过发出将 Botan 克隆到您的机器中，

git clone https://github.com/randombit/botan.git

然后按照他们的official website 的说明进行操作。构建和安装它。
您将需要创建一个根证书颁发机构以与应用程序一起使用，为此您必须在您的机器上安装 OpenSSL。
创建一个名为 testApplication 的文件夹和 cd进去。
然后使用 Bash，发出以下一系列命令来创建根 CA，

# Generate private key
openssl genrsa -des3 -out myCA.key 2048
# Generate root certificate
openssl req -x509 -new -nodes -key myCA.key -sha256 -days 825 -out myCA.pem
# Convert to Botan Format
openssl pkcs8 -topk8 -in myCA.key > myCAKey.pkcs8.pem

请使用thisispassword作为密码。
在你的机器上安装 clang 编译器，然后你可以编译源文件如下，

clang++ example.cpp -o example  -Wthread-safety -Wall -Wextra -g -std=c++17 -pthread -lssl -lcrypto -lbotan-2 --I/usr/include/botan-2

例子.cpp

#include <iostream>
#include <string>
#include <vector>
#include <fstream>
#include <sstream>
#include <botan/tls_server.h>
#include <botan/tls_callbacks.h>
#include <botan/tls_session_manager.h>
#include <botan/tls_policy.h>
#include <botan/auto_rng.h>
#include <botan/certstor.h>
#include <botan/pk_keys.h>
#include <botan/pkcs10.h>
#include <botan/pkcs8.h>
#include <botan/x509self.h>
#include <botan/x509path.h>
#include <botan/x509_ca.h>
#include <botan/x509_ext.h>
#include <botan/pk_algs.h>
#include <botan/ber_dec.h>
#include <botan/der_enc.h>
#include <botan/oids.h>
#include <botan/rsa.h>

namespace TLS
{
    typedef std::chrono::duration<int, std::ratio<31556926>> years;

    class credentials : public Botan::Credentials_Manager
    {
    private:
        struct certificate
        {
            std::vector<Botan::X509_Certificate> certs;
            std::shared_ptr<Botan::Private_Key> key;
        };

        std::vector<certificate> creds;
        std::vector<std::shared_ptr<Botan::Certificate_Store>> store;

    public:
        void createCert(std::string hostname)
        {
            /**
             * Initialize Root CA
            **/

            Botan::AutoSeeded_RNG rng;

            const Botan::X509_Certificate rootCert("myCA.pem");

            std::ifstream rootCertPrivateKeyFile("myCAKey.pkcs8.pem");

            Botan::DataSource_Stream rootCertPrivateKeyStream(rootCertPrivateKeyFile);

            std::unique_ptr<Botan::Private_Key> rootCertPrivateKey = Botan::PKCS8::load_key(rootCertPrivateKeyStream, "thisispassword");

            Botan::X509_CA rootCA(rootCert, *rootCertPrivateKey, "SHA-256", rng);

            /**
            * Generate a Cert & Sign with Root CA
            **/

            Botan::X509_Cert_Options opts;
            std::shared_ptr<Botan::Private_Key> serverPrivateKeyShared(new Botan::RSA_PrivateKey(rng, 4096));
            Botan::RSA_PrivateKey* serverPrivateKey = (Botan::RSA_PrivateKey*)serverPrivateKeyShared.get();

            opts.common_name = hostname;
            opts.country = "US";

            auto now = std::chrono::system_clock::now();

            Botan::X509_Time todayDate(now);
            Botan::X509_Time expireDate(now + years(1));

            Botan::PKCS10_Request req = Botan::X509::create_cert_req(opts, *serverPrivateKey, "SHA-256", rng);

            auto serverCert = rootCA.sign_request(req, rng, todayDate, expireDate);

            /**
             * Load Cert to In-Memory Database
            **/

            certificate cert;

            cert.certs.push_back(serverCert);
            cert.key = serverPrivateKeyShared;

            creds.push_back(cert);
        }
    };
}; // namespace TLS

TLS::credentials globalCreds;

int main() {
    globalCreds.createCert("www.google.com");

    std::cout << "End" << "\n";

    return 0;
}

这是 Valgrind 引用的 Botan Lib 中的函数，

void deallocate_memory(void* p, size_t elems, size_t elem_size)
   {
   if(p == nullptr)
      return;

   secure_scrub_memory(p, elems * elem_size);

#if defined(BOTAN_HAS_LOCKING_ALLOCATOR)
   if(mlock_allocator::instance().deallocate(p, elems, elem_size))
      return;
#endif

   std::free(p);
   }

Answer 1

Botan的作者回复我说

The problem is the globally defined object.

问题是 mlock 池是在第一次使用时创建的单例，然后在 main 返回后的某个时间销毁。首先创建您的对象。它分配内存。这将导致创建池。破坏发生后进先出。因此，首先销毁池。然后您的对象被销毁，并尝试触摸已经未映射的内存(将其归零)。
解决方法，

创建 Botan::Allocator_Initializer 对象以强制初始化
在你的对象被创建之前(因此池一直存在到你的
对象已被破坏)

禁用locking_allocator模块

将环境变量 BOTAN_MLOCK_POOL_SIZE 设置为 0

没有全局变量

原则上，锁定分配器而不是 munmaping 内存，只是将它归零，并让它在进程退出时由操作系统取消映射。这可能仍然会破坏不变量，但不会那么严重。它还导致 valgrind 报告令人讨厌的泄漏。
我认为因为它是直接映射而不是通过 malloc 映射的，所以 valgrind 不会跟踪它。