python - 无法打开 pickle 的 Sagemaker XGBoost 模型

标签 python pickle xgboost amazon-sagemaker

我正在尝试打开我在 AWS Sagemaker 中创建的 pickle XGBoost 模型,以查看模型中的特征重要性。我正在尝试遵循 this post 中的答案.但是,我收到如下所示的错误。当我尝试调用 Booster.save_model 时, 我收到一条错误消息 'Estimator' object has no attribute 'save_model' .我该如何解决这个问题?

# Build initial model
sess = sagemaker.Session()
s3_input_train = sagemaker.s3_input(s3_data='s3://{}/{}/train/'.format(bucket, prefix), content_type='csv')
xgb_cont = get_image_uri(region, 'xgboost', repo_version='0.90-1')
xgb = sagemaker.estimator.Estimator(xgb_cont, role, train_instance_count=1, train_instance_type='ml.m4.4xlarge',
                                    output_path='s3://{}/{}'.format(bucket, prefix), sagemaker_session=sess)
xgb.set_hyperparameters(eval_metric='rmse', objective='reg:squarederror', num_round=100)
ts = strftime("%Y-%m-%d-%H-%M-%S", gmtime())
xgb_name = 'xgb-initial-' + ts
xgb.set_hyperparameters(eta=0.1, alpha=0.5, max_depth=10)
xgb.fit({'train': s3_input_train}, job_name=xgb_name)

# Load model to get feature importances
model_path = 's3://{}/{}//output/model.tar.gz'.format(bucket, prefix, xgb_name)
fs = s3fs.S3FileSystem()
with fs.open(model_path, 'rb') as f:
    with tarfile.open(fileobj=f, mode='r') as tar_f:
        with tar_f.extractfile('xgboost-model') as extracted_f:
            model = pickle.load(extracted_f)

XGBoostError: [19:16:42] /workspace/src/learner.cc:682: Check failed: header == serialisation_header_: 

  If you are loading a serialized model (like pickle in Python) generated by older
  XGBoost, please export the model by calling `Booster.save_model` from that version
  first, then load it back in current version.  There's a simple script for helping
  the process. See:

    https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html

  for reference to the script, and more details about differences between saving model and
  serializing.

最佳答案

您在笔记本中使用的是哪个版本的 XGBoost? XGBoost 1.0 中的模型格式已更改。见 https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html .简短版本:如果您在笔记本中使用 1.0,则无法加载 pickle 模型。

这是一个在脚本模式下使用 XGBoost 的工作示例(比内置算法灵活得多):

  • https://gitlab.com/juliensimon/dlnotebooks/-/blob/master/sagemaker/09-XGBoost-script-mode.ipynb
  • https://gitlab.com/juliensimon/dlnotebooks/-/blob/master/sagemaker/xgb.py
  • 关于python - 无法打开 pickle 的 Sagemaker XGBoost 模型,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/60643094/

    相关文章:

    python - 动态类定义的 pickle

    python - CatBoost:我们是否过度拟合?

    python - xgboost + pipenv = </3

    python - 在 Google Colaboratory 上使用 GPU 的 XGBoost

    python - 使用 RPY2 将 psycopg2 的结果映射到 R 的数据帧中

    Python pickle : Unclear "AttributeError: can' t set attribute"

    python - 使用 Python Flask AJAX 动态更新图像

    python - python的pp模块上的PicklingError

    python - Pytorch:如何找到 2D 张量的每一行中第一个非零元素的索引?

    python - 从列表中创建随机分组